ABSTRACTS OF ARTICLES OF THE JOURNAL "INFORMATION TECHNOLOGIES".
No. 9. Vol. 27. 2021

DOI: 10.17587/it.27.461-469

D. Yu. Nartsev, Master Student, e-mail: dennartsev@gmail.com, Moscow Institute of Physics and Technology, Dolgoprudny, 141701, Moscow Region, Russian Federation,
A. N. Gneushev, Ph.D., Researcher, e-mail: gneushev@ccas.ru, Federal Research Center "Computer Science and Control" of RAS, Moscow, 119333, Russian Federation

Adam Optimization Method Modifications Comparison in the Regression Models Parameters Evaluation Tasks

There is considered a problem of optimization methods comparing for the neural network regression task. Various optimization methods, such as stochastic gradient descent with momentum (SGDM), Adam and its modifications, AdamW and RAdam, were considered. To compare optimization method two regression task were formulated. Both tasks are connected with the preprocessing subtasks in the field of image analysis. The first considered task was the filtering blurred eye images on which confident recognition cannot be achieved. The training samples were generated by Gaussian blurring of the images. The blurring degree was estimated. The test and training sample for the assessment problem was formed on the basis of the BATH and CASSIA eye image databases. The second task was aligning faces in assessment image in face recognition systems. The training samples were generated by rotating face images, and rotation angle was estimated. To solve these tasks the direct estimation of parameters by solving the image regression problem by training neural network models is proposed. The adequate accuracy was acquired with all considered optimization methods for both tasks. Modifications of Adam algorithm show better results than original method. Both AdamW and RAdam methods reduced the error twice in comparison with Adam. The modification of the RAdam algorithm proposed in the work reduced the error by more than 1.5 times in comparison with the model trained by the original algorithm.
Keywords: optimization methods, neural networks, regression task, face rotation estimation, blur degree estimation, image quality assessment, SGD, Adam, AdamW, RAdam, machine learning

P. 461–469

To the contents