ABSTRACTS OF ARTICLES OF THE JOURNAL "INFORMATION TECHNOLOGIES".
No. 3. Vol. 25. 2019

DOI: 10.17587/it.25.174-178

N. T. Abdullayev, Ph. D., Associate Professor, e-mail: a.namik46@mail.ru, Azerbaijan Technical University,
K. Sh. Ismayilova, Ph. D., Associate Professor, e-mail: Is_kamalya@yahoo.com, Azerbaijan State University of Oil and Industry

The Choice of the Iteration Step in the Process of Learning a Neural Network Using Relaxation Algorithms

In information sources, there are a lot of materials devoted to algorithms for optimization and implementation of these algorithms for neural networks. However, the issues of algorithmic support in selecting the iteration step, for learning neural networks are quite relevant. The choice in favor of gradient methods is justified by the fact that, as a rule, in learning problems, the learning criterion can be expressed in the form of a differentiable function of the weights of the neural network.
Nevertheless, the uncertainty of the choice of the method of instruction is preserved. In automated systems of neural network programming, one should strive to reduce the uncertainty that is inherent in these technologies. The uncertainty in the choice of the learning algorithm is to some extent eliminated in the proposed adaptive learning method. The algorithm for selecting astep consists of 5 levels (steps).

Step 1. At the beginning of the whole algorithm the initial step is set. Parameters of the algorithm must satisfy the following inequalities — j₁:j₁ > 1, j₂: 0 <j₂< 1.
Step 2. After finding the next direction of motion, the step from the previous iteration is taken as the initial approximation. Step 3. After that, the value of the error functional E(w_k + n_{k — 1}p_k) is calculated.
Step 4. If this value is less than the previous one E(w_k), then step umnazhaem to j₁ and again calculate the value of the error until the error begins to increase. The penultimate minimum point is fixed and a new direction is computed.
Step 5. If E(w_k + n_{k — 1}p_k) >E(w_k), the step is multiplied by j₂and the step decreases until the error becomes less than E(w_k). The resulting step is fixed and a new direction is calculated.
This algorithm is effective in real work and easy to implement. In addition, it is convenient to apply this algorithm together with the heavy ball, Partan and even BFGS algorithms, which are widely used in real problems. The table of methods for selecting a step and their implementation is useful to users when choosing the iteration step in experiments.
Keywords: neural network, relaxation algorithms, iteration step, gradient methods, efficiency, prostate implementation, optimization, learning

P. 174–178

To the contents