DOI: 10.17587/prin.16.134-142

Variability of the Wide Learning Neural Network Learning Algorithm

A. S. Yakovlev, Undergraduate Student, yakovlevsasha42@gmail.com, E. V. Shayakberov, Postgraduate Student, eduard.shayakberov@mail.ru, V. M. Giniyatullin, Associate Professor, fentazer@mail.ru, Ufa State Petroleum Technological University, Ufa, 450064, Russian Federation

Corresponding author: Vakhit M. Giniyatullin, Associate Professor, Ufa State Petroleum Technological University, Ufa, 450064, Russian Federation E-mail: fentazer@mail.ru

There are a large number of datasets on the publicly available kaggle resource, from which five datasets and their corresponding structures of artificial neural networks were selected. After training neural networks, the work of the neurons of the first hidden layers was reproduced in spreadsheets. A significant number of useless neurons were found (20—60 %). A neuron is called useless if the scalar products of all instances of the training sample are less than zero. The relu activation function converts negative values to zero, therefore, such a neuron does not contribute to the work of the neural network. Thus, it is necessary to learn how to find and remove useless neurons in trained networks. The following describes the principle of functioning of a reproducible neural network learning algorithm. To do this, the instance belonging to a certain class is replaced by belonging to one of the categories of ternary logic (true, zero, false) and a linearly separable subsample is formed. The properties of the Wide Learning algorithm are considered, the code of the first version of the software implementation is posted in the Github repository under the Apache 2.0 license https://github.com/brinkinvision/wideLearning. The scalar product of the input vector by the weights of the neuron projects the n-dimensional input space onto a one-dimensional digital line. The ternary activation threshold function on this line marks with true (false) categories a part of instances belonging to a certain class. The hypothesis of the normality of the distribution of scalar products of labeled instances was tested. In most cases, the hypothesis was justified, and sometimes it was possible to prove normality after additional manipulations. Once a two-modal distribution was discovered, the emulation of two-modality demonstrates the adequacy of the described methodology..

Keywords: artificial neural networks, dataset, activation function, ambiguity, training algorithm, reproducibility, scalar multiplication, normal distribution, p-value, nonnumeric parameters

pp. 134—142

For citation:
Yakovlev A. S., Shayakberov E. V., Giniyatullin V. M. Variability of the Wide Learning Neural Network Learning Algorithm, Programmnaya Ingeneria, 2025, vol. 16, no. 3, pp. 134—142. DOI: 10.17587/ prin.16.134-142 (in Russian).

The work was carried out with the financial support of the Foundation for the Promotion of Small Forms of Enterprises in the Scientific and Technical field (the project "WideLearning Open Library for searching for the architecture of an artificial neural network using discretized convolutional layers and complex-valued algebras" Agreement No.19 ГУКодИИС12-Б7/76725 dated 05/23/2022).

References:

1. Brief A. Introduction to Neural Networks, available at: https://www.dkriesel.com/en/science/neural_networks (date of ac­cess 17.08.2024).
2. Nikolenco S., Kadurin A., Arkhangelskaya E. Deep Learning, St. Petersburg, Piter, 2019, 480 p. (in Russian).
3. Indian Liver Patient Dataset, available at: https://www.kaggle.com/datasets/fatemehmehrparvar/liver-disorders (date of access 17.08.2024).
4. Ionosphere, available at: https://archive.ics.uci.edu/data-set/52/ionosphere (date of access 17.08.2024).
5. Sleep Health and Lifestyle Dataset, available at: https://www.kaggle.com/datasets/uom190346a/sleep-health-and-lifestyle-dataset (date of access 17.08.2024).
6. Pumpkin Seeds Dataset, available at: https://www.kaggle.com/datasets/muratkokludataset/pumpkin-seeds-dataset (date of access 17.08.2024).
7. Crystal System Properties for Li-ion batteries, available at: https://www.kaggle.com/code/divyansh22/neural-network-for-li-ion-classification/input (date of access 17.08.2024).
8. Classification of liver disorder, available at: https://www.kaggle.com/code/pantitsiripornanukul/nn-sequential-classification-of-liver-disorder#Load-Data (date of access 17.08.2024).
9. Binary Classification, available at: https://www.kaggle.com/code/ryanholbrook/binary-classification/tutorial (date of access 17.08.2024).
10. Sleep Disorder Prediction Using Neural Network, available at: https://www.kaggle.com/code/kyroyen/sleep-disorder-prediction-using-neural-network (date of access 17.08.2024).
11. Artificial Neural Network without any ML Libraries, available at: https://www.kaggle.com/code/sharif8410/artificial-neural-network-with-out-any-ml-libraries#3.5-%7CANN-Model (date of access 17.08.2024).
12. Neural Network for Li-Ion Classification, available at: https://www.kaggle.com/code/divyansh22/neural-network-for-li-ion-classification/notebook (date of access 17.08.2024).
13. Kingma D. P., Salimans T., Welling M. Variational dropout and the local reparameterization trick, Advances in Neural Information Processing Systems, 2015, vol. 28, pp. 2575—2583.
14. Jinjoo L. AI Is About to Boost Power Bills—Who'll Take Heat for That? The Wall Street Journal, 12.08.2024, available at: https://www.wsj.com/business/energy-oil/ai-is-about-to-boost-pow-er-billswholl-take-heat-for-that-c527f27b (date of access 17.08.2024).
15. McCulloch W. S., Pitts W. A Logical Calculus of the Ideas Immanent in Nervous Activity, Bull. Mathematical Biophysics, 1943, vol. 5, pp. 115—133.
16. Sedgwick R. Fundamental Algorithms in C++. Part 5. Graph Algorithms, Moscow, DiaSoft, 2002, 452 p. (in Russian).
17. Rosenblatt F. Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms, Moscow, Mir, 1965, 427 p. (in Russian).
18. Giniyatullin V. M., Skrypin A. R., Taisin R. R. Linear Separability of Ternary Logic Functions, Actual problems of science and technology — 2015. Proceedings of the VIII International Scientific and Practical Conference of young scientists, 2015, pp. 116—119. (in Russian).
19. Library for Creating Simplified Neural Network Structures, available at: https://github.com/brinkinvision/wideLearning (date of access 17.08.2024).
20. Cristianini N., Shawe-Taylor J. An Introduction to Support Vector Machines and Other Kernel-based Learning Methods, Cambridge University Press, 2000, 189 p.
21. Nivorozhkina L. I., Morozova Z. A. Probability Theory and Mathematical Statistics, Moscow, EKSMO, 2008, 316 p. (in Russian).
22. Budenny S. A., Lazarev V. D., Zakharchenko N. N. et al. eco2AI: Carbon Footprint Control of Machine Learning Models as a First Step Towards Sustainable Artificial Intelligence, Reports of the Russian Academy of Sciences. Mathematics, Informatics, Control Processes, 2022, vol. 508, pp. 134—145. DOI: 10.31857/S2686954322070232 (in Russian)