ABSTRACTS OF ARTICLES OF THE JOURNAL "INFORMATION TECHNOLOGIES".
No. 7. Vol. 31. 2025

DOI: 10.17587/it.31.364-369

À. À. Vasilev, Middle Engineer, LLC "Alphachip", Moscow, À. I. Kapitanov, Associate Professor,
SPINTech Institute, National Research University "MIET", Moscow

Application of Integer Tables for Quantisation of Activation Functions of Neural Networks

Received on 01.04.2025
Accepted on 22.04.2025

The paper considers the problem of efficient hardware implementation of nonlinear activation functions of neural networks under low-bit computing conditions. Standard activations, such as sigmoid and hyperbolic tangent, require resource-intensive floating-point operations, which limits their use on microcontrollers, FPGAs and other peripheral platforms. As a solution, an approach based on precomputed integer substitution tables (LUTs) is proposed to reduce computational complexity and power consumption. Using the example of the SiLU activation function widely used in popular object detection networks (e.g., YOLO), the quantisation procedure is demonstrated, the principles of constructing and using LUTs are formulated, and a practical algorithm for computing activations using them is described.
Keywords: quantisation, integrated circuits, neural networks, convolutional neural networks, hardware implementation of neural networks

P. 364-369

Full text on eLIBRARY

References

1. Romanov A. Y., Stempkovsky A. L., Lariushkin I. V., Novoselov G. E., Solovyev R. A., Starykh V. A., Romanova I. I., Telpukhov D. V., Mkrtchan I. A. Analysis of posit and bfloat arithmetic of real numbers for machine learning, IEEE Access, 2021, no. 9, pp. 82318—82324.
2. Zhao X., Wang Y., Cai X., Liu C., Zhang L. Linear symmetric quantization of neural networks for low-precision integer hardware, International Conference on Learning Representations, 2020, April.
3. Krishnamoorthi R. Quantizing deep convolutional networks for efficient inference: A whitepaper, arXiv preprint arXiv: 1806.08342, 2018.
4. Solovyev R. A., Kalinin A. A., Kustov A. G., Telpukhov D. V., Ruhlov V. S. FPGA implementation of convolutional neural networks with fixed-point calculations, CoRR, 2018, Available: https://openreview.net/forum?id=ktBJlHahoT.
5. Solovyev R. A., Telpukhov D. V., Romanova I. I., Kustov A. G., Mkrtchan I. A. Real-time Object Detection with FPGA Using CenterNet, 2021 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (ElConRus), 2021, January, pp. 2029—2034.
6. Nagel M., Fournarakis M., Amjad R. A., Bondarenko Y., Van Baalen M., Blankevoort T. A white paper on neural network quantization, arXiv preprint arXiv:2106.08295, 2021.
7. Yang J., Shen X., Xing J., Tian X., Li H., Deng B., Huang J., Hua X. S. Quantization networks, Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 7308—7316.
8. Gholami A., Kim S., Dong Z., Yao Z., Mahoney M. W., Keutzer K. A survey of quantization methods for efficient neural network inference, Low-power computer vision. Chapman and Hall/ CRC, 2022, pp. 291—326.
9. Hubara I., Courbariaux M., Soudry D., El-Yaniv R., Ben-gio Y. Quantized neural networks: Training neural networks with low precision weights and activations, Journal of machine learning research, 2018, vol. 18, no. 187, pp. 1—30.
10. Hagiescu A., Langhammer M., Pasca B., Colangelo P., Thong J., Ilkhani N. Bfloat MLP training accelerator for FPGAs, In 2019 International Conference on ReConFigurable Computing and FPGAs (ReConFig), 2019, December, pp. 1—5.
11. Zhai H. Z., Du J. W., Ai Y. H., Hu T. J. Edge Deployment of Deep Networks for Visual Object Detection: A Review, IEEE Sensors Journal, 2024, Available: https://ieeexplore.ieee.org/abstract/document/10786287

To the contents