DOI: 10.17587/prin.17.486-496

Server-Side Component for Centralized Log Collection and Storage in Distributed Computing Systems

A. A. Okonov, Engineer, k151201@yandex.ru, K. S. Zaitsev, D. Sc. (Eng.), Professor, kszajtsev@mephi.ru, M. E. Dunaev, Lecturer, max.dunaev@mail.ru, National Research Nuclear University "MEPhI", Moscow, 115409, Russian Federation

Corresponding author: Konstantin S. Zaitsev, D. Sc. (Eng.), Professor, National Research Nuclear University "MEPhI", Moscow, 115409, Russian Federation, E-mail: KSZajtsev@mephi.ru

The objective of this study is to improve the efficiency and reliability of centralized collection and analytical availability of diagnostic data in distributed information systems. The study reviews local Syslog-based approaches, broker-based solutions, architectural styles, and relevant technology stacks. A centralized event logging system was developed that combines lightweight local log collection, asynchronous delivery, and scalable centralized storage. Compared with local Syslog solutions, it provides centralized accumulation and analytical access to logs; compared with broker-based platforms, it reduces deployment and operational complexity for medium- and low-load environments. The system is implemented in Go using the Gorilla framework. A sharded ClickHouse-based storage architecture with Round Robin load balancing is proposed. Experimental results confirm stable event collection, analytical availability of logs, and a balanced trade-off between throughput, reliability, and infrastructure complexity.

Keywords: server application, Golang, Syslog, ClickHouse, sharding, load balancing, Docker

pp. 486—496

For citation:
Okonov A. A., Zaitsev K. S., Dunaev M. E. Server-Side Component for Centralized Log Collection and Storage in Distributed Computing Systems, Programmnaya Ingeneria, 2026, vol. 17, no. 9, pp. 486—496. DOI: 10.17587/prin.17.486-496 (in Russian).

References:

1. Rosenberg J. Data Discoverability in Science Gateways at Scale using Elasticsearch Cluster Architecture, Practice and Experience in Advanced Research Computing, 2022, article 41. DOI: 10.1145/3491418.3535170.
2. Sankar P., George D. E., Aparna Sankar N. S. Social media monitoring using ELK Stack, 2022 IEEE International Conference on Signal Processing, Informatics, Communication and Energy Systems (SPICES), 2022, vol. 1, pp. 231—235. DOI: 10.1109/SPIC-ES52834.2022.9774273.
3. Tokar D., Morozova O., Kharchenko V. The IoT Applications Productivity: Data Management Model and ELK Tool Based Monitor­ing and Research, 2022 IEEE 16th International Conference on Advanced Trends in Radioelectronics, Telecommunications and Computer Engineering (TCSET), 2022, pp. 162—167. DOI: 10.1109/TCSET55632.2022.9766930.
4. Sukhanov M. Y. Apache Kafka as a Means of Ensuring Reliable Communication in Microservice Systems Based on the Spring Framework, Young Scientist: Collection of Articles of the 8th International Scientific and Practical Conference, Penza, July 5, 2024, Penza, Nauka i Prosveshchenie, 2024, pp. 59—65 (in Russian).
5. Kalandarev A. ClickHouse in Statistics Collection Systems, Sistemnyj administrator, 2017, no. 3 (172), pp. 56—59 (in Russian).
6. Gribanov I. V. Network Monitoring System Using syslog, Studencheskij vestnik, 2021, no. 41-5 (186), pp. 34—40 (in Russian).
7. Chernyshev K. D., Yashonkov A. V. Methods of Load Balancing on Information Systems Servers, Development of Science and the Global Community: Proceedings of the 13th International Scientific Research Competition, Anapa, August 26, 2024, Anapa, Nauchno-issledovatel'skij centr ekonomicheskih i social'nyh processov v Yuzhnom Federal'nom okruge, 2024, pp. 48—57 (in Russian).
8. Osokin D. A., Maslakov A. P. Development of a Distributed SIEM System for Log Collection and Analysis in an Organization's Computer Network, Modern Technologies in the Theory and Practice of Programming: Proceedings of the Scientific-Practical Conference of Students, Postgraduates and Young Scientists, Saint Petersburg, April 24—25, 2024, Saint Petersburg: Peter the Great St. Petersburg Polytechnic University, 2024, pp. 81—83 (in Russian).
9. Denisov V. V., Samoilov N. K. Advantages of Hexagonal Architecture in a Microservice Application, Proceedings of Young Scientists of the Faculty of Computer Science of VSU, Voronezh, VELBORN, 2022, vol. 2, pp. 257—263 (in Russian).
10. Sergushkin V. V., Bodrov O. A. Features of Implementing a Multithreaded Application on Data-Transmission Hardware, 7th In­ternational Scientific and Technical Conference "K. E. Tsiolkovsky — 160 Years Since Birth. Astronautics. Radioelectronics. Geoinformatics", Conference Abstracts, Ryazan, October 4—6, 2017, Ryazan, Ryazan State Radio Engineering University, 2017, pp. 288—290 (in Russian).
11. Alexandria G. Eclarin E. Round Robin Rumble: A Comparative Analysis of Round Robin-based Hybrid Scheduling Algo­rithms, International Journal of Science and Research, 2024, vol. 13, no. 9, pp. 386—389. DOI: 10.21275/sr24904185322.
12. Parshikov P. A., Parshikova G. V. Dynamic Sharding Using a Table Function, Priority Directions of Innovative Activity in Industry: Collection of Scientific Articles Based on the Proceedings of the Fourth International Scientific Conference, Kazan, April 29—30, 2020, Kazan, Konvert, 2020, vol. 2, pp. 101—103 (in Russian).
13. Nada S., Nuredin A. Microservices vs. Monolithic Architectures [The Differential Structure between two Architectures], MINAR International Journal of AppliedSciences and Technology, 2022, vol. 4, pp. 484—490. DOI: 10.47832/2717-8234.12.47 (in Russian).