DOI: 10.17587/prin.15.611-622

Evaluation of Scalability of Distributed Systems based on Algorithms for Iterative Task Execution with Coordination via Key-Value Storage

T. E. Zeynally, Postgraduate Student, Assistant, z.teymur.e@gmail.com, D. G. Demidov, Ph.D., Associate Professor, Dean of IT Faculty, d.g.demidov@mospolytech.ru, Moscow Polytechnic University, Moscow, 107023, Russian Federation

Corresponding author: Teymur E. Zeynally, Postgraduate Student, Assistant, Moscow Polytechnic University, Moscow, 107023, Russian Federation, E-mail: z.teymur.e@gmail.com

This paper presents the results of research aimed at evaluating the scalability of distributed fault-tolerant systems for iterative task execution. The study focuses on peer-to-peer distributed systems capable of executing tasks iteratively according to a schedule, where the nodes are coordinated through a key-value store. The operation of such systems is based on Recurrent Worker Service (RWS) algorithms. The primary focus is on studying the dependency of metrics such as iteration accuracy and resource utilization on the number of system nodes, including both application and storage nodes. The experimental environment is deployed in Yandex Cloud infrastructure and consists of four servers, one of which is low-performance and three are high-performance. Tests were conducted under three scenarios: application node scalability, storage node scalability, and simultaneous scalability of both node categories. During the tests, performance differences between the two modes of RWS algorithms were studied. The results obtained in this work can be used in the design and optimization of distributed systems that require high performance and fault tolerance.

Keywords: distributed information systems, fault tolerance, system scaling, key-value storage, cloud infrastructure, iterative task execution, peer-to-peer distributed systems, iteration accuracy, payload

pp. 611—622

For citation:
Zeynally T. E., Demidov D. G. Evaluation of Scalability of Distributed Systems based on Algorithms for Iterative Task Execution with Coordination via Key-Value Storage, Programmnaya Ingeneria, 2024, vol. 15, no. 12, pp. 611—622. DOI: 10.17587/prin.15.611-622. (in Russian).

References:

1. Mykhailyshyn H., Pasyeka N., Sheketa V. et al. Designing network computing systems for intensive processing of information flows of data, Lecture Notes on Data Engineering and Communications Technologies, 2021, vol. 48, pp. 391—422. DOI: 10.1007/978-3-030-43070-2_18.
2. Blinowski G., Ojdowska A., Przybylek A. Monolithic vs. Microservice Architecture: A Performance and Scalability Evaluation, IEEE Access, 2022, vol. 10, pp. 20357—20374. DOI: 10.1109/ACCESS.2022.3152803.
3. Distler T., Bahn C., Bessani A. et al. Extensible distributed coordination, Proceedings of the 10th European Conference on Computer Systems, EuroSys 2015, 2015, pp. 143—158. DOI: 10.1145/2741948.2741954.
4. Zeynally T., Demidov D. Fault Tolerance of Distributed Worker Processes in Corporate Information Systems and Technologies, Communications in Computer and Information Science, 2022, pp. 32—41. DOI: 10.1007/978-3-031-21340-3_4.
5. Etcd, available at: https://etcd.io/ (date of access 06.10.2024).
6. Zeynally T., Demidov D., Dimitrov L. Prioritization of Distributed Worker Processes Based on Etcd Locks, Communications in Computer and Information Science, 2022, pp. 93—103. DOI: 10.1007/978-3-031-21340-3_9.
7. Zeynally T., Demidov D. Evaluation of the Efficiency of Fault Tolerance Algorithms for Distributed Peer-To-Peer Worker Processes Connected Through a Key-Value Store, Communications in Computer and Information Science, 2023, pp. 87—98. DOI: 10.1007/978-3-031-31353-0_8.
8. Zeynally T. E. Comparative analysis of.NET-based solutions for iterative task execution in distributed fault-tolerant systems, International Journal of Open Information Technologies, 2024, vol. 12, no. 6, pp. 120—127 (in Russian).
9. Larsson L., Tarneberg W., Klein C. et al. Impact of etcd deployment on Kubernetes, Istio, and application performance, Software — Practice and Experience, 2020, vol. 50, no. 10, pp. 1986—2007. DOI: 10.1002/spe.2885.
10. Jeffery A., Howard H., Mortier R. Rearchitecting Kubernetes for the Edge, EdgeSys 2021, Proceedings of the 4th International Workshop on Edge Systems, Analytics and Networking, Part of Euro-Sys 2021, 2021, pp. 7—12. DOI: 10.1145/3434770.3459730.
11. Leesatapornwongsa T., Stuardo C. A., Suminto R. O. et al. Scalability Bugs: When 100-Node Testing is Not Enough, Proceedings of the Workshop on Hot Topics in Operating Systems — HOTOS,2017, pp. 24—29. DOI: 10.1145/3102980.3102985.
12. Avritzer A., Ferme V., Janes A. et al. Scalability Assessment of Microservice Architecture Deployment Configurations: A Domain-based Approach Leveraging Operational Profiles and Load Tests, Journal of Systems and Software, 2020, vol. 165, article 110564. DOI: 10.1016/j.jss.2020.110564.
13. van Dongen G., van den Poel D. Influencing Factors in the Scalability of Distributed Stream Processing Jobs, IEEE Access, 2021, vol. 9, pp. 109413—109431. DOI: 10.1109/ACCESS.2021.3102645.
14. Docker overview, available at: https://docs.docker.com/get-started/overview/ (date of access 08.01.2024).
15. Telegraf Docker Input Plugin, available at: https://github.com/influxdata/telegraf/blob/master/plugins/inputs/docker/ README.md (date of access 08.08.2024).
16. InfluxDB, available at: https://docs.influxdata.com/influx-db/v1/concepts/key_concepts/ (date of access 08.01.2024).
17. DateTimeOffset Ticks, available at: https://learn.microsoft.com/en-us/dotnet/api/system.datetimeoffset.ticks?view=net-7.0 (date of access 08.08.2024).
18. Gibadullin R. F. Thread-safe Control Calls in Enriched Client Applications, Programmnye sistemy i vychislitel'nye metody, 2022, no. 4, pp. 45—47. DOI: 10.7256/2454-0714.2022.4.39029 (in Russian).
19. Frequently asked questions, available at: https://etcd.io/docs/v3.4/faq/ (date of access).