DOI: 10.17587/prin.14.603-615

Reinforcement Learning Methods and Algorithms for Modeling Individual Trajectories of Professional Development

E. V. Orlova, Dr. of Sc. (Eng), Professor, ekorl@mail.ru, Ufa University of Science and Technology, Ufa, 450076, Russian Federation

Corresponding author: Ekaterina V. Orlova, Dr. of Sc. (Eng), Professor, Ufa University of Science and Technology, Ufa, 450076, Russian Federation, E-mail: ekorl@mail.ru

Corporate human capital is one of the most important drivers of sustainable economic growth that is becoming increasingly important under changing nature of work. Due to the expansion of various areas of human activity, an employee portrait is getting multifaceted. Therefore, the problem of human capital management based on the individual trajectories of professional development, aimed at increasing the labor efficiency and contributing to the growth of he corporate operational efficiency, is relevant, timely, socially and economically significant. The paper proposes an approach for the dynamic regimes for human capital development (DRHC), to design individual trajectories for the employees professional development, based on reinforcement learning methods. The DRHC develops an optimal management regime as a set of programs aimed at developing an employee in the professional field, taking into account their individual characteristics (health quality, major and interdisciplinary competencies, motivation and social capital). The DRHC architecture consists of an environment — an employee model as a Markov decision making process and an agent — decision making center of a company. The DRHC uses Double Deep Q-Network algorithms to maximize the agents utility function. The implementation of the proposed by DRHC policies would enhance the corporate HC quality, improve its resource efficiency and ensure the performance growth.

Keywords: machine learning, reinforcement learning, Q-learning, corporate human capital, individual development trajectories

pp. 603–615

For citation:
Orlova E. V. Reinforcement Learning Methods and Algorithms for Modeling Individual Trajectories of Professional Development, Programmnaya Ingeneria, 2023, vol. 14, no. 12, pp. 603—615. DOI: 10.17587/prin.14.603-615. (in Russian).

References:

• Samuelson P. A. Consumption Theory in Terms of Revealed Preference, Economica, 1948, vol. 60, no. 15, pp. 243—253.
• Neumann J., Morgenstern O. Theory of Games and Economic Behavior, Princeton, Princeton University Press, 1944, 625 p.
• Orlova E. V. Methods and Models of Data Analysis and Machine Learning in the Problem of Labor Productivity Management, Programmnaya Ingeneria, 2020, vol. 11, no. 4, pp. 219—229. DOI: 10.17587/prin.11.219-229 (in Russian).
• Orlova E. V. System Engineering of the Organizational and Technical Systems' Digital Twins Using Artificial Intelligence Methods, Programmnaya Ingeneria, 2022, vol. 13, no. 9, pp. 425— 439. DOI: 10.17587/prin.13.425-439 (in Russian).
• Markov A. A. The Theory of Algorithms, Journal of Symbolic Logic, 1953, vol. 18, no. 4, pp. 340—341. DOI: 10.2307/2266585.
• Sutton R. S., Barto A. G. Reinforcement Learning. Am Introduction, Cambridge, MA, MIT Press, 2020, 552 p.
• Bellman R. A Markovian decision process, Journal о/Mathematics And Mechanics, 1957, vol. 6, no. 4, pp. 679—684.
• Ding Q., Jahanshahi H., Wang Y. et al. Optimal Reinforcement Learning-Based Control Algorithm for a Class of Nonlinear Macroeconomic Systems, Mathematics, 2022, vol. 10, no. 3, article 499. DOI: 10.3390/math10030499.
• Li Q., Lin T., Yu Q. et al. Review of Deep Reinforcement Learning and Its Application in Modern Renewable Power System Control, Energies, 2023, vol. 16, no. 10, article 4143. DOI: 10.3390/en16104143.
• Orlova E. V. Decision-Making Techniques for Credit Resource Management Using Machine Learning and Optimization, Information, 2020, vol. 11, no. 3, article 144. DOI: 10.3390/info11030144.
• Wang R., Chen Z., Xing Q. et al. A Modified RainbowBased Deep Reinforcement Learning Method for Optimal Scheduling of Charging Station, Sustainability, 2022, vol. 14, no. 3, article 1884. DOI: 10.3390/su14031884.
• Abideen A.Z, Sundram V. P. K., Pyeman J. et al. Digital Twin Integrated Reinforced Learning in Supply Chain and Logistics, Logistics, 2021, vol. 5, no. 4, article 84. DOI: 10.3390/logistics5040084.
• Yan Y., Chow A. H., Ho C. P. et al. Reinforcement Learning for Logistics and Supply Chain Management: Methodologies, State of the Art, and Future Opportunities, Transportation Research Part E: Logistics and Transportation Review, 2022, vol. 162, article 102712. DOI: 10.1016/j.tre.2022.102712.
• Han D., Mulyana B., Stankovic V., Cheng S. A Survey on Deep Reinforcement Learning Algorithms for Robotic Manipulation, Sensors, 2023, vol. 23, no. 7, article 3762. DOI: 10.3390/s23073762.
• Orr J., Dutta A. Multi-Agent Deep Reinforcement Learning for Multi-Robot Applications: A Survey, Sensors, 2023, vol. 23, no. 7, айкк; 3625. DOI: 10.3390/s23073625.
• Mohammadi М., Al-Fuqaha А. Guizani М., Oh J. Semisupervised deep reinforcement leaming in support of loT and smart city ser­vices, IEEE Intemet of Things Joumal, 2018, vol. 5, no. 2, pp. 624—635.
• Yu C., Liu J., Nemati S. Reinforcement Learning in Healthcare: A Survey. arXiv:1908.08796. 2019.
• Chi M., VanLehn K., Litman D. et al. Empirically evaluating the application of reinforcement learning to the induction of effective and adaptive pedagogical strategies, User Model User-Adap Inter, 2011, vol. 21, pp. 137—180 DOI: 10.1007/s11257-010-9093-1.
• Azhikodan A. R., Bhat A. G., Jadhav M. V. Stock Trading Bot Using Deep Reinforcement Learning, In Innovations in Computer Science and Engineering; Springer: Berlin/Heidelberg, Germany, 2019, pp. 41—49.
• Ponomarev E. S., Oseledets I. V., Chikhotsky A. S. Using Reinforcement Learning in the Problem of Algorithmic Trading, Informacionnye processy, 2019, vol. 19, no. 2, pp. 122—131 (in Russian).
• Gurin A. S., Gurin Ya. S., Gorohova R. I. et al. Increasing the profitability of a sales agent based on the Q-learning method using derivative financial indicators, Sovremennye informacionnye tehnologii i IT-obrazovanie, 2020, vol. 16, no. 3, pp. 799—809. DOI: 10.25559/SITITO.16.202003.799-809 (in Russian).
• Gataullin S. T., Khasanshin I. Ya., Nikitin P. V. et al. Different approaches to the application of reinforcement learning tech­nology in algorithmic trading, Fundamental'nye issledovanija, 2021, no. 12, pp. 86—91. DOI 10.17513/fr.43158 (in Russian).
• Jiang Z.; Liang J. Cryptocurrency portfolio management with deep reinforcement learning., In Proceedings of the 2017Intelligent Systems Conference (IntelliSys), London, UK, 7—8 September 2017, pp. 905—913.
• Yu P., Lee J. S., Kulyatin I. et al. Model-based Deep Re­inforcement Learning for Dynamic Portfolio Optimization. arXiv 2019, arXiv:1901.08740.
• Amirzadeh R., Nazari A., Thiruvady D. Applying Artificial Intelligence in Cryptocurrency Markets: A Survey, Algorithms, 2022, vol. 15, no. 11, айкк; 428. DOI: 10.3390/a15110428.
• Feng L., Tang R., Li X. et al. Deep reinforcement learning based recommendation with explicit user-item interactions model­ing. arXiv 2018, arXiv:1810.12027.
• Liu J., Zhang Y., Wang X. et al. Dynamic Pricing on E-commerce Platform with Deep Reinforcement Learning. arXiv 2019, arXiv:1912.02572.
• Zheng G., Zhang F., Zheng, Z. et al. DRN: A deep reinforcement learning framework for news recommendation, In Proceedings of the 2018WorldWideWeb Conference, Lyon, France, 23—27 April 2018, pp. 167—176.
• Orlova E. V. Inference of Factors for Labor Productivity Growth Used Randomized Experiment and Statistical Causality, Mathematics, 2023, vol. 11, no. 4, article 863. DOI: 10.3390/math11040863.
• Orlova E. V. Methodology and Statistical Modeling of Social Capital Influence on Employees' Individual Innovative-ness in a Company, Mathematics, 2022, vol. 1, no. 11, article 1809. DOI: 10.3390/math10111809.
• Orlova E. V. Innovation in Company Labor Productivity Management: Data Science Methods Application, Applied System Innovation, 2021, vol. 4, no. 3, article 68. DOI: 10.3390/asi4030068.
• Orlova E. V. Design of Personal Trajectories for Employees' Professional Development in the Knowledge Society under Industry 5.0, Social Sciences, 2021, vol. 10, no. 11, article 427. DOI: 10.3390/socsci10110427.
• Orlova E. V. Assessment of the Human Capital of an Enterprise and its Management in the Context of the Digital Transformation of the Economy, Journal of Applied Economic Research, 2021, vol. 20, no. 4, pp. 666—700. DOI: 10.15826/vestnik.2021.20.4.026.