Journal "Software Engineering"
a journal on theoretical and applied science and technology
ISSN 2220-3397

Issue N12 2025 year

DOI: 10.17587/prin.16.603-611
Method of Explaining Machine Learning Models’ Results Using Fuzzy Logical Rules for Decision Support Systems
A. I. Khalyasmaa, PhD, Head of the Scientific laboratory, a.i.khaliasmaa@urfu.ru, Ural Federal University named after the first President of Russia B. N. Yeltsin, Ekaterinburg, 620062, Russian Federation
Corresponding author: Alexandra I. Khalyasmaa, PhD, Head of the Scientific Laboratory, Ural Federal University named after the first President of Russia B. N. Yeltsin, Ekaterinburg, 620062, Russian Federation, E-mail: a.i.khaliasmaa@urfu.ru
Received on May 16, 2025
Accepted on July 09, 2025

The implementation of machine learning models in decision support systems requires the improvement of explainable artificial intelligence methods, especially in problems with a high cost of erroneous decisions. In this paper, a new method for generating user-friendly explanations of the results of machine learning models is proposed. The main idea of the method is to transform the vector of local feature contributions obtained using the local a posteriori explanation algorithm into a fuzzy logic rule with linguistic variables. Representing explanations in the form of rules allows increasing the understandability and transparency of the model for users of decision support systems. The method was tested on the problem of technical diagnostics of transformer equipment on real-life datasets.

Keywords: explainable artificial intelligence, human-machine interface, fuzzy logic, intelligent software packages, technical diagnostics, high-voltage equipment
pp. 603—611
For citation:
Khalyasmaa A. I. Method of Explaining Machine Learning Models’ Results Using Fuzzy Logical Rules for Decision Support Systems, Programmnaya Ingeneria, 2025, vol. 16, no. 12, pp. 603—611. DOI: 10.17587/prin.16.603-611. (in Russian).
The research was carried out within the state assignment with the financial support of the Ministry of Science and Higher Education of the Russian Federation (subject No. FEUZ-2025-0005, development of models and methods of explainable artificial intelligence to improve the reliability and safety of the implementation of distributed intelligent systems at power facilities).
References:
  1. Khalyasmaa K. I., Stepanova A. I., Zinivieva E. L. et al. Legal aspects of the use of intelligent systems and methods of interpreting, Novosibirsk, Novosibirskiy gosudarstvennyy tekhnicheskiy universitet, 2024, 191 p. (in Russian).
  2. Lundberg S. M., Lee S. A unified approach to interpreting model predictions, NIPS’17: Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017, рр. 4768—4777. DOI: 10.48550/arXiv.1705.07874.
  3. Ribeiro M. T., Singh S., Guestrin C. “Why Should I Trust You?”: Explaining the Predictions of Any Classifier, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics, 2016, pp. 97—101. DOI: 10.48550/arXiv.1602.04938.
  4. Alexander Z., Chau D. H., Saldana C. An Interrogative Survey of Explainable AI in Manufacturing, IEEE Transactions on Industrial Informatics, 2024, vol. 20, no. 5, pp. 7069—7081. DOI: 10.1109/TII.2024.3361489.
  5. Sun Y., Ma H. Interpretable analysis of transformer winding vibration characteristics: SHAP and multi-classification feature optimization, International Journal of Electrical Power & Energy Systems, 2025, vol. 166, article 110585. DOI: 10.1016/j.ijepes.2025.110585.
  6. Khalyasmaa A. I., Matrenin P. V., Eroshenko S. A. Shapley Additive Explanations for Machine Learning-based Decision Support Systems for Power Transformers’ Diagnostics, 2025 IEEE Ural-Siberian Conference on Biomedical Engineering, Radioelectronics and Information Technology (USBEREIT), 2025, pp. 423—426. DOI: 10.1109/USBEREIT65494.2025.11054163.
  7. Zereen A. N., Das A., Uddin J. Machine Fault Diagnosis Using Audio Sensors Data and Explainable AI Techniques-LIME and SHAP, Computers, Materials & Continua, 2024, vol. 80, no. 3, pp. 3463—3484. DOI: 10.32604/cmc.2024.054886.
  8. Brito L. C., Susto G. A., Brito J. N., Duarte M. A. V. An Explainable Artificial Intelligence Approach for Unsupervised Fault Detection and Diagnosis in Rotating Machinery, ArXiv, 2021. DOI: 10.48550/arXiv.2102.11848.
  9. Sharma J., Mittal M. L., Soni G. Explainable AI Enhanced Fault Diagnosis of Power Transformer: Unveiling the Black Box, The 14th Annual International Conference on Industrial Engineering and Operations Management, 2024. DOI: 10.46254/AN14.202400032.
  10. Brusa E., Cibrario L., Delprete C., Di Maggio L. G. Explainable AI for Machine Fault Diagnosis: Understanding Features’ Contribution in Machine Learning Models for Industrial Condition Monitoring, Applied Sciences, 2023, vol. 13, no. 4, article 2038. DOI: 10.3390/app13042038.
  11. Bourokba A., El Hamdi R., Njah M. A. A Shapley based XAI approach for a turbofan RUL estimation, 21st International Multi-Conference on Systems, Signals & Devices (SSD), 2024, pp. 832—837. DOI: 10.1109/SSD61670.2024.10548499.
  12. Ghadekar P., Manakshe A., Madhikar S. et al. Predictive Maintenance for Industrial Equipment: Using XGBoost and Local Outlier Factor with Explainable AI for analysis, 14th International Conference on Cloud Computing, Data Science & Engineering (Confluence), 2024, pp. 25—30. DOI: 10.1109/Confluence60223.2024.10463280.
  13. Harinarayan R. A. R., Shalinie S. M. XFDDC: eXplainable Fault Detection Diagnosis and Correction framework for chemical process systems, Process Safety and Environmental Protection, 2022, vol. 165, pp. 463—474. DOI: 10.1016/j.psep.2022.07.019.
  14. Lundberg S. M., Erion G. G., Lee S. I. Consistent Individualized Feature Attribution for Tree Ensembles, Arxiv. 2018. DOI: 10.48550/arXiv.1802.03888.
  15. Khalyasmaa A. I., Matrenin P. V., Eroshenko S. A. Assessment of Power Transformer Technical State Using Explainable Artificial Intelligence, Problemy Regionalnoy Energetiki, 2024, no. 4 (64), pp. 1—9. DOI 10.52254/1857-0070.2024.4-64.01 (in Russian).
  16. Cao J., Zhou T., Zhi S. et al. Fuzzy inference system with interpretable fuzzy rules: Advancing explainable artificial intelligence for disease diagnosis—A comprehensive review, Information Sciences, 2024, vol. 662, article 120212. DOI: 10.1016/j.ins.2024.120212.
  17. Pickering L., Cohen K., De Baets B. A Narrative Review on the Interpretability of Fuzzy Rule-Based Models from a Modern Interpretable Machine Learning Perspective, International Journal of Fuzzy Systems, 2025. DOI: 10.1007/s40815-025-02022-z.
  18. Niskanen V. A. Methodological Aspects on Integrating Fuzzy Systems with Explainable Artificial Intelligence, Advances in Artificial Intelligence-Empowered Decision Support Systems. Learning and Analytics in Intelligent Systems. 2024, pp 415—438. DOI: 10.1007/978-3-031-62316-5_16.
  19. Apostolopoulos I. D., Groumpos P. P. Fuzzy Cognitive Maps: Their Role in Explainable Artificial Intelligence, Applied Science, 2023, vol. 13, no. 6, article 3412. DOI: 10.3390/app13063412.
  20. Levin V. M., Khalyasmaa A. I., Ahioev D. S., Manusov V. Z. Application of artificial intelligence methods in the tasks of technical diagnostics of electrical equipment of electrical systems, Novosibirsk, Novosibirskiy gosudarstvennyy tekhnicheskiy universitet, 2020, 446 p. (in Russian).
  21. Aghaeipoor F., Sabokrou M., Fernandez A. Fuzzy Rule-Based Explainer Systems for Deep Neural Networks: From Local Explainability to Global Understanding, IEEE Transactions on Fuzzy Systems, 2023, vol. 31, no. 9, pp. 3069—3080. DOI: 10.1109/TFUZZ.2023.3243935.
  22. Ferdaus M. M., Dam T., Alam S., Pham D.-T. X-Fuzz: An Evolving and Interpretable Neuro-Fuzzy Learner for Data Streams, IEEE Transactions on Artificial Intelligence, 2024, vol. 5,no. 8, pp. 4001—4012. DOI: 10.1109/TAI.2024.3363116.
  23. Singh B., Doborjeh M., Doborjeh Z. et al. Constrained neuro fuzzy inference methodology for explainable personalized modelling with applications on gene expression data, Scientific Reports, 2023, vol. 13, article 456. DOI: 10.1038/s41598-022-27132-8.
  24. Gokmen O. B., Guven Y., Kumbasar T. FAME: Introducing Fuzzy Additive Models for Explainable AI, ArXiv, 2025. DOI: 10.48550/arXiv.2504.07011.
  25. Velasquez A. M. R., Lara J. V. M. Data for: Root cause analysis improved with machine learning for failure analysis in power transformers, Engineering Failure Analysis, 2020, vol. 115, article 104684. DOI: 10.1016/j.engfailanal.2020.104684.
  26. Khalyasmaa A. I., Senyuk M. D., Eroshenko S. A. Analysis of the State of High-Voltage Current Transformers Based on Gradient Boosting on Decision Trees, IEEE Transactions on Power Delivery, 2021, vol. 36, no. 4, pp. 2154—2163. DOI: 10.1109/TPWRD.2020.3021702.
  27. Khalyasmaa A. I., Ovchinnikov V. K. Methods of interpretation of results of chromatographic analysis of transformer equipment oil, Bulletin of Kazan State Power Engineering University, 2021, vol. 13, no. 1(49), pp. 177—190 (in Russian).
  28. Chen T., Guestrin C. XGBoost: A Scalable Tree Boosting System, ArXiv. 2016. DOI: 10.1145/2939672.2939785