DOI: 10.17587/prin.12.189-199

Methods and Technologies for Quality Assurance of Intelligent Decision-Making Systems

O. N. Dolinina, odolinina09@gmail.com, Yuri Gagarin State University of Saratov, Saratov, 410054, Russian Federation, V. A. Kushnikov, kushnikoff@yandex.ru, Saratov Scientific Center of Russian Academy of Sciences, Saratov, 410028, Russian Federation

Corresponding author: Dolinina Olga N., PhD, Director of the Institute, Yuri Gagarin State University of Saratov, Saratov, 410054, Russian Federation, E-mail: odolinina09@gmail.com

An increase in the degree of intellectualization of tasks requires the creation of methodology for improving the quality of intelligent decision-making systems. The possibility of automating decision-making in poorly formalized areas through the using of the expert knowledge leads to increasing of the number of errors in the software, and as a consequence to increasing of the number of various sources of failures.The article provides a detailed overview of existing methods and technologies for quality assurance of intelligent decision systems. The first part of the article describes the methodology for ensuring the quality of the intelligent systems (IS), based on the GOST/ ISO standards, where it is proposed to use a multilevel model to describe the quality of the IS software. It is shown that to ensure the required level of quality, an action plan can be formed and the use of a system dynamics model for the implementation of an action plan for ensuring the quality of IS is described. A comparative analysis of the complex criteria of quality and reliability is given. In the second part, the quality of knowledge base (KB) as a special element of the IS software is described, a comparative analysis of methods for static and dynamic analysis of knowledge bases is considered. An overview of research results in the classification of errors in the knowledge bases and their debugging is given. Special attention is given to the "forgetting about exception" type of errors. The concept of a statically correct knowledge base at the level of the knowledge structure is described and it is shown that statically correct knowledge bases can nevertheless give errors due to errors in the rules themselves because of the inconsistency of the field of studies. Neural network knowledge bases are allocated in a separate class, for neural networks methods of debugging are described.

Keywords: quality of intelligent systems, quality assurance plan, reliability, knowledge base errors, "forgetting about an exception" error, debugging, inconsistency of knowledge, testing

pp. 189–199

For citation:
Dolinina O. N., Kushnikov V. A. Methods and Technologies for Quality Assurance of Intelligent Decision-Making Systems, Programmnaya Ingeneria, 2021, vol. 12, no. 4, pp. 189—199

This work was supported by the Russian Foundation for Basic Research, project nos. 20-17-50028/20