Journal "Software Engineering"
a journal on theoretical and applied science and technology
ISSN 2220-3397

Issue N1 2025 year

DOI: 10.17587/prin.16.13-27
Method of Collaboration between Humans and Generative Artificial Intelligence in the Development of Information Systems
G. E. Rego, Senior Lecturer, Researcher, regoGr@yandex.ru, E. A. Pitukhin, Professor, Leading Researcher, eugene@petrsu.ru, Institute of Mathematics and Information Technology, Petrozavodsk State University, Petrozavodsk, 185910, Russian Federation
Corresponding author: Grigorij E. Rego, Senior Lecturer, Researcher, Scientist, Institute of Mathematics and Information Technology, Petrozavodsk State University, Petrozavodsk, 185910, Russian Federation, E-mail: regoGr@yandex.ru
Received on September 24, 2024
Accepted on November 14, 2024

We propose a method for information system (IS) design with a focus on the applicability of intelligent chatbots. An analysis of existing chatbots shows that current solutions do not have the ability to formulate IS requirements during designing without human intervention. The study proposes to integrate ChatGPT (GPT4 model) as a chatbot to solve the problem of writing requirements elicitation for IS. The proposed method for solving the problem is based on a gradual reduction in entropy of chatbot response, which can be achieved by controlling such basic query parameters as the form, depth and breadth of the prompt. Based on the proposed method, an algorithm for controling prompts during a dialogue with a chatbot was developed, regulating the form, depth and breadth of the question to obtain the required level of content in the answer. The following results were obtained during the study: a method for obtaining new knowledge based on entropy of chatbot response reduction; algorithm for prompts control; the second phase of IS development is automated (including identification of business requirements and system objectives; analysis of existing solutions and technologies; determination of functional requirements and system capabilities; development of system architecture, including service definition and interaction; creation of interfaces for service interaction; establishment of a data model and database and formulation of business process logic). The method was tested to design an IS to support strategic decision making in the forestry industry. The design results received a positive assessment from experts. Analysis of the obtained results revealed the advantages and limitations of using chatbots as co-pilots in the design of IS and outlined directions for future research.

Keywords: artificial intelligence, chatbots, collaborative design, information systems, design method
pp. 13—27
For citation:
Rego G. E., Pitukhin E. A. Method of Collaboration between Humans and Generative Artificial Intelligence in the Development of Information Systems, Programmnaya Ingeneria, 2025, vol. 16, no. 1, pp. 13—27. DOI: 10.17587/prin.16.13-27.
The research was supported by the Russian Federation Ministry of Science and Higher Education (state research task, theme no. 075-03-2023-128).
References:
  1. Chattopadhyay S., Nelson N., Au A. et al. Cognitive biases in software development, Communications of the ACM, 2022, vol. 65, no. 4, pp. 115—122. DOI: 10.1145/3517217.
  2. Galassi A., Lippi M., Torroni P. Attention in natural lan­guage processing, IEEE transactions on neural networks and learning systems, 2020, vol. 32, no. 10, pp. 4291—4308. DOI: 10.1109/ TNNLS.2020.3019893.
  3. Bang Y., Cahyawijaya S., Lee N. et al. A multitask, multi­lingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023, 2023.
  4. Storey M.-A., Ernst N., Williams C., Kalliamvakou E. The who, what, how of software engineering research: a socio-technical framework, Empirical Software Engineering, 2020, vol. 25, pp. 4097— 4129. DOI: 10.1007/s10664-020-09858-z.
  5. Khurana D., Koli A., Khatter K., Singh S. Natural language processing: State of the art, current trends and challenges, Multimedia tools and applications, 2023, vol. 82, no. 3, pp. 3713—3744. DOI: 10.1007/s11042-022-13428-4.
  6. Pauzi Z., Capiluppi A. Applications of natural language processing in software traceability: A systematic mapping study, Journal of Systems and Software, 2023, vol. 198, article 111616. DOI: 10.1016/j.jss.2023.111616.
  7. Yalla P., Sharma N. Integrating natural language processing and software engineering, International Journal of Software Engineering and Its Applications, 2015, vol. 9, no. 11, pp. 127—136. DOI: 10.14257/ijseia.2015.9.11.12.
  8. Devlin J., Chang M.-W., Lee K., Toutanova K. BERT: Pretraining of Deep Bidirectional Transformers for Language Understanding, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Vol. 1 (Long and Short Papers) / Eds. J. Burstein, C. Doran, T. Solorio. Association for Computational Linguistics, 2019, pp. 4171—4186. DOI: 10.18653/V1/N19-1423.
  9. von der Mosel J., Trautsch A., Herbold S. On the Validity of Pre-Trained Transformers for Natural Language Processing in the Software Engineering Domain, IEEE Transactions on Software Engineering, 2023, vol 49, no. 4, pp. 1487—1507. DOI: 10.1109/TSE.2022.3178469.
  10. Zhao W., Zhou K., Li J. et al. A Survey of Large Language Models. doi: ArXiv abs/2303.18223, 2023.
  11. Hendrycks D., Burns C., Basart S. et al. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020. DOI: 10.48550/arXiv.2009.03300.
  12. Zowghi D., Coulin C. Requirements elicitation: A survey of techniques, approaches, and tools, Engineering and managing software requirements, 2005, pp. 19—46.
  13. Lim S., Henriksson A., Zdravkovic J. Data-driven require­ments elicitation: A systematic literature review, SN Computer Science, 2021, vol. 2, article 16. DOI: 10.1007/s42979-020-00416-4.
  14. Ronanki K., Berger C., Horkoff J. Investigating ChatGPT's Potential to Assist in Requirements Elicitation Processes, 2023 49th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE Computer Society, Los Alamitos, CA, USA, pp. 354—361. DOI: 10.1109/SEAA60479.2023.00061.
  15. Fan A., Gokkaya B., Harman M. et al. Large Language Models for Software Engineering: Survey and Open Problems, 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE). IEEE Computer Society, Los Alamitos, CA, USA, 2023, pp. 31—53. DOI: 10.1109/ICSE-FoSE59343.2023.00008.
  16. Zamfirescu-Pereira J. D., Wong R., Hartmann B., Yang Q. Why Johnny can't prompt: how non-AI experts try (and fail) to design LLM prompts, Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 2023, pp. 1—21.
  17. Abdelnabi S., Greshake K., Mishra S. et al. Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection, Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security, 2023, pp. 79—90.
  18. Sallam M. ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns, Healthcare, 2023, vol. 11. no. 6, article 887. DOI: 10.3390/healthcare11060887.
  19. Lo C. K. What is the impact of ChatGPT on education? A rapid review of the literature, Education Sciences, 2023, vol. 13, no. 4, article 410. DOI: 10.3390/educsci13040410.
  20. Huang J. Tan M. The role of ChatGPT in scientific commu­nication: writing better scientific review articles, American Journal of Cancer Research, 2023, vol. 13, no. 4, article 1148.
  21. Liu P., Yuan W., Fu J. et al. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing, Comput. Surveys, 2023, vol. 55, no. 9, Article 195. DOI: 10.1145/356081.
  22. Clavie B., Ciceu A., Naylor F. et al. Large Language Models in the Workplace: A Case Study on Prompt Engineering for Job Type Classification, Natural Language Processing and Information Systems / Eds. E. Mutais, F. Meziane, V. Sugumaran, W Manning, S. Reiff-Marganiec, Springer Nature Switzerland, Cham, 2023, pp. 3—17.
  23. Brown T. B., Mann B., Ryder N. et al. Language Models are Few-Shot Learners. 2020. CoRR abs/2005.14165. arXiv:2005.14165.
  24. Li L., Zhang Y., Chen L. Prompt distillation for efficient llmbased recommendation, Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, 2023, pp. 1348—1357. DOI: 10.1145/3583780.3615017.
  25. Min S., Lyu X., Holtzman A. et al. Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processin, 2022, pp. 11048—11064. DOI: 10.18653/v1/2022.emnlp-main.759.
  26. Wei J., Bosma M., Zhao V. et al. Finetuned Language Models are Zero-Shot Learners, International Conference on Learning Representations, 2022, available at: https://openreview.net/ forum?id=gEZrGCozdqR (date of access 25.05.2024).
  27. White J., Fu Q., Hays S. et al. A prompt pattern cata­log to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302.11382, 2023.
  28. Zhang R., Hu X., Li B. et al. Prompt, generate, then cache: Cascade of foundation models makes strong few-shot learners?, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 15211—15222.
  29. Christiano P., Leike J., Brown T. Deep Reinforcement Learning from Human Preferences. 2017, ArXiv, abs/1706.03741.
  30. Kaplan J., McCandlish S., Henighan T. et al. Scal­ing Laws for Neural Language Models, 2020. arXiv:2001.08361 DOI: 10.48550/arXiv.2001.08361.
  31. Wei J., Wang X., Schuurmans D. et al. Chain of Thought Prompting Elicits Reasoning in Large Language Models, Proceedings of the 36th International Conference on Neural Information Processing Systems, 2022, pp. 24824 — 24837.
  32. Kojima T., Gu S. S., Reid M. et al. Large language models are zero-shot reasoners, 2022. arXiv:2205.11916. DOI: 10.48550/ arXiv.2205.11916.
  33. Arvidsson S., Axell J. Prompt engineering guidelines for LLMs in Requirements Engineering. 2023, available at: https://hdl. handle.net/2077/77967 (date of access 25.05.2024).
  34. Singh R., Gehlot A., Akram S. et al. Forest 4.0: Digitalization of forest using the Internet of Things (IoT), Journal of King Saud University-Computer and Information Sciences, 2022, vol. 34, no. 8, pp. 5587—5601.
  35. Rubi J., de Carvalho P., Gondim P. Forestry 4.0 and Industry 4.0: Use case on wildfire behavior predictions, Computers and Electrical Engineering, 2022, vol. 102, article 108200. DOI: 10.1016/j.compeleceng.2022.108200.
  36. Sahal R., Alsamhi S., Breslin J., Ali M. Industry 4.0 towards Forestry 4.0: Fire detection use case, Sensors, 2021, vol. 21, no. 3, article 694. DOI: 10.3390/s21030694.
  37. Molinaro M. Orzes G. From forest to finished products: The contribution of Industry 4.0 technologies to the wood sector, Computers in Industry, 2022, vol. 138, article103637. DOI: 10.1016/j. compind.2022.103637.
  38. Prabha C., Connaway L., Olszewski L., Jenkins L. What is Enough? Satisficing Information Needs, Journal of Documentation, 2007, vol. 63, no. 1, pp. 74—89. DOI: 10.1108/00220410710723894.