Journal "Software Engineering"
a journal on theoretical and applied science and technology
ISSN 2220-3397

Issue N3 2026 year

DOI: 10.17587/prin.17.142-154
DeepSeek-Rl Family Large Language Models in the Tasks of Identification, Classification, and Detection of Vulnerabilities from the CWE Top 25 List
V. V. Shvyrov, Cand. (Phys. & Math.), Associate Professor, slshj@yandex.ru, D. A. Kapustin, D. Sc. (Eng.), Head of the Department of Informational Educational Technologies and Systems, kap-kapchik@mail.ru, R. N. Sentyay, Senior Lecturer, sentyayroman@yandex.ru, Lugansk State Pedagogical University, 91011, Russian Federation
Corresponding author: Denis A. Kapustin, D. Sc. (Eng.), Head of the Department of Informational Educational Technologies and Systems, Lugansk State Pedagogical University, 91011, Lugansk, Russian Federation, E-mail: kap-kapchik@mail.ru
Received on August 09, 2025
Accepted on September 16, 2025

The study explores the capabilities of these large language models in the domain of security analysis. Specifically, it investigates the models' ability to classify vulnerability types and descriptions from the CWE Top 25 category, which includes the most dangerous vulnerabilities for 2024 according to the CWE catalog ranking. To evaluate the models, several sets of specialized queries are constructed. These queries test the models' knowledge of vulnerability types as well as their ability to detect these types of vulnerabilities in source code. The experiments conducted with the models yielded a body of statistical data, allowing for an assessment of the models' effectiveness in the tasks of security analysis and vulnerability detection.

Keywords: large language models, vulnerability, static analysis, DeepSeek-R1, CWE
pp. 142—154
For citation:
Shvyrov V. V., Kapustin D. A., Sentyay R. N. DeepSeek-Rl Family Large Language Models in the Tasks of Identification, Classification, and Detection of Vulnerabilities from the CWE Top 25 List, Programmnaya Ingeneria, 2026, vol. 17, no. 3, pp. 142-154. DOI: 10.17587/prin.17.142-154.
References:
  1. Hou X., Zhao Y., Liu Y. et al. Large Language Models for Software Engineering: A Systematic Literature Review, ArXiv, abs/2308.10620, 2023. DOI: 10.48550/arXiv.2308.10620.
  2. Vaswani A., Shazeer N. M., Parmar N. et al. Attention is All you Need, Neural Information Processing Systems, 2017, pp. 5998-6008. DOI: 10.48550/arXiv.1706.03762.
  3. Achiam J., Adler S., Agarwal S. et al. GPT-4 Technical Report. 2023, ArXiv, abs/2306.08568, 2023. DOI: 10.48550/arXiv.2303.08774.
  4. Luo Z., Xu C., Zhao P. et al. WizardCoder: Empowering Code Large Language Models with Evol-Instruct, ArXiv, abs/2306.08568, 2023. DOI: 10.48550/arXiv.2306.08568.
  5. Roziere B., Gehring J., Gloeckle F. et al. Code Llama: Open foundation models for code, ArXiv, abs/2308.12950, 2023. DOI: 10.48550/arXiv.2308.12950.
  6. Cheshkov A., Zadorozhny P. A., Levichev R. Evaluation of ChatGPT Model for Vulnerability Detection, ArXiv, abs/2304.07232, 2023. DOI: 10.48550/arXiv.2304.07232.
  7. Fu M., Tantithamthavorn С. K., Nguyen V., Le T. ChatGPT for Vulnerability Detection, Classification, and Repair: How Far Are We? 2023 30th Asia-Pacific Software Engineering Conference (AP-SEC), 2023, pp. 632-636. DOI: 10.1109/APSEC60848.2023.00085.
  8. Shestov A., Cheshkov A., Levichev R. et al. Finetuning Large Language Models for Vulnerability Detection, ArXiv, abs/2401.17010, 2024. DOI: 10.48550/arXiv.2401.17010.
  9. Shvyrov V. V., Kapustin D. A., Sentyay R. N., Shulika T. I. Analysis of Datasets and Large Language Models for Vulnerability Detection in Imperative Programming Language Code, Programmnaya Ingeneria, 2024, vol. 15, no. 11, pp. 555-569. DOI: 10.17587/prin.15.555-569.
  10. Shvyrov V. V., Kapustin D. A., Sentyay R. N., Shulika T. I. Using Large Language Models to Classify Vulnerabilities in Program Code, Programmnaya Ingeneria, 2024, vol. 15, no. 9, pp. 465—475. DOI: 10.17587/prin.15.465-475.
  11. Fu M., Tantithamthavorn С. K. LineVul: A Transformer-based Line-Level Vulnerability Prediction, 2022 IEEE/ACM 19th International Conference on Mining Software Repositories (MSR), 2022, pp. 608—620. DOI: 10.1145/3524842.3528452.
  12. Chan A., Kharkar A., Moghaddam R. Z. et al. Transformer-based Vulnerability Detection in Code at EditTime: Zero-shot, Few-shot, or Fine-tuning? ArXiv, abs/2306.01754, 2023. DOI: 10.48550/ arXiv.2306.01754.
  13. Feng Z., Guo D., Tang D. et al. CodeBERT: A Pre-Trained Model for Programming and Natural Languages, Findings of the Association for Computational Linguistics: EMNLP 2020. 2020, рр. 1536—1547. DOI: 10.18653/v1/2020.findings-emnlp.139.
  14. Liu A., Feng B., Xue B. et al. DeepSeek-V3 Technical Report, ArXiv, abs/2412.19437, 2024. DOI: 10.48550/arXiv.2412.19437.
  15. Qwen Team. Qwen2 Technical Report, ArXiv, abs/2407.10671, 2024. DOI: 10.48550/arXiv.2407.10671.
  16. Guo D., Yang D., Zhang H. et al. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning, ArXiv, abs/2501.12948, 2025. DOI: 10.48550/arXiv.2501.12948.
  17. Hui B., Yang J., Cui Z. et al. Qwen2.5 Technical Report, ArXiv, abs/2412.15115, 2024. DOI: 10.48550/arXiv.2412.15115.
  18. Hurst A., Lerer A., Goucher A. P. et al. GPT-4o system card, ArXiv, abs/2410/21276, 2024. DOI: 10.48550/arXiv.2410.21276.