Journal "Software Engineering"
a journal on theoretical and applied science and technology
ISSN 2220-3397

Issue N5 2026 year

DOI: 10.17587/prin.17.239-252
Rare Events in Deep Learning Models
F. M. Baryshnikov, Graduate Student, baryshnikovfm@mail.ru, D. E. Namiot, D. Sc. (Eng.), Leading Researcher, dnamiot@gmail.com, Lomonosov Moscow State University, Moscow, 119991, Russian Federation
Corresponding author: Dmitry E. Namiot, D. Sc. (Eng.), Leading Researcher, Lomonosov Moscow State University, Moscow, 119991, Russian Federation, E-mail: dnamiot@gmail.com
Received on September 23, 2025
Accepted on December 10, 2025

Rare event detection is the task of identifying events that occur with very low frequency but have high practical or societal impact. The identification and prediction of such events present a fundamental challenge due to the scarcity of positive samples, high dimensionality of observational data, and the presence of complex temporal dependen­cies. Examples of rare events include certain types of rare diseases, equipment failures in manufacturing that lead to unplanned downtime, and complex cyberattacks. The frequency and nature of rare events vary substantially across different domains. In recent years, numerous methods have been developed for effective detection and classification of rare events, including modern deep learning models that achieve strong performance even when only a limited amount of labeled data is available. This article presents a systematic review and proposes а unified conceptual framework for rare event detection, distinguishing it from related tasks and clarifying the terminology used in the literature. The primary sources of rarity are analysed, including intrinsic rarity, sampling biases, annotation subjectivity, and measurement limitations. Particular attention is given to challenges arising in multivariate time series data, such as spatial correlations, short- and long-term temporal dependencies, unknown lag structures, and combined spatio-temporal effects. These factors substantially complicate modelling and often render classical balancing and augmentation techniques ineffective. The article reviews state-of-the-art methods for data preprocessing, feature selection, and feature engineering tailored to rare-event scenarios, as well as advanced deep learning models capable of capturing complex dependencies, including recurrent architectures, attention-based models, transformers, and deep reinforcement learning approaches. Specialized performance metrics designed to address extreme class imbalance are also discussed. The survey consolidates current methodological advances and identifies key challenges that remain open for future research. It provides a foundation for developing robust diagnostic, monitoring, and early-warning systems operating under conditions of rare and underrepresented events.

Keywords: machine learning, deep learning, rare events
pp. 239—252
For citation:
Baryshnikov F. M., Namiot D. E. Rare Events in Deep Learning Models, Programmnaya Ingeneria, 2026, vol. 17, no. 5, pp. 239—252. DOI: 10.17587/prin.17.239-252. (in Russian).
References:
  1. Abubakar Y. I., Othmani A., Siarry P., Sabri A. Q. M. A systematic review of rare events detection across modalities using machine learning and deep learning, IEEE Access, 2024, vol. 12, pp. 47091—47109. DOI: 10.1109/ACCESS.2024.3382140.
  2. King G., Zeng L. Logistic regression in rare events data, Political Analysis, 2002, vol. 9, online. DOI: 10.1093/oxfordjournals.pan.a004868.
  3. Carreno A., Inza I., Lozano J. A. Analyzing rare event, anomaly, novelty and outlier detection terms under the supervised classification framework, Artificial Intelligence Review, 2020, vol. 53, no. 5, pp. 3575—3594. DOI: 10.1007/s10462-019-09771-y.
  4. Murray J. F., Hughes G. F., Kreutz-Delgado K. Machine learning methods for predicting failures in hard drives: a multiple-instance application, Journal of Machine Learning Research, 2005, vol. 6, pp. 783—816.
  5. Mori U., Mendiburu A., Dasgupta S., Lozano J. A. Early classification of time series by simultaneously optimizing the accura­cy and earliness, IEEE Transactions on Neural Networks and Learning Systems, 2018, vol. 29, no. 10, pp. 4569—4578. DOI: 10.1109/TNNLS.2017.2764939.
  6. Ogbechie A., Dfaz-Rozo J., Larranaga P., Bielza C. Dynamic Bayesian network-based anomaly detection for in-process visual inspection of laser surface heat treatment, Machine Learning for Cyber Physical Systems. Technologien fur die intelligente Automation / Eds. J. Beyerer, O. Niggemann, C. Kiihnert. Berlin, Heidelberg, Springer Vieweg, 2017. DOI: 10.1007/978-3-662-53806-7_3.
  7. Chandola V., Banerjee A., Kumar V. Anomaly detection: a survey, ACM Computing Surveys, 2009, vol. 41, no. 3, article 15. DOI: 10.1145/1541880.1541882.
  8. Rostami S., Ahmadzadeh M. Extracting predictor variables to construct breast cancer survivability model with class imbalance problem, Journal of AI and Data Mining, 2018, vol. 6, no. 2,pp. 263—276. DOI: 10.22044/JADM.2017.5061.1609.
  9. Fiore U., De Santis A., Perla F. et al. Using generative adversarial networks for improving classification effectiveness in credit card fraud detection, Information Sciences, 2019, vol. 479, pp. 448—455. DOI: 10.1016/j.ins.2017.12.030.
  10. Khreich W., Khosravifar B., Hamou-Lhadj A., Talhi C. An anomaly detection system based on variable N-gram features and one-class SVM, Information and Software Technology, 2017, vol. 91, pp. 186—197. DOI: 10.1016/j.infsof.2017.07.009.
  11. Einarsdottir H., Emerson M. J., Clemmensen L. H. et al. Novelty detection of foreign objects in food using multi-modal X-ray imaging, Food Control, 2016, vol. 67, pp. 39—47. DOI: 10.1016/j. foodcont.2016.02.023.
  12. Masud M. M., Chen Q., Khan L. et al. Classification and adaptive novel class detection of feature-evolving data streams, IEEE Transactions on Knowledge and Data Engineering, 2013, vol. 25, no. 7, pp. 1484—1497. DOI: 10.1109/TKDE.2012.109.
  13. de Faria E. R., Ponce de Leon Ferreira Carvalho A. C., Gama J. MINAS: multiclass learning algorithm for novelty detection in data streams, Data Mining and Knowledge Discovery, 2016, vol. 30, pp. 640—680. DOI: 10.1007/s10618-015-0433-y.
  14. Spinosa E. J., de Leon F. de Carvalho A. P., Gama J. OLINDDA: a cluster-based approach for detecting novelty and con­cept drift in data streams, Proceedings of the 2007 ACM Symposium on Applied Computing (SAC '07), New York, ACM, 2007, pp. 448—452. DOI: 10.1145/1244002.1244107.
  15. Zhu Y., Ting K., Zhou Z.-H. Multi-label learning with emerging new labels, IEEE Transactions on Knowledge and Data Engineering, 2018, vol. 30, no. 10, pp. 1901—1914. DOI: 10.1109/TKDE.2018.2810872.
  16. Barai A. (Deb), Dey L. Outlier detection and removal algorithm in K-means and hierarchical clustering, World Journal of Computer Application and Technology, 2017, vol. 5, no. 2, pp. 24—29. DOI: 10.13189/wjcat.2017.050202.
  17. Radovanovic M., Nanopoulos A., Ivanovic M. Reverse near­est neighbors in unsupervised distance-based outlier detection, IEEE Transactions on Knowledge and Data Engineering, 2015, vol. 27, no. 5, pp. 1369—1382. DOI: 10.1109/TKDE.2014.2365790.
  18. Dang X. H., Assent I., Ng R. T. et al. Discriminative features for identifying and interpreting outliers, 2014 IEEE 30th International Conference on Data Engineering, 2014, pp. 88—99. DOI: 10.1109/ICDE.2014.6816642.
  19. Gupta M., Gao J., Aggarwal C. C., Han J. Outlier detection for temporal data: a survey, IEEE Transactions on Knowledge and Data Engineering, 2014, vol. 26, no. 9, pp. 2250—2267. DOI: 10.1109/TKDE.2013.184.
  20. Shyalika C., Wickramarachchi R., Sheth A. P. A comprehensive survey on rare event prediction, ACM Computing Surveys, 2024, vol. 57, no. 3, article 70. DOI: 10.1145/3699955.
  21. Ranjan C. Understanding deep learning and applica­tion in rare event prediction, 2020. Preprint. DOI: 10.13140/RG.2.2.34297.49765.
  22. Xu D., Zhang Z., Shi J. Training data selection by cat­egorical variables for better rare event prediction in multiple prod­ucts production line, Electronics, 2022, vol. 11, no. 7, article 1056. DOI: 10.3390/electronics11071056.
  23. Coffinet J., Kien J.-N. Detection of rare events: a machine learning toolkit with an application to banking crises, The Journal of Finance and Data Science, 2019, vol. 5, no. 4, pp. 183—207. DOI: 10.1016/j.jfds.2020.04.001.
  24. Xiu Z., Tao C., Gao M. et al. Variational disentanglement for rare event modeling, Proceedings of the AAAI Conference on Artificial Intelligence, 2021, vol. 35, no. 12, pp. 10469—10477. DOI: 10.1609/aaai.v35i12.17253.
  25. Omar Z. A., Chin S. N., Hashim S. R. M., Hamzah N. Exploring clusters of rare events using unsupervised random forests, Journal of Physics: Conference Series, 2022, vol. 2314, no. 1, article 012019. DOI: 10.1088/1742-6596/2314/1/012019.
  26. Ashraf M. T., Dey K., Mishra S. Identification of high-risk roadway segments for wrong-way driving crash using rare event modeling and data augmentation techniques, Accident Analysis & Prevention, 2023, vol. 181, article 106933. DOI: 10.1016/j.aap.2022.106933.
  27. Rehab A., Ali I., Gomaa W., Fors M. N. Bearings fault detection using hidden Markov models and principal component analysis enhanced features, PHM Society European Conference, 2021, vol. 6, no. 1, article 11. DOI: 10.36001/phme.2021.v6i1.2947.
  28. Xu H., Ma R., Yan L., Ma Z. Two-stage prediction of machinery fault trend based on deep learning for time series analysis, Digital Signal Processing, 2021, vol. 117, article 103150. DOI: 10.1016/j.dsp.2021.103150.
  29. Bhanja S., Das A. A black swan event-based hybrid model for Indian stock markets' trends prediction, Innovations in Sys­tems and Software Engineering, 2024, vol. 20, no. 2, pp. 121—135. DOI: 10.1007/s11334-021-00428-0.
  30. Neuman Y., Cohen Y., Erez E. Extreme rare events identification through Jaynes inferential approach, Big Data, 2021, vol. 9, no. 6, pp. 417—426. DOI: 10.1089/big.2021.0191.
  31. Zhao Y., Wong Z. S.-Y., Tsui K. L. A framework of rebalancing imbalanced healthcare data for rare events' classifica­tion: a case of look-alike sound-alike mix-up incident detec­tion, Journal of Healthcare Engineering, 2018, article 6275435. DOI: 10.1155/2018/6275435.
  32. Li J., Fong S., Hu S. et al. Rare event prediction using simi­larity majority under-sampling technique, Lecture Notes in Computer Science, 2017, pp. 23—39. DOI: 10.1007/978-981-10-7242-0_3.
  33. Liu H., Ma R., Li D. et al. Machinery fault diagnosis based on deep learning for time series analysis and knowledge graphs, Journal of Signal Processing Systems, 2021, vol. 93, no. 12, pp. 1433—1455. DOI: 10.1007/s11265-021-01718-3.
  34. Fathy Y., Jaber M., Brintrup A. Learning With Imbal­anced Data in Smart Manufacturing: A Comparative Analysis, IEEE Access, 2021, vol. 9, pp. 2734—2757. DOI: 10.1109/AC-CESS.2020.3047838.
  35. Lee W., Seo K. Early failure detection of paper manufacturing machinery using nearest neighbor-based feature extraction, Engineering Reports, 2021, vol. 3, no. 2, article e12291. DOI: 10.1002/eng2.12291.
  36. Marins M. A., Barros B. D., Santos I. H. et al. Fault detection and classification in oil wells and production/service lines using random forest, Journal of Petroleum Science and Engineering, 2021, vol. 197, article 107879. DOI: 10.1016/j.petrol.2020.107879.
  37. Zhang Y., Zhou T., Huang X. et al. Fault diagnosis of rotating machinery based on recurrent neural networks, Measurement, 2021, vol. 171, article 108774. DOI: 10.1016/j.measurement.2020.108774.
  38. Hochreiter S., Schmidhuber J. Long Short-Term Memory, Neural Comput., 1997, vol. 9, no. 8, pp. 1735—1780. DOI: 10.1162/neco.1997.9.8.1735.
  39. Gopali S., Abri F., Siami-Namini S., Siami-Namini A. A Comparative Study of Detecting Anomalies in Time Series Data Using LSTM and TCN Models, arXiv, 2021. DOI: 10.48550/arX-iv.2112.09293.
  40. Yadav S., Jain A., Sharma K. Ch., Bhakar R. Load Forecasting for Rare Events using LSTM, 2021 9th IEEE International Conference on Power Systems (ICPS), 2021, pp. 1—6. DOI: 10.1109/ICPS52420.2021.9670200.
  41. Zhang S., Bahrampour S., Ramakrishnan N. et al. Deep learning on symbolic representations for large-scale heterogeneous time-series event prediction, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017, pp. 5970—5974. DOI: 10.1109/ICASSP.2017.7953302.
  42. Peng H., Li H., Zhang Y. et al. Multi-Sensor Vibration Signal Based Three-Stage Fault Prediction for Rotating Mechanical Equipment, Entropy, 2022, vol. 24, no. 2, article 164. DOI: 10.3390/e24020164.
  43. Ravindranath M., Candan K. S., Sapino M. L. M2NN: Rare event inference through multi-variate multi-scale attention, 2020 IEEE International Conference on Smart Data Services (SMDS), 2020, pp. 53—62. DOI: 10.1109/SMDS49396.2020.00014.
  44. Li A., Fu Z., Grant C. et al. KamNet: An integrated spatiotemporal deep neural network for rare event searches in Kam-LAND-Zen, Phys. Rev. C, 2023, vol. 107, no. 1, article 014323. DOI: 10.1103/PhysRevC.107.014323.
  45. Vaswani A., Shazeer N., Parmar N. et al. Attention is all you need, Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS'17), Long Beach, California, USA, 2017. pp. 6000—6010.
  46. Wang R., Dong E., Cheng Z. et al. Transformer-based intelligent fault diagnosis methods of mechanical equipment: a survey, Open Physics, 2024, vol. 22, no. 1, article 20240015. DOI: 10.1515/phys-2024-0015.
  47. Miinchmeyer J., Bindi D., Leser U., Tilmann F. The transformer earthquake alerting model: a new versatile approach to earth-quake early warning, Geophysical Journal International, 2021, vol. 225, no. 1, pp. 646—656. DOI: 10.1093/gji/ggaa609.
  48. Bogdevicius M., Januteniene J., Razmas S. et al. 11th World Congress on Computational Mechanics (WCCM2014), 5th European Conference on Computational Mechanics (ECCM V), 6th European Conference on Computational Fluid Dynamics (ECFD VI), July 20— 25, 2014, Barcelona, Spain, 2014. DOI: 10.13140/2.1.4186.9441.