main| new issue| archive| editorial board| for the authors| publishing house|
Ðóññêèé
Main page
New issue
Archive of articles
Editorial board
For the authors
Publishing house

 

 


ABSTRACTS OF ARTICLES OF THE JOURNAL "INFORMATION TECHNOLOGIES".
No. 7. Vol. 30. 2024

DOI: 10.17587/it.30.357-366

M. V. Kopeliovich, Postgraduate Student, I. V. Shcherban, Leading Researcher,
Southern Federal University, Rostov-on-Don, 349006, Russian Federation

Method of Remote Photoplethysmography Robust to Interference in Video Registration of Human Facial Skin

Monitoring of heart rate (HR) and its dynamics is necessary in ambulatory and telemedicine for diagnosis and treatment of diseases. Remote photoplethysmography (rPPG) allows for HR and its dynamics monitoring by video analysis of facial skin blood flow, and is ofparticular importance for patients with delicate skin such as infants, the elderly, or those with severe burn injuries. Unlike other contactless HR measurement methods, rPPG does not require special hardware, but allows to monitor HR on the basis of analyzing a sequence of video images of a person's face. rPPG involves a multi-step process including signal extraction, dimensionality reduction to estimate the photoplethysmographic (PPG) signal, and subsequent HR estimation. However, the presence of high-amplitude spikes due to subject movement, facial expressions, lighting fluctuations, video compression artifacts, ROI tracking errors, among others, can interfere the useful PPG signal, leading to inaccurate HR estimations. A method has been developed that allows to increase the accuracy of HR estimation in the rPPG problem due to its robustness to interferences inevitable during video recording. The proposed approach tackles the rPPG challenge by applying a pre-processing approximation of the signal finite difference using a single-layer neural network with radial basis function (RBF) inner layer. Transitioning to the signal finite difference helps reduce the amplitudes of irrelevant low-frequency peaks within the HR search range, thus avoiding their masking effect on the HR-related spectral peaks. The neural network's RBF approximation further diminishes irrelevant high-frequency spectral peaks when the number of RBF nodes is less than half the signal sample count. The correctness of the solutions is confirmed by numerical experiments carried out on the Mahnob-HCI public database.
Keywords: remote photoplethysmography, gradient descent method, radial basis functions, Mahnob-HCIpublic database

P. 357-366

References

  1. Larinskii N. E. History of Physical Diagnostic Methods, Ryazan, RyazGMU, 2007, 216 p. (in Russian).
  2. Svistunov A. A. Heart Rate as a Risk Factor for Cardiovas­cular Diseases Development, Arterial Hypertension, 2008, vol. 14, no. 4, pp. 324—331 (in Russian).
  3. Laba E. V. The Influence of Heart Rate on Acute Coro­nary Syndrome Progression, MNS, 2011, vol. 5, no. 36, pp. 82—84 (in Russian).
  4. Voynov V. B., Voronova N. V., Zolotukhin V.V. Methods of Assessing the State of Human Organism Oxygen Supply Systems, Rostov-on-Don, UNII Valeology RSU, 2002, 99 p. (in Russian).
  5. Snezhitskii V. A., Shpak N. V. The Significance of Heart Rate Dynamics Data from Holter ECG Monitoring in the Diagnosis of Sinus Node Weakness Syndrome, Journal of GrGMU, 2008, vol. 3, no. 23, pp. 111—114 (in Russian).
  6. Novikova D. S. et al. Dynamics of Heart Rate, Heart Rate Variability Indicators, and Q-T Interval in Women with Rheumatoid Arthritis During Treatment with Rituximab, Scientific and Practical Rheumatology, 2014, vol. 3, pp. 270—276 (in Russian).
  7. Oleinikov V. E., Kulyutsin A. V., Lukyanova M. V. Aspects of Physiological Regulation and Available Methods for Recording Heart Rate, News of Universities. Volga Region. Medical Sciences, 2014, vol. 1, no. 29, pp. 70—80 (in Russian).
  8. Polevaya S. A. et al. Discrete Monitoring and Telemetry of Heart Rhythm During Intensive Computer Work for Fatigue and Stress Assessment and Prevention, Medical Almanac, 2013, vol. 2, no. 26 (in Russian).
  9. Nekrasova M. M. Stress Assessment in Professional Activ­ity of Engineers Working with Visual Display Terminals, Medical Almanac, 2015, vol. 5, no. 40. pp. 189—192 (in Russian).
  10. Sadykova E. V., Semenovskaya M.V. Doctor's Expert System for Evaluating the Psychophysiological Condition of an Operator, Biotechnosphere, 2015, vol. 4, no. 40, pp. 50—54 (in Russian).
  11. Baevskii R. M. et al. Studies on Heart Rate Variability During Sleep Aboard the International Space Station, Bulletin of Udmurt University. Series "Biology. Earth Sciences", 2012, vol. 1, pp. 30—37 (in Russian).
  12. Plewa L., Luis S. iStress: Stress Classification from Heart Rate Variability, Cal Poly: CSC, 2015, pp. 96.
  13. Kublanov V. S., Purtov K. S. Remote Photoplethysmography in the Task of Studying Heart Rate Variability, Biomedical Radioelectronics, 2015, no. 8, pp. 3—9 (in Russian).
  14. Li X. et al. Remote Heart Rate Measurement From Face Videos Under Realistic Situations, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2014.
  15. Poh M. Z., McDuff D. J., Picard R. W. Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam, IEEE Trans. Biomed. Eng., 2011, vol. 58, no. 1, pp. 7—11.
  16. Kopeliovich M. V., Shcherban I. V. Method for Selecting Maximally Discriminative Areas Based on Spectral Entropy in Remote Photoplethysmography, Information Technologies, 2022, vol. 28, no. 2, pp. 102—112 (in Russian).
  17. Verkruysse W., Svaasand L. O., Nelson J. S. Remote Plethysmographic Imaging Using Ambient Light, Opt. Express, 2008, vol. 16, no. 26, pp. 21434—21445.
  18. Haan G. De, Jeanne V. Robust Pulse-Rate from Chrominance-Based rPPG, IEEE Trans. Biomed. Eng., 2013, pp. 1—9.
  19. Amelard R., Clausi D. A., Wong A. Spectral-Spatial Fusion Model for Robust Blood Pulse Waveform Extraction in Photoplethysmographic Imaging, Biomed. Opt. Express, 2016, vol. 7, no. 12, pp. 4874—4885.
  20. McDuff D. Deep Super Resolution for Recovering Physiological Information from Videos, IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2018, pp. 1367—1374.
  21. Teplov V. et al. Ambiguity of Mapping the Relative Phase of Blood Pulsations, Biomed. Opt. Express, 2014, vol. 5, no. 9, pp. 3123—3139.
  22. Wu T. PPGI: New Development in Noninvasive and Contactless Diagnosis of Dermal Perfusion Using Near Infrared Light, J. GCPD eV, 2003, vol. 7, no. 1, pp. 17—24.
  23. Poh M.-Z., McDuff D. J., Picard R. W. Non-contact, Automated Cardiac Pulse Measurements Using Video Imaging and Blind Source Separation, Opt. Express, 2010, vol. 18, no. 10, pp. 10762—10774.
  24. Taranov A. A. et al. Contactless Measurement of Neonatal Contractions, Neonatology, 2015, vol. 3, no. 9, pp. 69—73 (in Russian).
  25. Zaunseder S. et al. Cardiovascular Assessment by Imaging Photoplethysmography — A Review, Biomed. Eng., Biomed. Tech., 2018, vol. 63, no. 06.
  26. Van Luijtelaar R. et al. Automatic RoI Detection for Camera-Based Pulse-Rate Measurement, Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), 2015, vol. 9009, pp. 360—374.
  27. McDuff D. Advancements in Remote Physiological Measurement and Applications in Human-Computer Interaction, SPIE, 2017, vol. 10251, pp. 102510V.
  28. Irani R., Nasrollahi K., Moeslund T. B. Improved Pulse Detection from Head Motions Using DCT, IIIE Int. Conf. Comput. Vis. Theory Appl., 2014, vol. 3, pp. 118—124.
  29. Wieringa F. P., Mastik F., Van Der Steen A. F. W. Con-tactless Multiple Wavelength Photoplethysmographic Imaging: A First Step Toward "spO2 Camera" Technology, Ann. Biomed. Eng., 2005, vol. 33, no. 8, pp. 1034—1041.
  30. Hassan M. A. et al. Heart Rate Estimation Using Facial Video: A Review, Biomed. Signal Process. Control. Elsevier Ltd, 2017, vol. 38, no. October, pp. 346—360.
  31. Macwan R., Benezeth Y., Mansouri A. Remote Photoplethysmography with Constrained ICA Using Periodicity and Chrominance Constraints, Biomed. Eng. Online. BioMed Central, 2018, pp. 1—22.
  32. Bracewell R., Peter B. Kahn. The Fourier Transform and its Applications, Am. J. Phys, 1966, vol. 34, no. 8, pp. 1—712.
  33. Oppenheim A. V., Schafer R. V. Digital Signal Processing, Moscow, Svyaz, 1979, 416 pp. (in Russian).
  34. Franke R. Scattered Data Interpolation: Tests of Some Methods, Math. Comput, 1982, vol. 38, no. 157, pp. 181—200.
  35. Wu, Y. et al. Using Radial Basis Function Networks for Function Approximation and Classification, ISRN Appl. Math., 2012, vol. 2012, pp. 1—34.
  36. Ilyin V. A., Kim G. D. Linear Algebra and Analytic Geometry, Moscow, Moscow University Press, 1998 (in Russian).
  37. Bellman R. Introduction to Matrix Theory, Moscow, Science. Main ed. physical and mathematical literature, 1976 (in Russian).
  38. Widrow B., Hoff M. E. Adaptive Switching Circuits, IRE WESCON Conv. Rec, 1960, vol. 4, no. 1, pp. 96—104.
  39. Kingma D. P., Ba J. L. Adam: A Method for Stochastic Optimization, 3rd Int. Conf. Learn. Represent. ICLR 2015 — Conf. Track Proc., 2015, pp. 1—15.
  40. Zou F. et al. A Sufficient Condition for Convergences of Adam and RMSprop, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2019, Vol. 2019-June, no. 1, pp. 11119—11127.
  41. Park J., Sandberg I. W. Universal Approximation Using Ra­dial-Basis-Function Networks, Neural Comput., 1991, vol. 3, no. 2, pp. 246—257.
  42. Wu H.-Y. et al. Eulerian Video Magnification for Revea­ling Subtle Changes in the World, ACM Trans. Graph., 2012, vol. 31, no. 4, pp. 1—8.
  43. Ruder S. An Overview of Gradient Descent Optimiza­tion, arXiv preprint arXiv:1609.04747, 2016, pp. 1—14, available at: https://arxiv.org/abs/1609.04747 (Accessed 08 May 2024).
  44. Soleymani M. et al. A Multimodal Database for Affect Recognition and Implicit Tagging, IEEE Trans. Affect. Comput., 2012, vol. 3, no. 1, pp. 42—55.
  45. Bobbia S. et al. Unsupervised Skin Tissue Segmentation for Remote Photoplethysmography, Pattern Recognit. Lett. Elsevier B. V., 2019, vol. 124, pp. 82—90.
  46. Li J., Yu Z., Shi J. Learning Motion-Robust Remote Photoplethysmography Through Arbitrary Resolution Videos, Proc. 37th AAAI Conf. Artif. Intell. AAAI 2023, 2023, vol. 37, pp. 1334—1342.
  47. Bengio Y. Practical Recommendations for Gradient-Based Training of Deep Architectures, Neural Networks: Tricks of the Trade, Berlin, Heidelberg, Springer Berlin Heidelberg, 2012, pp. 437—478.
  48. Zhang K. et al. Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks, IEEE Signal Pro­cess. Lett., 2016, vol. 23, no. 10, pp. 1499—1503.
  49. Wang W. et al. Algorithmic Principles of Remote-PPG, IEEE Trans. Biomed. Eng., 2017, vol. 64, no. 7, pp. 1479—1491.


To the contents