DOI: 10.17587/prin.16.240-251

Improving the Accuracy of Facial Geometry Reconstruction in Dual-Camera Systems

A. D. Borisov, Postgraduate Student, radiatus@yandex.ru, S. D. Makhortov, Professor, Head of Department of Programming and Information Technologies, msdexp@yandex.ru, Voronezh State University, Voronezh, 394018, Russia Federation

Corresponding author: Aleksandr D. Borisov, Postgraduate Student, Voronezh State University, Voronezh, 394018, Russia Federation, E-mail: radiatus@yandex.ru

The article presents a method for facial geometry reconstruction that combines a local blendshape model with a geometrically controlled As-Rigid-As-Possible (ARAP) model. The local model provides a rough approximation of the facial shape using segmentation and regularization, while ARAP refines local details, enhancing consistency and detailization. Experimental results demonstrated a reduction in errors and an improvement in reconstruction quality compared to traditional methods. The proposed method is versatile and applicable to both limited and multi-camera systems, making it promising for the film industry and virtual reality.

Keywords: facial geometry reconstruction, local blendshape model, photometric energy, stereo constraints

pp. 240—251

For citation:
Borisov A. D., Makhortov S. D. Improving the Accuracy of Facial Geometry Reconstruction in Dual-Camera Systems, Programmnaya Ingeneria, 2025, vol. 16, no. 5, pp. 240—251. DOI: 10.7587.prin.16.240-251 (in Russian).

References:

1. Xu H., Dai L., Fu J., Wang X., Wang Q. High-Quality Real Time Facial Capture Based on Single Camera, arXiv preprint, 2021. DOI: 10.48550/arXiv.2111.07556.
2. Peres V. M. X., Dal Molin G. P., Musse S. R. Can We Truly Transfer an Actor's Genuine Happiness to Avatars? An Inves­tigation into Virtual, Real, Posed, and Spontaneous Faces, arXiv preprint, 2023. DOI: 10.48550/arXiv.2312.02128.
3. Li H., Yu J., Ye Y., Bregler C. Realtime facial animation with on-the-fly correctives, ACM Transactions on Graphics, 2013, vol. 32, no. 4, article 42. DOI: 10.1145/2461912.2462019.
4. Beeler T., Bickel B., Beardsley P. et al. High-quality singleshot capture of facial geometry, ACM Transactions on Graphics, 2010, vol. 29, no. 4, Article 40. DOI: 10.1145/1778765.1778777.
5. Fyffe G., Jones A., Alexander O. et al. Driving high-resolution facial scans with video performance capture, ACM Transactions on Graphics, 2014, vol. 34, no. 1, article 8. DOI: 10.1145/2638549.
6. Lewis J. P., Anjyo K., Rhee T. et al. Practice and Theory of Blendshape Facial Models, Eurographics 2014 — State of the Art Reports, 2014, pp. 199—218. DOI: 10.2312/egst.20141042.
7. Pighin F., Hecker J., Lischinski D. et al. Synthesizing Realistic Facial Expressions from Photographs, Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '98),1998. P. 75—84. DOI: 10.1145/280814.280825.
8. Galantucci L., Lavecchia F., Percoco G. 3D Face Measurement and Scanning Using Digital Close Range Photogrammetry: Evaluation of Different Solutions and Experimental Approaches, Proceedings of the International Conference on 3D Body Scanning Technologies, 2010, article 34. DOI: 10.15221/10.052.
9. Weise T., Bouaziz S., Li H., Pauly M. Realtime performance-based facial animation, ACM Transactions on Graphics, 2011, vol. 30, no. 4, article 77. DOI: 10.1145/2010324.1964972.
10. Sorkine O., Alexa M. As-Rigid-As-Possible Surface Modeling, Proceedings of the Fifth Eurographics Symposium on Geometry Processing, 2007, pp. 109—116. DOI: 10.2312/SGP/SGP07/109-116.
11. Levi Z., Gotsman C. Smooth Rotation Enhanced As-Rigid-As-Possible Mesh Animation, IEEE Transactions on Visualization and Computer Graphics, 2015, vol. 21, no. 10, pp. 1135—1144. DOI: 10.1109/TVCG.2014.2359463.
12. Serra J., Moser D., McLean D., Hendler D. Simplified Facial Capture with Head-Mounted Cameras, ACM SIGGRAPH 2021 Talks, 2021, article 43. DOI: 10.1145/3450623.3464637.
13. Moser L., Williams M., Roble D. High-Quality CostEffective Facial Motion Capture Pipeline with 3D Regression, ACM SIGGRAPH 2018 Talks, 2018, аrticle 58, 2 p. DOI: 10.1145/3214745.3214755.
14. Danecek R., Black M. J., Bolkart T. EMOCA: Emotion Driven Monocular Face Capture and Animation, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 20279—20290. DOI: 10.1109/CVPR52688.2022.01967.
15. Taigman Y., Yang M., Ranzato M. A., Wolf L. DeepFace: Closing the Gap to Human-Level Performance in Face Verification, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 1701—1708. DOI: 10.1109/CVPR.2014.220.
16. Ma W.-C., Fyffe G., Debevec P. Optimized local blendshape mapping for facial motion retargeting, ACM SIGGRAPH 2011 Talks, 2011, article 58. DOI: 10.1145/2037826.2037903.
17. Borisov A. D., Makhortov S. D. Non-Rigid Registration of a Human Face from Images from a Stereo Camera, Programmnaya Ingeneria, 2023, vol. 14, no. 2, pp. 82—92. DOI: 10.7587/prin.14.82-92 (in Russian).
18. Ferrari C., Berretti S., Pala P., Del Bimbo A. A Sparse and Locally Coherent Morphable Face Model for Dense Semantic Correspondence Across Heterogeneous 3D Faces, arXiv preprint, 2020. DOI: 10.48550/arXiv.2006.03840.
19. Peyre G., Pechaud M., Keriven R., Cohen L. Geodesic Methods in Computer Vision and Graphics, Foundations and Trends in Computer Graphics and Vision, 2009. vol. 5, no. 3—4, pp. 197—397. DOI: 10.1561/0600000029.
20. Marquardt D. W. An Algorithm for Least-Squares Estimation of Nonlinear Parameters, SIAM Journal on Applied Mathematics, 1963, vol. 11, no. 2, pp. 431—441. DOI: 10.1137/0111030.
21. Chai T., Draxler R. R. Root mean square error (RMSE) or mean absolute error (MAE)? — Arguments against avoiding RMSE in the literature, Geoscientific Model Development, 2014, vol. 7, no. 3, pp. 1247—1250. DOI: 10.5194/gmd-7-1247-2014.