Enhanced face recognition in unconstrained environments: leveraging 3D morphable model and focal modulation network
DOI:
https://doi.org/10.38152/bjtv8n1-007Abstract
Due to the fact that faces can vary significantly in appearance when viewed from different angles or orientations, face recognition systems can easily be impacted by this issue. In this paper, we propose a novel approach for face frontalization leveraging a 3D morphable model (3DMM) and advanced deep learning techniques. Our architecture comprises two main modules: the 3D face fitting module, where we redesign this strategy using a focal modulation network-based encoder, and the texture completion module, which recovers the unseen regions caused by self-occlusion. Experimental results demonstrate the effectiveness of our method in producing high-quality frontalized face images across different poses, improving the performance of face recognition in unconstrained environments.
References
BLANZ, V.; VETTER, T. Face recognition based on fitting a 3D morphable model. IEEE Transactions on pattern analysis and machine intelligence, v. 25, n. 9, p. 1063–1074, 2003.
GOODFELLOW, I. et al. Generative adversarial nets. Advances in neural information processing systems, v. 27, 2014.
DENG, Y. et al. Accurate 3d face reconstruction with weakly-supervised learning: From single image to image set. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. Anais...2019.
TEWARI, A. et al. Mofa: Model-based deep convolutional face autoencoder for unsupervised monocular reconstruction. Proceedings of the IEEE international conference on computer vision workshops. Anais...2017.
GECER, B. et al. Ganfit: Generative adversarial network fitting for high fidelity 3d face reconstruction. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Anais...2019.
HE, K. et al. Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition. Anais...2016.
KRIZHEVSKY, A.; SUTSKEVER, I.; HINTON, G. E. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, v. 25, 2012.
PARKHI, O.; VEDALDI, A.; ZISSERMAN, A. Deep face recognition. BMVC 2015-Proceedings of the British Machine Vision Conference 2015. Anais...British Machine Vision Association, 2015.
YANG, J. et al. Focal modulation networks. Advances in Neural Information Processing Systems, v. 35, p. 4203–4217, 2022.
HUANG, R. et al. Beyond face rotation: Global and local perception gan for photorealistic and identity preserving frontal view synthesis. Proceedings of the IEEE international conference on computer vision. Anais...2017.
YIN, Y. et al. Dual-attention GAN for large-pose face frontalization. 2020 15th IEEE international conference on automatic face and gesture recognition (FG 2020). Anais...IEEE, 2020.
CEN, S. et al. Pre-Trained Feature Fusion and Multidomain Identification Generative Adversarial Network for Face Frontalization. IEEE Access, v. 10, p. 77872–77882, 2022.
CAO, J. et al. Face frontalization with deep GAN via multi-attention mechanism. Signal, Image and Video Processing, v. 17, n. 5, p. 1965–1973, jul. 2023.
KAS, M. et al. Generative adversarial networks for 2D-based CNN pose-invariant face recognition. International Journal of Multimedia Information Retrieval, v. 11, n. 4, p. 639–651, dez. 2022.
HU, Y. et al. Pose-guided photorealistic face rotation. Proceedings of the IEEE conference on computer vision and pattern recognition. Anais...2018.
YIN, X. et al. Towards large-pose face frontalization in the wild. Proceedings of the IEEE international conference on computer vision. Anais...2017.
ZHAO, J. et al. Towards pose invariant face recognition in the wild. Proceedings of the IEEE conference on computer vision and pattern recognition. Anais...2018.
ZHOU, H. et al. Rotate-and-render: Unsupervised photorealistic face rotation from single-view images. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Anais...2020.
ZHU, X. et al. Face alignment in full pose range: A 3d total solution. IEEE transactions on pattern analysis and machine intelligence, v. 41, n. 1, p. 78–92, 2017.
GECER, B.; DENG, J.; ZAFEIRIOU, S. Ostec: One-shot texture completion. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Anais...2021.
DENG, J. et al. Uv-gan: Adversarial facial uv map completion for pose-invariant face recognition. Proceedings of the IEEE conference on computer vision and pattern recognition. Anais...2018.
LIU, Z. et al. Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF international conference on computer vision. Anais...2021.
DOSOVITSKIY, A. et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
BULAT, A.; TZIMIROPOULOS, G. How far are we from solving the 2d & 3d face alignment problem?(and a dataset of 230,000 3d facial landmarks). Proceedings of the IEEE international conference on computer vision. Anais...2017.
DENG, J. et al. Arcface: Additive angular margin loss for deep face recognition. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Anais...2019.
LIU, Z. et al. Deep Learning Face Attributes in the Wild. Proceedings of International Conference on Computer Vision (ICCV). Anais...dez. 2015.
HUANG, G. B. et al. Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. Workshop on faces in’Real-Life’Images: detection, alignment, and recognition. Anais...2008.
SENGUPTA, S. et al. Frontal to profile face verification in the wild. 2016 IEEE winter conference on applications of computer vision (WACV). Anais...IEEE, 2016.
GUO, Y. et al. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition. Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14. Anais...Springer, 2016.
PAYSAN, P. et al. A 3D face model for pose and illumination invariant face recognition. 2009 sixth IEEE international conference on advanced video and signal based surveillance. Anais...Ieee, 2009.
CAO, C. et al. Facewarehouse: A 3d facial expression database for visual computing. IEEE Transactions on Visualization and Computer Graphics, v. 20, n. 3, p. 413–425, 2013.
HASSNER, T. et al. Effective face frontalization in unconstrained images. Proceedings of the IEEE conference on computer vision and pattern recognition. Anais...2015.
ZHU, X. et al. High-fidelity pose and expression normalization for face recognition in the wild. Proceedings of the IEEE conference on computer vision and pattern recognition. Anais...2015.
CAO, J. et al. Learning a high fidelity pose invariant model for high-resolution face frontalization. Advances in neural information processing systems, v. 31, 2018.
