Occluded Face Recognition Algorithm Based on Fine-Grained Deep Feature Mask Estimation

WANG Fuping; WANG Dingsha; LI Ou; LIU Weihua; LIU Hongwei

doi:10.7652/xjtuxb202502017

您当前的位置：

首页 >

文章列表页 >

Occluded Face Recognition Algorithm Based on Fine-Grained Deep Feature Mask Estimation

更新时间：2026-01-06

- Occluded Face Recognition Algorithm Based on Fine-Grained Deep Feature Mask Estimation
- Journal of Xi’an Jiaotong University Vol. 59, Issue 2, Pages: 170-179(2025)
- 作者机构：
  
  西安邮电大学通信与信息工程学院， 710121，西安
- 作者简介：
- 基金信息：
- DOI：10.7652/xjtuxb202502017
  CLC： TP391
- Received：11 August 2024，
  
  Online First：30 October 2024，
  
  Published：10 February 2025
- 稿件说明：
移动端阅览
WANG Fuping, WANG Dingsha, LI Ou, et al. Occluded Face Recognition Algorithm Based on Fine-Grained Deep Feature Mask Estimation[J]. Journal of Xi’an Jiaotong University, 2025, 59(2): 170-179.
DOI：

WANG Fuping, WANG Dingsha, LI Ou, et al. Occluded Face Recognition Algorithm Based on Fine-Grained Deep Feature Mask Estimation[J]. Journal of Xi’an Jiaotong University, 2025, 59(2): 170-179. DOI： 10.7652/xjtuxb202502017.

摘要

针对人脸遮挡产生面部结构信息丢失，从而导致人脸识别准确率降低的问题，提出了一种细粒度深度特征掩码估计的遮挡人脸识别算法。首先，将人脸图像输入特征金字塔网络中，从而得到多尺度深度语义特征；其次，将从特征金字塔网络提取的特征经过空洞卷积处理后，与MobileNetV3网络提取的精细浅层特征进行融合，并以像素级二值掩码为标签训练网络以获得细粒度特征掩码；进而，利用该深度特征掩码与深层特征相乘，以抑制由遮挡产生的干扰特征，获得更准确的人脸表征；最后，采用余弦损失和掩码估计损失联合训练网络，提高遮挡人脸识别算法的性能。在LFW数据集基础上创建了口罩、围巾和中心遮挡3种类型的人脸遮挡数据集，实验结果表明：在不同的数据集上，所提算法与现有算法相比均具有更高的识别准确率，并在不同类型遮挡情况下均能获得十分稳定的人脸识别结果；所提算法在数据集LFW和LFW口罩遮挡上的识别准确率分别达到了99.38%和98.42%，在数据集LFW围巾遮挡和LFW中心遮挡上的识别准确率分别达到了98.72%和98.65%，均优于对比算法。

Abstract

To solve the problem of accuracy decrease in facial recognition caused by the loss of facial structural information in the case of facial occlusion

an occluded face recognition algorithm based on fine-grained deep feature mask estimation was proposed. Firstly

the face image was fed into feature pyramid network (FPN) to obtain multi-scale deep semantic features. Next

the features extracted from the FPN were processed by the atrous convolution and fused with shallow features extracted using MobileNetV3; a pixel-wise binary mask was used as label to train the network to obtain the fine-grained deep feature mask; this deep feature mask was multiplied with the deep features to suppress the corrupted feature produced by occlusion and to obtain the better face representation. Finally

CosFace loss and mask estimation loss were jointly used to train the network to improve the performance of occluded face recognition algorithm. Three face occlusion datasets with mask

scarf

and center occlusion were created based on the LFW dataset

respectively. The experimental results show that on four different datasets

the proposed algorithm was more accurate than existing algorithms and obtained very stable face recognition results under different occlusion situations. The recognition accuracy on LFW and LFW-mask-occlusion datasets reached 99.38% and 98.42%

respectively and that on LFW-scarf-occlusion and LFW-center-occlusion datasets reached 98.72% and 98.65%

respectively

outperforming the algorithms compared.

关键词

Keywords

references

李明 , 党青霞 . 融合Transformer和CNN的轻量级人脸识别算法 [J ] . 计算机工程与应用 , 2024 , 60 ( 14 ): 96 - 104 .

LI Ming , DANG Qingxia . Lightweight face recognition algorithm combining transformer and CNN [J ] . Computer Engineering and Applications , 2024 , 60 ( 14 ): 96 - 104 .

岳也 , 温瑞萍 , 王川龙 . 带有特征信息卷积神经网络的人脸识别算法 [J ] . 工程数学学报 , 2024 , 41 ( 3 ): 410 - 420 .

YUE Ye , WEN Ruiping , WANG Chuanlong . Face recognition algorithm based on convolutional neural network with feature information [J ] . Chinese Journal of Engineering Mathematics , 2024 , 41 ( 3 ): 410 - 420 .

余璀璨 , 李慧斌 . 基于深度学习的人脸识别方法综述 [J ] . 工程数学学报 , 2021 , 38 ( 4 ): 451 - 469 .

YU Cuican , LI Huibin . Deep learning based 2D face recognition: a survey [J ] . Chinese Journal of Engineering Mathematics , 2021 , 38 ( 4 ): 451 - 469 .

TOMAR V , KUMAR N , SRIVASTAVA A R . Single sample face recognition using deep learning: a survey [J ] . Artificial Intelligence Review , 2023 , 56 ( 1 ): 1063 - 1111 .

JEEVAN G , ZACHARIAS G C , NAIR M S , et al . An empirical study of the impact of masks on face recognition [J ] . Pattern Recognition , 2022 , 122 : 108308 .

WRIGHT J , YANG A Y , GANESH A , et al . Robust face recognition via sparse representation [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2009 , 31 ( 2 ): 210 - 227 .

ZHOU Zihan , WAGNER A , MOBAHI H , et al . Face recognition with contiguous occlusion using Markov random fields [C ] // 2009 IEEE 12th International Conference on Computer Vision . Piscataway, NJ, USA : IEEE , 2009 : 1050 - 1057 .

张铮 , 芦天亮 , 曹金璇 . 基于分割和多级掩膜学习的遮挡人脸识别方法 [J ] . 计算机科学与探索 , 2024 , 18 ( 7 ): 1814 - 1825 .

ZHANG Zheng , LU Tianliang , CAO Jinxuan . Occluded face recognition based on segmentation and multi-stage mask learning [J ] . Journal of Frontiers of Computer Science and Technology , 2024 , 18 ( 7 ): 1814 - 1825 .

CHEN Yuan , CHEN Weiche , WEI C P , et al . Occlusion-aware face inpainting via generative adversarial networks [C ] // 2017 IEEE International Conference on Image Processing (ICIP) . Piscataway, NJ, USA : IEEE , 2017 : 1202 - 1206 .

ZHAO Fang , FENG Jiashi , ZHAO Jian , et al . Robust LSTM-autoencoders for face de-occlusion in the wild [J ] . IEEE Transactions on Image Processing , 2018 , 27 ( 2 ): 778 - 790 .

SONG Lingxue , GONG Dihong , LI Zhifeng , et al . Occlusion robust face recognition based on mask learning with pairwise differential Siamese network [C ] // 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway, NJ, USA : IEEE , 2019 : 773 - 782 .

QIU Haibo , GONG Dihong , LI Zhifeng , et al . End2end occluded face recognition by masking corrupted features [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022 , 44 ( 10 ): 6939 - 6952 .

HAO Shaozhe , CHEN Chaofeng , CHEN Zhenfang , et al . A unified framework for masked and mask-free face recognition via feature rectification [C ] // 2022 IEEE International Conference on Image Processing (ICIP) . Piscataway, NJ, USA : IEEE , 2022 : 726 - 730 .

WANG Hao , WANG Yitong , ZHOU Zheng , et al . CosFace: large margin cosine loss for deep face recognition [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway, NJ, USA : IEEE , 2018 : 5265 - 5274 .

LIN T Y , DOLLÁR P , GIRSHICK R , et al . Feature pyramid networks for object detection [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway, NJ, USA : IEEE , 2017 : 936 - 944 .

史加荣 , 王丹 , 尚凡华 , 等 . 随机梯度下降算法研究进展 [J ] . 自动化学报 , 2021 , 47 ( 9 ): 2103 - 2119 .

SHI Jiarong , WANG Dan , SHANG Fanhua , et al . Research advances on stochastic gradient descent algorithms [J ] . Acta Automatica Sinica , 2021 , 47 ( 9 ): 2103 - 2119 .

YI Dong , LEI Zhen , LIAO Shengcai , et al . Learning face representation from scratch [EB/OL ] . ( 2014-11-28 ) [ 2024-08-01 ] . https://arxiv.org/abs/1411.7923 https://arxiv.org/abs/1411.7923 .

HUANG G B , MATTAR M , BERG T , et al . Labeled faces in the wild: a database for studying face recognition in unconstrained environments [C ] // Workshop on Faces in ‘Real-Life’ Images: Detection, Alignment, and Recognition . Marseille, France : Inria , 2008 : 617 - 624 .

LIU Weiyang , WEN Yandong , YU Zhiding , et al . SphereFace: deep hypersphere embedding for face recognition [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway, NJ, USA : IEEE , 2017 : 6738 - 6746 .

CHEN Sheng , LIU Yang , GAO Xiang , et al . MobileFaceNets: efficient CNNs for accurate real-time face verification on mobile devices [C ] // Biometric Recognition . Cham : Springer International Publishing , 2018 : 428 - 438 .

LIU Jingtuo , DENG Yafeng , BAI Tao , et al . Targeting ultimate accuracy: face recognition via deep embedding [EB/OL ] . ( 2015-07-23 ) [ 2024-08-01 ] . https://arxiv.org/abs/1506.07310 https://arxiv.org/abs/1506.07310 .

WEN Yandong , ZHANG Kaipeng , LI Zhifeng , et al . A discriminative feature learning approach for deep face recognition [C ] // Computer Vision-ECCV 2016 . Cham : Springer International Publishing , 2016 : 499 - 515 .

TAIGMAN Y , YANG Ming , RANZATO M , et al . DeepFace: closing the gap to human-level performance in face verification [C ] // 2014 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway, NJ, USA : IEEE , 2014 : 1701 - 1708 .

PARKHI O M , VEDALDI A , ZISSERMAN A . Deep face recognition [C ] // BMVC 2015-Proceedings of the British Machine Vision Conference 2015 . Swansea, UK : BMVA , 2015 : 1 - 12 .

DENG Jiankang , GUO Jia , XUE Niannan , et al . ArcFace: additive angular margin loss for deep face recognition [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway, NJ, USA : IEEE , 2019 : 4685 - 4694 .

KIM M , JAIN A K , LIU Xiaoming . AdaFace: quality adaptive margin for face recognition [C ] // 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway, NJ, USA : IEEE , 2022 : 18729 - 18738 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

An Algorithm for Multimodal Biometric Recognition Based on Feature Level and the Second-Generation Curvelet Transform

Face Recognition with Features Extraction Based on Particle Swarm Optimization

A Fast Implementation of Null Space Based Algorithm

Method for Face Recognition Using Second-Generation Curvelet Transform and Back Propagation Neural Network

Numerical Simulation of the Energy Conversion Process in Thermoacoustic Engines with Different Stack Materials

Related Author

许学斌 1

张德运 1

张新曼 1

张铁登 2

邓万宇 1

温浩 1

郭崇慧 2

卢桂馥 1

Related Institution

School of Electronics and Information Engineering, Xi'an Jiaotong University

State Key Laboratory of Integrated Service Networks, Xidian University

School of Computer Science and Information, Anhui Polytechnic University

School of Computer Science and Technology, Nanjing University of Science and Technology

⁰