Multimodal biometric recognition technology has garnered significant attention in recent years owing to its benefits in security and recognition precision compared to traditional single biometric recognition methods, the fusion of multiple biometric features can effectively mitigate the constraints of individual features, reducing issues such as recognition failure caused by forgery or inability to capture a single biometric trait. Given that scenarios including missing modalities may occur, most approaches primarily address missing modalities as independent tasks, ignoring the interrelationships among different combinations of missing modalities and insufficiently adapting to varied input modalities. Thus, we introduce a progressive adversarial learning framework that enables the joint utilization of various missing modality combinations. Specifically, a semantic consistency learning is developed, employing an adversarial technique for arbitrarily missing modality combinations. Additionally, a progressive modality completion learning network is utilized, incorporating arbitrary combinations of missing modalities with complete modality combinations for adversarial learning. This progressively modifies the interrelation across modality combinations, enabling the model to acquire multi-modal semantic information that resembles a complete modality combination. Extensive experiments have been performed to validate the effectiveness of the proposed method.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Progressive Adversarial Learning for Multi-modality Biometric Recognition with Missing Modality

  • Hai Yuan,
  • Xiao Yang,
  • Jun Wang,
  • Zhengwen Shen,
  • Zaiyu Pan

摘要

Multimodal biometric recognition technology has garnered significant attention in recent years owing to its benefits in security and recognition precision compared to traditional single biometric recognition methods, the fusion of multiple biometric features can effectively mitigate the constraints of individual features, reducing issues such as recognition failure caused by forgery or inability to capture a single biometric trait. Given that scenarios including missing modalities may occur, most approaches primarily address missing modalities as independent tasks, ignoring the interrelationships among different combinations of missing modalities and insufficiently adapting to varied input modalities. Thus, we introduce a progressive adversarial learning framework that enables the joint utilization of various missing modality combinations. Specifically, a semantic consistency learning is developed, employing an adversarial technique for arbitrarily missing modality combinations. Additionally, a progressive modality completion learning network is utilized, incorporating arbitrary combinations of missing modalities with complete modality combinations for adversarial learning. This progressively modifies the interrelation across modality combinations, enabling the model to acquire multi-modal semantic information that resembles a complete modality combination. Extensive experiments have been performed to validate the effectiveness of the proposed method.