With the evolution of biometric recognition technology, palmprint and palm vein fusion recognition have gained significant attention due to their high stability, accuracy, and security. However, existing multimodal biometrics systems encounter challenges in real-world application scenarios, particularly modality absence resulting from device limitations, sensor malfunctions, environmental interference, and suboptimal user interaction. To address this challenge, we propose a novel Graph Contrastive Learning with Mutual Information Maximization (GCL-MIM) framework that effectively mitigates modality absence and enhances clustering performance in the incomplete palmprint and palm vein fusion recognition task. Specifically, multimodal graphs are constructed based on nearest neighbors to identify similar samples of available modalities and then propagated to the missing modality to establish corresponding relational structures on the incomplete data. Following this, the proposed GCL-MIM explores the intra-modal graph contrastive learning and inter-modal graph consistency learning to maximize mutual information across different modalities. To this end, we introduce a mutual information maximization loss function, which effectively strengthens the adaptability and robustness of the model to incomplete data. Comparative experiments on the publicly available dataset show that the proposed approach achieves higher recognition accuracy and clustering performance when dealing with incomplete multimodal data.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Graph Contrastive Learning with Mutual Information Maximization for Incomplete Multimodal Recognition

  • Jinlong Li,
  • Jianian Hu,
  • Yuqi Wang,
  • Hao Yang,
  • Shuyi Li,
  • Lifang Wu

摘要

With the evolution of biometric recognition technology, palmprint and palm vein fusion recognition have gained significant attention due to their high stability, accuracy, and security. However, existing multimodal biometrics systems encounter challenges in real-world application scenarios, particularly modality absence resulting from device limitations, sensor malfunctions, environmental interference, and suboptimal user interaction. To address this challenge, we propose a novel Graph Contrastive Learning with Mutual Information Maximization (GCL-MIM) framework that effectively mitigates modality absence and enhances clustering performance in the incomplete palmprint and palm vein fusion recognition task. Specifically, multimodal graphs are constructed based on nearest neighbors to identify similar samples of available modalities and then propagated to the missing modality to establish corresponding relational structures on the incomplete data. Following this, the proposed GCL-MIM explores the intra-modal graph contrastive learning and inter-modal graph consistency learning to maximize mutual information across different modalities. To this end, we introduce a mutual information maximization loss function, which effectively strengthens the adaptability and robustness of the model to incomplete data. Comparative experiments on the publicly available dataset show that the proposed approach achieves higher recognition accuracy and clustering performance when dealing with incomplete multimodal data.