A flexible multimodal motor fault diagnosis framework with inter-modal learning discrepancy mitigation under modal missing scenarios
摘要
Multimodal learning has recently gained considerable attention in fault diagnosis for its ability to integrate multisource heterogeneous data and extract comprehensive fault features. However, existing multimodal fault diagnosis studies have the following limitations: (1) existing multimodal methods lack flexibility in handling arbitrary modality combinations under missing-modality conditions, thereby limiting their applicability in practical industrial environments; (2) disparities in data distributions and feature scales across modalities often lead to imbalanced convergence during training, hindering the realization of its theoretical potential. To address these issues, a flexible multimodal framework with inter-modal learning discrepancy mitigation is proposed for motor fault diagnosis under modal missing scenarios. Firstly, a flexible sparse mixture of experts framework is designed to adaptively integrate arbitrary modality combinations while maintaining robustness against missing data. Then, a discrepancy-aware alignment scheme grounded in contrastive learning is introduced to harmonize fault-category representations across modalities, bridging heterogeneous features and mitigating inter-modal learning gaps. Finally, regularization-guided synergistic loss optimization is developed to dynamically integrate unsupervised contrastive learning with multimodal representation learning, which enables more effective cross-modal feature fusion. The effectiveness of the proposed method is validated through simulated motor fault experiments, and its superiority is demonstrated by comparisons with advanced approaches for multimodal missing and imbalance scenarios.