Accurately assessing workload and promptly identifying abnormal workload states are crucial for aviation safety. Workload assessment methods based on physiological signals are characterized by their objectivity and effectiveness. Different modalities of physiological characteristics and their combinations have varying impacts on the accuracy of workload assessment. However, most of the current research focuses on designing and compare different classifiers to recognize the workload level, few of them pay attention to choose more suitable modalities of physiological signals, especially in operating tasks. Therefore, our paper study on different multimodal physiological data fusions to improve abnormal workload recognition. Taking the flight operating task as an example, the concept of ablation experiments was employed to analyze the assessment effects of different modal physiological features and their combinations. Different combinations of modalities are tested based on Support Vector Machine (SVM), CatBoost, and K-Nearest Neighbors (KNN) respectively to find the optimal combination. The study finds that in specific task scenarios, the evaluation performance of the combination of ECG, eye movement, and respiratory modal features on different classifiers reached a high level.