<p>Recognition of wafer map defects plays a pivotal role in semiconductor manufacturing operations. As the manufacturing industry advances, wafer map defects increasingly present themselves in a complex, mixed form rather than as isolated occurrences, which brings a certain challenge for accurate recognition of wafer map defects. Existing detection models mainly focus on the single type defect detection tasks, neglecting the mixing of multiple defects. Faced with this issue, this paper proposes a Hybrid Attention-Based Fusion Network, named HABF-Net, based on the Convolutional Neural Network (CNN)-Transformer architecture, for accurate mixed wafer map defect recognition. Firstly, an effective backbone via Octave Convolution is designed, which broadens the model’s receptive field through the synergistic processing of high- and low-frequency information. Simultaneously, an improved transformer structure is employed to perform self-attention modeling for global context extraction. Moreover, proposed network integrates a Hybrid Attention-Guided Fusion (HAGF) block, which effectively harnesses multi-scale feature representations. The efficacy of proposed network is substantiated through a series of qualitative and quantitative analysis experiments, which attains remarkable scores of 98.83%, 98.72%, 98.77%, and 98.79% in Precision, Recall, F1 score, and Accuracy, respectively. Comparative assessments against contemporary state-of-the-art methodologies further demonstrate that proposed network delivers superior classification performance on wafer map defects.</p>