Obstructive sleep apnea (OSA) is a common sleep-disordered breathing (SDB) characterized by recurrent apnea events during sleep due to partial or complete obstruction of the upper airway, which impairs the patient’s quality of sleep and daily life and increases the risk of several chronic diseases. Polysomnography (PSG) is clinically used as the gold standard for detecting OSA, but its expensive, complex and time-consuming procedure limits its widespread use. In this study, we propose a novel pseudo-modal fusion network (PMFNet) using single-lead ECG signals for the task of obstructive sleep apnea. We innovatively propose the concept of “pseudo-modal” in ECG signal analysis, using the QRS wave detection algorithm and the continuous wavelet transform (CWT) to obtain the R-wave data features and time-frequency maps as pseudo-modal data, respectively. Different feature extractors are carefully designed for these two types of pseudo-modal data to complete the time-domain and frequency-domain feature extraction of ECG signals, and a bilinear attention network (BAN) is introduced to effectively integrate our proposed pseudo-modal data. We validate the performance of PMFNet on the publicly available PhysioNet Apnea-ECG dataset and make a fair comparison with previous studies. Extensive experimental results show that PMFNet achieves optimal performance, with specific accuracy, sensitivity, specificity, and F1 scores of 92.79%, 90.90%, 93.95%, and 90.42%, respectively. Our proposed PMFNet is able to determine end-to-end whether an OSA event has occurred for a given ECG signal segment, providing an effective and convenient alternative to current clinical approaches.