In modern communication systems, signal recognition is a key technology for spectrum management, information security, and communication efficiency improvement. Nevertheless, conventional signal recognition approaches depend largely on manually designed feature extraction and fixed decision rules, which limits their ability to address complex and evolving signal environments. In recent years, the rapid advancements in machine learning have introduced novel possibilities for the field of signal recognition (Li et al., 2019). This study proposes a machine learning-based communication signal recognition method aimed at automatically extracting signal features and efficiently classifying specific communication signals (Liao et al., 2020). We constructed a comprehensive signal dataset encompassing multiple modulation types (e.g., AM, FM, PSK, QAM) and covering a range of signal-to-noise ratio (SNR) levels, combining synthetic signals with real measured signals to ensure the model’s generalization and robustness. Based on this, we designed a hybrid neural network model integrating Convolutional Neural Networks (CNNs) to extract signal spectral features alongside Recurrent Neural Networks (RNNs) to model the temporal dynamics of signals. Experimental findings indicate that the model attains an average recognition rate of over 95% in high SNR (SNR ≥ 7 dB) environments, and maintains an accuracy of over 80% even in low SNR (SNR ≤ 3 dB) conditions. Furthermore, we verified the model’s adaptability through transfer learning, where it rapidly adapts to new signal types with only a small number of samples. In conclusion, the machine learning-based communication signal recognition method demonstrates significant advantages in performance, robustness, and adaptability, providing a new technological approach for advancing the design and enhancing the optimization of next-generation intelligent communication systems.