Automatic signal modulation recognition in AI-based wireless communication systems can be implemented using combinatorial deep learning neural network techniques to improve resource shortage and spectrum utilization efficiency for dynamic spectrum allocation. An automatic signal modulation classification model using combinatorial deep learning technique is presented. The proposed deep learning model increase accuracy for low signal to noise ratio SNR and maintain a high classification accuracy for high SNR signals. Using a hybrid deep learning model combining both ConvLSTM2D with Transformer-block neural networks, the proposed modulation classifier architecture can learn the signal for both low and high SNR and get better accuracy for signals with high noise.