<p>An Intrusion Detection System (IDS) is a commonly employed security mechanism for detecting, mitigating, and minimizing the impact of concealed and unrecognized intrusions in the Internet of Things (IoT). This study proposes a novel hybrid Deep Maxout–Quantum Neural Network (Maxout–QNN) intrusion detection system to address high-dimensional data, class imbalance, slow convergence, and premature stagnation in IoT security applications. Two optimisation-based variants are introduced: Hybrid + Self-Upgraded Cat and Mouse Optimisation (SUCMO) and Hybrid + Seagull-Adopted Elephant Herding Optimisation (SAEHO). The proposed IDS operates through three fundamental stages: preprocessing, feature extraction, and classification. The input data are first subjected to enhanced Z-score normalization during preprocessing. Afterward, relevant features—including statistical and higher-order statistical measures, enhanced entropy-based, and correlation-based attributes—are extracted. Finally, classification is performed using a hybrid Deep Maxout–QNN model based on the extracted features. Existing IoT intrusion detection systems often struggle with high-dimensional data, class imbalance, and limited scalability. To overcome these challenges, this work introduces a hybrid Deep Maxout–QNN model optimized using the SUCMO algorithm. The proposed SUCMO and SAEHO algorithms dynamically adjusts exploration–exploitation parameters to accelerate convergence and prevent premature stagnation. This mechanism distinguishes SUCMO and SAEHO from conventional metaheuristics, ensuring faster convergence and higher accuracy. The proposed Hybrid + SUCMO model achieved 96.65% accuracy and 93.68% F-measure on the UNSW-NB15 dataset at 90% learning, demonstrating superior performance compared to traditional optimizers such as Rock Hyraxes Swarm Optimization (RHSO), Butterfly Optimization Algorithm (BOA), and Salp Swarm Optimization Algorithm (SSOA). The proposed SAEHO algorithm also performs better with same experimental settings.</p>