By separating the control plane and data plane, software-defined networking (SDN) offers programmability, adaptability, and flexibility, improving network performance and reducing service providers’ costs. However, with limited computational resources available to the controller, separating the control and data planes in SDN implementations raises scalability concerns. To address these challenges, we developed an expandable hybrid switching system using machine learning methods to enhance SDN flexibility. We propose the flexible hybrid routing with reinforcement learning and deep learning (fHRRLDL) framework, which integrates an SDN OpenFlow model switch with conventional switching. Using the Q-learning technique, we apply a reinforcement algorithm to explore different paths randomly and select the best one. This model-free reinforcement learning periodically updates the Q-values in the Q-table for each transition in the network environment state using the Markov decision process and Bellman’s equation until the Q-function converges to the optimal Q-values. The reinforcement learning agent is instructed to choose the best Q-values from the Q-table using the greedy method. We compare the proposed mixed switching architecture with the typical SDN OpenFlow switch regarding network performance parameters such as central processing unit (CPU) load, latency, average throughput, and packet exchange transmission rates. Our quantitative analysis of the test data reveals that the hybrid switch exchanged over 60% more packets than the OpenFlow switch. The hybrid switching routing scheme produces superior throughput, as evidenced by the average throughput statistics. The OpenFlow switch has an average CPU load that is 20% higher than the hybrid switch. Our analysis of the simulation data demonstrates that the hybrid switch with deep learning (DL) performs significantly better than with reinforcement learning (RL), thanks to the fHRRLDL system, which leverages both deep-learning- and Q-learning-based reinforcement learning. Therefore, we are convinced that scaling issues in SDN controller network architecture can be effectively addressed by implementing a hybrid switching model that includes machine learning methods, particularly in data center environments where high switching rates are essential.