<p>Brain tumors are a serious and increasingly prevalent condition resulting from the abnormal growth of brain cells. Early detection is crucial, as it enables timely treatment initiation and improves patient outcomes. In the initial stage, the tumor is small in size, so early detection makes it easier to treat. Also, it improves the patient survival rate, minimizes serious health issues and reduces the cost of treatment. Moreover, it helps doctors plan better treatment and avoid damage to important parts of the brain. Thus, early brain tumor detection leads to faster treatment, better results, and higher survival rates. Manual detection and classification of brain tumors can be labor-intensive and difficult for healthcare practitioners. To improve accuracy and enable quicker, more consistent diagnoses, computer-aided diagnosis (CAD) systems are increasingly utilized. Moreover, the segmentation process of brain tumors using Magnetic Resonance Images (MRI) is a more challenging process because of dissimilarity in positions, shapes, and image intensities. Thus, a hybrid optimization-driven deep learning algorithm is implemented for brain tumor classification. Here, a Recurrent Residual Convolutional Neural Network using U-Net (R2U-Net) is designed for segmentation, whereas a Deep Q Network (DQN) is applied for the detection of brain tumor. The developed optimized deep learning technique attained improved performance than other existing methods, such as Artificial Neural Network (ANN) + Fuzzy K-means algorithm, hybrid deep autoencoder + Bayesian Fuzzy Clustering (BFC), Dolphin-Sine Cosine Algorithm (SCA)-based Deep Convolution Neural Network (CNN), multiscale CNN, CNN, Support Vector Machine (SVM), Patch-Based Vision Transformer (PBVit), and Densely Connected Convolutional Network (DenseNet) with respect to testing accuracy of 0.9517, sensitivity of 0.9294, and specificity of 0.9583.</p>