This paper introduces a novel methodology for rainfall-runoff modeling utilizing transformer-based deep learning architectures with a focus on different optimization strategies. Hydrological modeling remains a challenging task, particularly in capturing the complex temporal relationships between meteorological variables and runoff behavior across diverse watershed conditions. We propose a transformer neural network that employs multi-head self-attention mechanisms to effectively learn long-range dependencies in hydro-meteorological time series data. A comparative analysis is conducted between the conventional Adam optimizer and Rectified Adam (RAdam), the latter of which is designed to address early-stage training instability. Using a comprehensive dataset comprising rainfall, temperature, and discharge data, our findings demonstrate that the transformer model optimized with RAdam achieves superior performance, yielding lower mean squared errors and higher R \(^{{\textbf {2}}}\) scores compared to traditional methods. The model is capable of accurately representing both short-term dynamics and long-term seasonal trends in runoff, indicating its strong potential for flood forecasting applications. This work contributes to the expanding domain of deep learning in hydrology and underscores the importance of optimizer selection in complex environmental modeling scenarios.