Invasive Ductal Carcinoma (IDC) continues to be a major global health concern. It significantly impacts mortality rates, particularly among individuals diagnosed at later stages with limited access to advanced diagnostic tools. This research focuses on enhancing histopathological image analysis for the diagnosis of Invasive Ductal Carcinoma (IDC) in breast cancer therefore early and accurate diagnosis is critical, as delayed intervention can lead to severe complications by leveraging deep learning models and attention mechanisms. The study aims to bridge gaps in diagnostic accuracy and clinical decision-making. A novel hybrid model combining Convolutional Neural Network (CNN), Artificial Neural Network (ANN), and Recurrent Neural Network (RNN) architectures is developed, further enhanced with an attention mechanism. This hybrid model achieved an accuracy of ~ 0.84, showing a 6–10% improvement over standalone CNN, ANN and RNN models demonstrating strong effectiveness in identifying clinically relevant patterns in histopathological images. However, dataset imbalance and interpretability issues remain challenges. In future work we can plan to evaluate on applying this model to larger, multi-center datasets and incorporating explainability tools to make the model prediction more transparent for clinical use.