Despite being a common method of communication for speech-impaired individuals, sign language often goes unnoticed due to a lack of understanding among the general public and many deaf and dumb individuals. This paper aims to design a mediator system that can translate sign language alphabets, addressing the lack of systematic communication methods for these individuals. The system will help identify the alphabets of Indian Sign Language, enhancing their communication capabilities. In recent years, the neural network has extensively used sign language recognition. The CNN network, which is commonly used in sign language recognition, struggles to achieve optimal accuracy due to the equal importance of each image component. This problem can be solved with attention. The proposed model uses the convolution block attention module (CBAM) to focus on important areas. This helps the model make decisions based on the most relevant data, which makes the CNN network more accurate overall. The model gives 99.89% accuracy on the validation test, while only the CNN model gives 97.21% accuracy.