Breast cancer can be considered one of the fatal disorders distinguished by unusual and uncontrolled development of breast cells, which requires early and precise detection in order to be effectively treated. Ultrasound image is frequently used for breast cancer screening as it is widely available and non- invasive. Diagnostic difficulties may arise from the subjective and variable manual interpretation of ultrasound pictures. In this work, we suggest a Deep Learning Model utilizing EfficientNetB4 for automated classification of ultrasound images of breast cancer. There are 647 images in the collection that have been categorized as either benign or malignant. Binary masking, histogram equalization, and grayscale conversion are some of the steps to improve feature extraction. The proposed model outperforms traditional CNN architectures achieving an accuracy of 90.24% on the testing dataset after being trained with a transfer learning approach. The model indicates exceptional sensitivity in identifying malignant cases, decreased the rate of false-negative outcomes, and increased diagnosis accuracy. To ensure efficient model training, accurate iterative improvement, and consistent performance improvement, learning rate scheduling and check pointing are used. The experimental results demonstrate how well EfficientNetB4 performs feature extraction and classification, making it a potentially useful tool to help radiologists diagnose breast cancer. By providing a better approach for identifying and categorizing breast cancer, this research promotes the use of deep learning in the field of medical imaging.