The most occurring cancer worldwide is breast cancer in which abnormal breast cells grow and form tumors. If left unchecked, it can be more fatal and can spread throughout the body. Globally, 2.3 million cases have been diagnosed and there were 685,000 fatalities in 2020, according to the World Health Organization (WHO). To overcome the fatalities, an early diagnosis is the best approach. Manual diagnosis is not an easy process for this cancer, using mammography images, and always requires an expert person. In this paper, we have used a VGG16-based model to eradicate the feature from the intermediate layers and concatenate the intermediate layers. Then, we used a Convolutional Neural Network (CNN) to train the feature and predict the accuracy and classification. We have evaluated our performance analysis with confusion matrix analysis, ROC curve analysis, and other methods. In our results, we have accomplished an accuracy of 94% among the datasets. Comparing our suggested framework to state-of-the-art (SOTA) technology demonstrates that it increased accuracy, and additional study may enable it to be enhanced to decrease false positives and false negatives in screening mammography findings.