The class activation map (CAM) is useful in identifying significant image features that the convolutional neural network (CNN) model is considering while making the prediction. This is critical especially in medical diagnosis like scenarios. However, existing gradient-based methods like Grad-CAM often produce low-quality visualization results due to gradient errors despite their computational efficiency. On the other hand, non-gradient methods like Score-CAM produce quality visualization that comes with high computational costs. The proposed method SVD filters Grad-CAM (SVD-Grad-CAM), which leverages singular value decomposition (SVD) to overcome the limitations of Grad-CAM. SVD-Grad-CAM filters gradients within the gradient matrix to compute the weight of the feature map for a specific class. This filtering process is achieved by selecting the top k principal components from the SVD decomposition, which discards less important patterns and potential error data. Consequently, SVD-Grad-CAM enhances the quality of Grad-CAM by reducing the clutter of multiple region highlights. The MURA dataset, focusing on elbow study type, is utilized to assess CAM visualization quality, with a DenseNet-169 CNN model fine-tuned via transfer learning. A total of 564 validation radiographs are used in empirical comparison, showing that SVD-Grad-CAM improves average drop, average increase, maximum coherency, and Average DCC by 30%, 21.67%, 19.91% and 22.56% respectively, in comparison to Grad-CAM. Code:: https://github.com/ramaiahthota02/SVD-Grad-CAM-v1.git