Contrastive explanations make the predictive behavior of machine-learned classification models transparent and evaluable based on example features at the decision boundary. An established method for explaining image classifiers is the Contrastive Explanations Method (CEM). This method explains the classification of an image based on minimal pixel changes that are either necessary for the target class or that lead to a class change. How similarity between classes influences the interpretability and robustness of CEM had not yet been investigated quantitatively and hypothesis-test-driven. We therefore re-evaluated the robustness and interpretability of CEM by quantitatively analyzing its Pertinent Negatives (PNs) in scenarios with varying levels of class similarity. The goal was to determine whether the minimal changes in PNs truly reflected meaningful, human-understandable differences between similar classes, thereby addressing a key research gap in evaluating CEM explanations beyond anecdotal evidence.