Understanding the morphological differences between male and female brain is a long-standing area of scientific interest. Exploring sexual dimorphism can shed light on differences in cognitive function, behavior, and susceptibility to certain neurological disorders across sexes. Machine Learning (ML) represents a powerful tool for exploring such differences, contributing to the advancement of personalized medicine and tailored interventions also for neurological conditions. In particular, EXplainable Artificial Intelligence (XAI) techniques could enhance the study of sexual dimorphism, providing interpretable findings. Given these premises, we compared two XAI classifiers in distinguishing males and females: XGBoost (XGB) with the Shapley Additive Explanations (SHAP) and the Explainable Boosting Machines (EBM), trained on FreeSurfer morphological features from the HCP Young Adult. Both models reached high accuracy after hyperparameter tuning, 87% XGB, and respectively 85% and 86% EBM without interactions and with 20 interactions. Notably, global explanations of SHAP on XGB and EBM demonstrated consistency in feature importance rankings, where bilateral amygdala volume emerged as a key predictor, which had high contribution also in the local explanations of individual predictions. Our findings corroborated the well-known role of amygdala in sexual dimorphisms and highlighted the importance of employing interpretable ML methods in neuroimaging research to elucidate complex relationships between brain structure and sex. In conclusions, the insights gained from this study could aid in early diagnosis and targeted interventions for neurological and psychiatric conditions, taking into account sex-specific differences in brain structure and function.