Making machine learning (ML) model predictions understandable to diverse users is a critical challenge in Explainable Artificial Intelligence (XAI). While XAI techniques such as SHAP and LIME provide feature attributions, their outputs often require technical expertise to interpret. This study investigates the use of Large Language Models (LLMs) to bridge this gap by generating accessible natural language explanations from technical XAI outputs and to evaluate the quality of these explanations. We conduct a comparative analysis employing two distinct LLMs (OpenAI’s GPT-4o and o4-mini) and two prompting strategies (Zero-Shot and Few-Shot learning) for explanation generation using the TEXEN dataset. The generated explanations are evaluated using traditional NLP metrics (BLEU, ROUGE, METEOR, BERTScore) against human-authored references, and through an LLM-as-a-Judge approach (Prometheus 2) assessing Soundness, Completeness, Fluency, and Context-Awareness. Results indicate that while GPT-4o with Few-Shot prompting achieved higher scores on traditional NLP metrics (e.g., BERTScore F1 of 0.858), the LLM-as-a-Judge evaluation revealed that the smaller o4-mini model, particularly with Few-Shot prompting, often matched or surpassed GPT-4o in human-aligned qualitative criteria, achieving, for instance, a fluency score of 4.30 and completeness of 4.10. These findings highlight the nuanced capabilities of different LLMs and the importance of multifaceted evaluation frameworks for developing truly interpretable XAI systems.