In recent years, significant progress has been made in Natural Language Processing (NLP), yet a gap persists in tailoring explanations for text classification to non-AI users. In this research, we conduct a qualitative study to compare the suitability of several explanation alternatives generated with existing Explainable AI (XAI) methods using visualisations and textual explanations created by LLMs. Our preliminary findings reveal that explanations emphasising the paper’s methodology enhance user confidence: while textual explanations are generally preferred, effective visualisations highlighting methodological aspects are valuable. Finally, results show that what users expect from explanations does not align with current efforts in XAI: they need explanations that support how they conduct the same task and are not concerned with fidelity to the model’s inner workings.