Evaluating Large Language Models in Analyzing Student Sentiments: A Course Feedback Case Study
摘要
The emergence of Large Language Models (LLMs) has revolutionized Natural Language Processing (NLP), with significant implications for educational sentiment analysis. Instructors and institutions increasingly rely on student feedback to enhance teaching effectiveness, but manual analysis of qualitative comments is resource-intensive. This study investigates the potential of general-purpose LLMs, specifically GPT-3.5, for automated sentiment analysis of student course evaluations. We compare its performance against fine-tuned transformer models, including BERT, XLNet, BART-large-MNLI, and RoBERTa-large-MNLI, using an open-access dataset of student course feedback. Sentiment classification was conducted using both a three-label (negative, neutral, positive) and a more granular five-label (very negative to very positive) scheme. To assess GPT-3.5’s interpretive capacity, we applied various prompting strategies, such as Zero-shot, One-shot, Few-shot, Chain-of-Thought (CoT), and Role-Playing (RP). Our findings indicate that while fine-tuned models generally outperform GPT-3.5 in five-label classification, GPT-3.5 performs competitively in three-label settings when guided by effective prompts. These results suggest that LLMs, despite certain limitations, can be effectively deployed in educational contexts for scalable and cost-efficient sentiment analysis, contributing to improved responsiveness and personalized learning environments.