Evaluating an AI-Driven Chatbot Tutor for Enhancing Introductory Programming Learning
摘要
Recent advancements in Artificial Intelligence and Natural Language Processing have catalyzed innovative educational tools, enhancing learning experiences across several domains. This paper investigates the development and evaluation of an AI-powered chatbot designed to support programming education in the Computer Science 1 course at University of the Andes. Addressing the limitations of traditional teaching assistants, the chatbot offers round-the-clock assistance, promotes problem-solving skills, and provides personalized feedback on students’ Python exercises. A user study involving 45 participants, comprising students, teaching assistants, and professors, assessed the chatbot’s effectiveness. Results indicate high satisfaction: 95.56% of participants reported clear communication, 91.11% found the chatbot’s conversational flow effective, and 88.89% confirmed its ability to maintain context. Additionally, 97.78% found responses easy to understand, and 95.56% felt the chatbot contributed to their learning goals. While minor limitations were identified in handling ambiguous queries and response latency during peak usage, the chatbot demonstrated strong usability and pedagogical alignment. Future work will focus on optimizing response time, improving long-session context tracking, and evaluating the chatbot’s impact on student performance at scale.