Scholarlink

<p>In this study in German higher education, we draw on the first panel of dataset from a&#xa0;longitudinal assessment of domain-specific (DOM) Critical Online Reasoning (COR) in economics in the winter terms 2023/24 (t0) and 2024/25 (t1). DOM-COR tasks were completed as a&#xa0;digital assessment, combining open web search, information evaluation, and reasoning to solve domain-specific scenarios. For this, many students used Artificial Intelligence (AI) chatbots employing Large Language Models (LLMs). This offered a&#xa0;natural quasi-experiment in a&#xa0;real-life learning setting to compare students’ DOM-COR performance with and without LLMs. Students also reported on LLM-tools use for study and everyday purposes as well as on their learning outcomes (e.g., credit points, exams). This study investigates both the relationships between LLM-tools use and students DOM-COR assessment performance as well as their learning outcomes at universities. We explicitly distinguish between the qualitative indicators of learning and the speed thereof in the context of LLM-chatbots use. By comparing the time students spent completing DOM-COR tasks and their performance between LLM-users and non-users, we examine if, and to what extent, LLM-users had an advantage. While LLM-users were faster, they did not perform better on written task responses. The relationships with learning outcomes trend in the same direction, indicating that regular LLM-users passed more exams, but did not appear to achieve better grades. The findings point to opportunities and risks when using LLM for domain-specific COR-like tasks as well as for regular study assignments.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

The relationship between AI-chatbots use, student assessment performance and learning outcomes in higher education

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年2月5日

查看全文
  • Dimitri Molerov,
  • Denis Federiakin,
  • Olga Zlatkin-Troitschanskaia,
  • Kevin Shenavai,
  • Lukas Trierweiler,
  • Marie-Theres Nagel

摘要

In this study in German higher education, we draw on the first panel of dataset from a longitudinal assessment of domain-specific (DOM) Critical Online Reasoning (COR) in economics in the winter terms 2023/24 (t0) and 2024/25 (t1). DOM-COR tasks were completed as a digital assessment, combining open web search, information evaluation, and reasoning to solve domain-specific scenarios. For this, many students used Artificial Intelligence (AI) chatbots employing Large Language Models (LLMs). This offered a natural quasi-experiment in a real-life learning setting to compare students’ DOM-COR performance with and without LLMs. Students also reported on LLM-tools use for study and everyday purposes as well as on their learning outcomes (e.g., credit points, exams). This study investigates both the relationships between LLM-tools use and students DOM-COR assessment performance as well as their learning outcomes at universities. We explicitly distinguish between the qualitative indicators of learning and the speed thereof in the context of LLM-chatbots use. By comparing the time students spent completing DOM-COR tasks and their performance between LLM-users and non-users, we examine if, and to what extent, LLM-users had an advantage. While LLM-users were faster, they did not perform better on written task responses. The relationships with learning outcomes trend in the same direction, indicating that regular LLM-users passed more exams, but did not appear to achieve better grades. The findings point to opportunities and risks when using LLM for domain-specific COR-like tasks as well as for regular study assignments.