This study investigates the use of ChatGPT for the development of inductive (data-driven) codebooks from qualitative datasets in underrepresented languages (Azerbaijani and Finnish). Although prior work has leveraged GPT- 4o as a “co-researcher” that can support more efficient and comprehensive inductive codebook development, further work is needed to understand how consistent results are across languages and for translated text. The study found GPT- 4o to be useful for identifying relevant codes, but also found limitations, particularly in terms of the quality of example sentences generated for less-resourced languages. Social moderation by humans and construct evaluations were applied to refine the generated codebooks to ensure clarity and reduce redundancy. The results demonstrated that, while GPT-4o significantly aids in multilingual qualitative analysis, human intervention remains essential to validate and enhance the accuracy of the outputs. This research is particularly significant for the learning analytics field as it demonstrates scalable methods for multilingual qualitative analysis, a critical step in expanding the inclusivity and applicability of learning analytics across educational contexts.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

ChatGPT-Assisted Codebook Design for Learning Analytics Datasets in Multiple Languages: A Case Study

  • Ayaz Karimov,
  • Mirka Saarela,
  • Xiner Liu,
  • Zhanlan Wei,
  • Andres Felipe Zambrano,
  • Amanda Barany,
  • Ryan S. Baker,
  • Jaclyn Ocumpaugh,
  • Sabina Mammadova,
  • Tommi Kärkkäinen

摘要

This study investigates the use of ChatGPT for the development of inductive (data-driven) codebooks from qualitative datasets in underrepresented languages (Azerbaijani and Finnish). Although prior work has leveraged GPT- 4o as a “co-researcher” that can support more efficient and comprehensive inductive codebook development, further work is needed to understand how consistent results are across languages and for translated text. The study found GPT- 4o to be useful for identifying relevant codes, but also found limitations, particularly in terms of the quality of example sentences generated for less-resourced languages. Social moderation by humans and construct evaluations were applied to refine the generated codebooks to ensure clarity and reduce redundancy. The results demonstrated that, while GPT-4o significantly aids in multilingual qualitative analysis, human intervention remains essential to validate and enhance the accuracy of the outputs. This research is particularly significant for the learning analytics field as it demonstrates scalable methods for multilingual qualitative analysis, a critical step in expanding the inclusivity and applicability of learning analytics across educational contexts.