Scholarlink

<p>Large Language Models (LLMs) are increasingly incorporated into academic writing workflows, raising questions about reliability, epistemic accountability, and responsible use. While prior studies have evaluated LLM performance on isolated tasks such as summarization, less attention has been given to their behavior across multi-stage scholarly writing functions under controlled conditions. This study presents a structured comparison of three ChatGPT models (GPT-4o, GPT-5.0, GPT-5.1) across ten representative scientific-writing tasks, including abstract revision, methods expansion, literature synthesis, statistical interpretation, and citation formatting. Each model generated one response per task in isolated sessions. Outputs were evaluated holistically using a predefined 0–5 rubric assessing factual accuracy, academic writing quality, and hallucination behavior. Descriptive results indicate systematic cross-generational differences. GPT-5.1 obtained the highest mean scores within the present evaluation framework, particularly in high-constraint tasks requiring strict adherence to provided data. GPT-4o showed greater susceptibility to fabricated citations and unsupported inferences, while GPT-5.0 demonstrated intermediate performance. Although newer models exhibited improved constraint adherence, residual interpretive drift underscores the need for expert oversight. The findings support responsible integration frameworks in which LLMs function as drafting aids rather than autonomous scholarly agents, contributing to ongoing discussions on accountability and governance in AI-assisted scientific communication.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A comparative study of ChatGPT in scientific writing assistance: accuracy, style, and hallucination patterns across GPT-4o, GPT-5.0, and GPT-5.1

  • 期刊论文

  • 发表日期：

    2026年3月16日

查看全文
  • Usama Bin Anjum,
  • Kana Hirasawa,
  • Urooj Fatima

摘要

Large Language Models (LLMs) are increasingly incorporated into academic writing workflows, raising questions about reliability, epistemic accountability, and responsible use. While prior studies have evaluated LLM performance on isolated tasks such as summarization, less attention has been given to their behavior across multi-stage scholarly writing functions under controlled conditions. This study presents a structured comparison of three ChatGPT models (GPT-4o, GPT-5.0, GPT-5.1) across ten representative scientific-writing tasks, including abstract revision, methods expansion, literature synthesis, statistical interpretation, and citation formatting. Each model generated one response per task in isolated sessions. Outputs were evaluated holistically using a predefined 0–5 rubric assessing factual accuracy, academic writing quality, and hallucination behavior. Descriptive results indicate systematic cross-generational differences. GPT-5.1 obtained the highest mean scores within the present evaluation framework, particularly in high-constraint tasks requiring strict adherence to provided data. GPT-4o showed greater susceptibility to fabricated citations and unsupported inferences, while GPT-5.0 demonstrated intermediate performance. Although newer models exhibited improved constraint adherence, residual interpretive drift underscores the need for expert oversight. The findings support responsible integration frameworks in which LLMs function as drafting aids rather than autonomous scholarly agents, contributing to ongoing discussions on accountability and governance in AI-assisted scientific communication.