Scholarlink

<p>The emergence of large language models (LLMs) has transformed natural language processing (NLP), enabling significant advances in text generation and comprehension. However, LLMs often produce hallucinated content-text that is unverifiable, incorrect, or inconsistent with the input-which is especially problematic in high-stakes domains like medicine, law, and finance. While prior research has examined hallucinations in isolated tasks or models, a systematic, fine-grained understanding of hallucination types across critical domains remains lacking. In this work, we develop a methodology to synthetically generate hallucinated question-answer instances categorized into five distinct types: Factual Inconsistency, Factual Fabrication, Logical Inconsistency, Instruction Inconsistency, and Context Inconsistency. To enable this, we construct a benchmark dataset comprising 52,646 hallucinated instances from the medical, legal, and finance domains, using domain-specific datasets (MedQuAD, LegalQAv1, FIQA) for four hallucination types, and datasets with explicit context (CUAD-QA, PubMedQA) for context inconsistency. We evaluate multiple open- and closed-source LLMs on hallucination detection, finding that off-the-shelf models struggle to accurately detect hallucinated content. Fine-tuning LLMs on our dataset leads to significant performance improvements, boosting detection accuracy by 15.8%, 11.4%, and 11.6% in the medical, legal, and finance domains respectively. Our results highlight the importance of structured hallucination training data for building more reliable LLM-based systems.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A framework to synthetically generate fine-grained hallucinated data

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年9月27日

查看全文
  • Ashita Saxena,
  • Dishank Aggarwal,
  • Naveen Badathala,
  • Pushpak Bhattacharyya

摘要

The emergence of large language models (LLMs) has transformed natural language processing (NLP), enabling significant advances in text generation and comprehension. However, LLMs often produce hallucinated content-text that is unverifiable, incorrect, or inconsistent with the input-which is especially problematic in high-stakes domains like medicine, law, and finance. While prior research has examined hallucinations in isolated tasks or models, a systematic, fine-grained understanding of hallucination types across critical domains remains lacking. In this work, we develop a methodology to synthetically generate hallucinated question-answer instances categorized into five distinct types: Factual Inconsistency, Factual Fabrication, Logical Inconsistency, Instruction Inconsistency, and Context Inconsistency. To enable this, we construct a benchmark dataset comprising 52,646 hallucinated instances from the medical, legal, and finance domains, using domain-specific datasets (MedQuAD, LegalQAv1, FIQA) for four hallucination types, and datasets with explicit context (CUAD-QA, PubMedQA) for context inconsistency. We evaluate multiple open- and closed-source LLMs on hallucination detection, finding that off-the-shelf models struggle to accurately detect hallucinated content. Fine-tuning LLMs on our dataset leads to significant performance improvements, boosting detection accuracy by 15.8%, 11.4%, and 11.6% in the medical, legal, and finance domains respectively. Our results highlight the importance of structured hallucination training data for building more reliable LLM-based systems.