Scholarlink

<p>Hallucinations in Large Language Models (LLMs), defined as plausible-sounding yet factually incorrect outputs, pose a significant barrier to their adoption in high-stakes domains. This paper introduces a novel explainable AI-based framework that combines token-level attribution techniques (SHAP and LIME) with a quantitative Hallucination Score (HS) to detect and interpret hallucinated content. Our approach enables fine-grained analysis of factual inconsistencies by measuring attribution divergence between input and output tokens, offering both numerical and visual interpretability. We evaluated the framework on two benchmark datasets (TruthfulQA and QAGS) using three prominent LLMs (GPT<InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(-\)</EquationSource> </InlineEquation>3.5, LLaMA-2–13B, and Falcon-40B). The results show strong performance, with GPT<InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(-\)</EquationSource> </InlineEquation>3.5 achieving an F1-score of 0.84 and ROC-AUC of 0.89 on TruthfulQA, and the Hallucination Score demonstrating high alignment with human annotations (R<InlineEquation ID="IEq3"> <EquationSource Format="TEX">\(^2\)</EquationSource> </InlineEquation> = 0.84, MAE = 0.11). The framework’s interpretability not only enhances transparency but also empowers human auditors in real-time validation workflows. While limitations such as sensitivity to paraphrased truths and computational overhead remain, this work lays a foundation for trustworthy, scalable hallucination detection by bridging explainability with factual verification, paving the way for safer and more reliable LLM applications.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Quantifying Factual Divergence in Generative Models: SHAP-LIME Based Hallucination Score for LLMs

  • 期刊论文

  • 发表日期：

    2026年2月5日

查看全文
  • Ijazul Haq,
  • Muhammad Saqib,
  • Yingjie Zhang,
  • Irfan Ali Khan

摘要

Hallucinations in Large Language Models (LLMs), defined as plausible-sounding yet factually incorrect outputs, pose a significant barrier to their adoption in high-stakes domains. This paper introduces a novel explainable AI-based framework that combines token-level attribution techniques (SHAP and LIME) with a quantitative Hallucination Score (HS) to detect and interpret hallucinated content. Our approach enables fine-grained analysis of factual inconsistencies by measuring attribution divergence between input and output tokens, offering both numerical and visual interpretability. We evaluated the framework on two benchmark datasets (TruthfulQA and QAGS) using three prominent LLMs (GPT \(-\) 3.5, LLaMA-2–13B, and Falcon-40B). The results show strong performance, with GPT \(-\) 3.5 achieving an F1-score of 0.84 and ROC-AUC of 0.89 on TruthfulQA, and the Hallucination Score demonstrating high alignment with human annotations (R \(^2\) = 0.84, MAE = 0.11). The framework’s interpretability not only enhances transparency but also empowers human auditors in real-time validation workflows. While limitations such as sensitivity to paraphrased truths and computational overhead remain, this work lays a foundation for trustworthy, scalable hallucination detection by bridging explainability with factual verification, paving the way for safer and more reliable LLM applications.