Detecting and Correcting Hallucinations in LLMs via Substantive Uncertainty and Iterative Validation
摘要
Hallucination remains one of the most critical challenges in natural language generation (NLG), especially for large language models (LLMs) deployed in knowledge-intensive applications where factual consistency is critical. Existing hallucination detection methods often suffer from poor generalization across tasks and text types, while correction strategies typically require costly model retraining or extensive architectural changes. To address these limitations, we propose a lightweight, model-agnostic framework for hallucination detection and correction that integrates token-level uncertainty estimation with multi-turn verification. Central to our approach is the Substantive-word Uncertainty Score (SUScore), a novel metric that quantifies uncertainty over substantive words–nouns, verbs, numerals, and other semantically important tokens–by incorporating syntactic priors, lexical importance, and model confidence. Building on this, we introduce the Iterative Chain-Query (ICQ) framework, which performs targeted, question-driven validation of potentially hallucination-prone spans through multi-step consistency checking, optionally enhanced with retrieval-augmented generation from external knowledge sources. Our approach requires no retraining and generalizes across different LLM architectures and NLG tasks. Experiments on three benchmark datasets–UHGEval, SVAMP, and QUEST–demonstrate that SUScore achieves superior calibration for hallucination detection compared to standard lexical or reference-based metrics such as BLEU and ROUGE. Furthermore, ICQ significantly improves factual accuracy while preserving output fluency and linguistic coherence. Together, these contributions offer a scalable and efficient solution for mitigating LLM hallucinations in real-world NLG deployments.