Despite the remarkable progress made by large language models (LLMs) in natural language understanding, they still struggle to effectively comprehend visually-rich documents (VRDs) that demand complex semantic analysis. This paper introduces HiDReader, the first reinforcement learning-based framework designed to emulate human-like reading processes for visually-rich document comprehension. By integrating LLMs with specialized document understanding models, HiDReader leverages a self-optimizing, data-driven reading sequence mechanism that mimics human learning patterns rather than relying on traditional rule-based or fixed parsing strategies. Through iterative self-supervised learning, HiDReader dynamically adapts its reading path based on contextual cues and task-specific goals, refining its approach in a way that mirrors the adaptability of human cognition. Experimental results on two benchmark tasks demonstrate that HiDReader outperforms baseline methods in both document information extraction and question answering. Moreover, in cross-domain transfer learning scenarios, it showcases significant adaptability, autonomously extracting characteristic reading patterns from VRDs, thereby achieving superior performance in zero-shot domain adaptation tasks compared to conventional approaches.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

HiDReader: Human-Inspired Document Reading Agent via Reinforcement Learning

  • Changqing Wang,
  • Hao Wang,
  • Pinpin Zhu,
  • Huiran Zhang

摘要

Despite the remarkable progress made by large language models (LLMs) in natural language understanding, they still struggle to effectively comprehend visually-rich documents (VRDs) that demand complex semantic analysis. This paper introduces HiDReader, the first reinforcement learning-based framework designed to emulate human-like reading processes for visually-rich document comprehension. By integrating LLMs with specialized document understanding models, HiDReader leverages a self-optimizing, data-driven reading sequence mechanism that mimics human learning patterns rather than relying on traditional rule-based or fixed parsing strategies. Through iterative self-supervised learning, HiDReader dynamically adapts its reading path based on contextual cues and task-specific goals, refining its approach in a way that mirrors the adaptability of human cognition. Experimental results on two benchmark tasks demonstrate that HiDReader outperforms baseline methods in both document information extraction and question answering. Moreover, in cross-domain transfer learning scenarios, it showcases significant adaptability, autonomously extracting characteristic reading patterns from VRDs, thereby achieving superior performance in zero-shot domain adaptation tasks compared to conventional approaches.