Scholarlink

<p>The rapid advancement of Multimodal Large Language Models (MLLMs) has opened new frontiers in automatic document understanding. However, understanding legal documents, with their unstructured nature and complex domain requirements, remains an underexplored challenge. We introduce <Emphasis FontCategory="NonProportional">IndianPCL</Emphasis>, a large-scale document image dataset specifically curated for visual question answering (VQA) in the Indian legal context. <Emphasis FontCategory="NonProportional">IndianPCL</Emphasis> comprises 358 multi page police complaint letters (250 printed and 108 handwritten) sourced from Indian police stations, over 6,300 annotated question–answer pairs for information localization and extraction. We use this dataset to systematically benchmark both proprietary and open-source MLLMs – including GPT-4o, Gemini 2.0 Flash, Phi-3.5 Vision-Instruct, LLaVA 1.6, LLaVA-Interleave, DeepSeek-VL 2 Tiny, Qwen-3-VL, and InternVl-3.5 – on two core tasks: standard VQA and Legal VQA, the latter requiring deeper legal reasoning and factual grounding. Results reveal substantial performance gaps: VQA accuracies range from 10-46% on printed documents and 10-41% on handwritten documents, while Legal VQA accuracies span 0-37%, demonstrating that current MLLMs struggle with unstructured police complaint letters requiring legal domain expertise for FIR field extraction. Our Fine-tuned open-source models using Parameter-Efficient Fine-Tuning (PEFT) yields significant improvements, achieving VQA accuracies of 47-60% and Legal VQA accuracies of 31-48%, with hallucination rates decreasing from over 80% to below 11% exceeding the performance of both GPT 4o and Gemini 2.0 Flash. Our results confirm that the <Emphasis FontCategory="NonProportional">IndianPCL</Emphasis> establishes a challenging benchmark for advancing vision-language models in police complaint letter understanding, with implications for automating information extraction in legal intake workflows.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

IndianPCL: A Dataset for Multimodal Legal Document Understanding

  • 期刊论文

  • 发表日期：

    2026年7月1日

查看全文
  • Sagar Chakraborty,
  • Neeraj Boddeda,
  • Aditya Palakurty,
  • Gaurav Harit,
  • Saptarshi Ghosh

摘要

The rapid advancement of Multimodal Large Language Models (MLLMs) has opened new frontiers in automatic document understanding. However, understanding legal documents, with their unstructured nature and complex domain requirements, remains an underexplored challenge. We introduce IndianPCL, a large-scale document image dataset specifically curated for visual question answering (VQA) in the Indian legal context. IndianPCL comprises 358 multi page police complaint letters (250 printed and 108 handwritten) sourced from Indian police stations, over 6,300 annotated question–answer pairs for information localization and extraction. We use this dataset to systematically benchmark both proprietary and open-source MLLMs – including GPT-4o, Gemini 2.0 Flash, Phi-3.5 Vision-Instruct, LLaVA 1.6, LLaVA-Interleave, DeepSeek-VL 2 Tiny, Qwen-3-VL, and InternVl-3.5 – on two core tasks: standard VQA and Legal VQA, the latter requiring deeper legal reasoning and factual grounding. Results reveal substantial performance gaps: VQA accuracies range from 10-46% on printed documents and 10-41% on handwritten documents, while Legal VQA accuracies span 0-37%, demonstrating that current MLLMs struggle with unstructured police complaint letters requiring legal domain expertise for FIR field extraction. Our Fine-tuned open-source models using Parameter-Efficient Fine-Tuning (PEFT) yields significant improvements, achieving VQA accuracies of 47-60% and Legal VQA accuracies of 31-48%, with hallucination rates decreasing from over 80% to below 11% exceeding the performance of both GPT 4o and Gemini 2.0 Flash. Our results confirm that the IndianPCL establishes a challenging benchmark for advancing vision-language models in police complaint letter understanding, with implications for automating information extraction in legal intake workflows.