This paper addresses the SlideVQA task, which involves answering visual multi-hop questions on presentation slides. SlideVQA requires complex reasoning abilities, such as understanding relationships between multiple document images and numerical reasoning. Conventional methods have demonstrated significantly lower performance in multi-hop question answering than simple single-hop. Specifically, one instance has shown a performance drop of over 10 points in the F1 score, indicating insufficient performance. Furthermore, while conventional studies have demonstrated that accurate evidence selection improves overall task performance, there remains room for improvement. This paper focuses on improving the performance of multi-hop question answering and evidence selection in the SlideVQA task, thereby enhancing the overall performance of the question-answering task. To this end, we propose Multimodal Beam Retrieval (MMBR), a multimodal extension of Beam Retrieval, a retriever for answering multi-hop questions on text documents. Through experiments using the SlideVQA dataset, we aim to clarify whether the proposed method improves the performance of evidence selection and contributes to the overall performance enhancement of the question-answering task.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multi-hop Question Answering in SlideVQA Based on Beam Retrieval

  • Mizuki Yamano,
  • Hisashi Miyamori

摘要

This paper addresses the SlideVQA task, which involves answering visual multi-hop questions on presentation slides. SlideVQA requires complex reasoning abilities, such as understanding relationships between multiple document images and numerical reasoning. Conventional methods have demonstrated significantly lower performance in multi-hop question answering than simple single-hop. Specifically, one instance has shown a performance drop of over 10 points in the F1 score, indicating insufficient performance. Furthermore, while conventional studies have demonstrated that accurate evidence selection improves overall task performance, there remains room for improvement. This paper focuses on improving the performance of multi-hop question answering and evidence selection in the SlideVQA task, thereby enhancing the overall performance of the question-answering task. To this end, we propose Multimodal Beam Retrieval (MMBR), a multimodal extension of Beam Retrieval, a retriever for answering multi-hop questions on text documents. Through experiments using the SlideVQA dataset, we aim to clarify whether the proposed method improves the performance of evidence selection and contributes to the overall performance enhancement of the question-answering task.