Seamless automation of business processes requires the automatic analysis of complex business documents like invoices or contracts. Extracting key information in these semi-structured documents poses a significant challenge. While multimodal language models have demonstrated state-of-the-art results in this field, their application to document types with high complexity is still challenging. They neglect the underlying document structure and existing dependencies between information types. On complex document types commonly encountered in industry, this results in preventable errors such as missing predictions for mandatory elements or wrong extractions of interdependent elements. In this paper, we present SKIE-SRL, a Statistical Relational Learning model for Structured Key Information Extraction. This hybrid approach unifies symbolic reasoning with multimodal language models. It extends the capabilities of pretrained language models with expert-provided knowledge about the document structure and application domain. We evaluate our model on a multilingual invoice data set from industry and compare it to three state-of-the-art language models as well as a stacked ensemble. Our approach outperforms all benchmarks by 4% F1-score.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

SKIE-SRL: Structured Key Information Extraction from Business Documents Using Statistical Relational Learning

  • Birgit Kirsch,
  • Xiang Yu,
  • Nilesh Chakraborty,
  • Niclas Doll,
  • Sven Giesselbach,
  • Stefan Rueping

摘要

Seamless automation of business processes requires the automatic analysis of complex business documents like invoices or contracts. Extracting key information in these semi-structured documents poses a significant challenge. While multimodal language models have demonstrated state-of-the-art results in this field, their application to document types with high complexity is still challenging. They neglect the underlying document structure and existing dependencies between information types. On complex document types commonly encountered in industry, this results in preventable errors such as missing predictions for mandatory elements or wrong extractions of interdependent elements. In this paper, we present SKIE-SRL, a Statistical Relational Learning model for Structured Key Information Extraction. This hybrid approach unifies symbolic reasoning with multimodal language models. It extends the capabilities of pretrained language models with expert-provided knowledge about the document structure and application domain. We evaluate our model on a multilingual invoice data set from industry and compare it to three state-of-the-art language models as well as a stacked ensemble. Our approach outperforms all benchmarks by 4% F1-score.