Automatic summarization systems extract relevant information from documents to provide concise summaries that maintain the same information. Traditionally a human task, automatic summarization has been a challenge since the 1950s and continues to evolve with the advent of large language models (LLMs). The main techniques used in summarization are statistical and fall into two categories: extractive and abstractive summarization. Extractive summarization involves selecting relevant sections of the original text to form the summary, while abstractive summarization creates an entirely new text based on the information in the original document. Evaluation methods for these techniques have evolved from traditional metrics like ROUGE and BLEU to the use of advanced LLMs that assess summary quality in terms of writing, completeness, conciseness, and factuality. This tutorial introduces various summarization techniques and explores both historical and contemporary approaches to their evaluation.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Automatic Summarization Evaluation: Methods and Practices

  • Héctor D. Menéndez,
  • Aidan Dakhama

摘要

Automatic summarization systems extract relevant information from documents to provide concise summaries that maintain the same information. Traditionally a human task, automatic summarization has been a challenge since the 1950s and continues to evolve with the advent of large language models (LLMs). The main techniques used in summarization are statistical and fall into two categories: extractive and abstractive summarization. Extractive summarization involves selecting relevant sections of the original text to form the summary, while abstractive summarization creates an entirely new text based on the information in the original document. Evaluation methods for these techniques have evolved from traditional metrics like ROUGE and BLEU to the use of advanced LLMs that assess summary quality in terms of writing, completeness, conciseness, and factuality. This tutorial introduces various summarization techniques and explores both historical and contemporary approaches to their evaluation.