Automatic Summarization Evaluation: Methods and Practices
摘要
Automatic summarization systems extract relevant information from documents to provide concise summaries that maintain the same information. Traditionally a human task, automatic summarization has been a challenge since the 1950s and continues to evolve with the advent of large language models (LLMs). The main techniques used in summarization are statistical and fall into two categories: extractive and abstractive summarization. Extractive summarization involves selecting relevant sections of the original text to form the summary, while abstractive summarization creates an entirely new text based on the information in the original document. Evaluation methods for these techniques have evolved from traditional metrics like ROUGE and BLEU to the use of advanced LLMs that assess summary quality in terms of writing, completeness, conciseness, and factuality. This tutorial introduces various summarization techniques and explores both historical and contemporary approaches to their evaluation.