Unveiling pyAutoSummarizer: An Extractive and Abstractive Summarization Library Powered with Artificial Intelligence
摘要
Background: In the Natural Language Processing (NLP) domain, text summarization poses considerable challenges stemming from the text’s intricate linguistic and contextual dependencies. To address these hurdles, we present pyAutoSummarizer, an advanced Python library that harnesses diverse text summarization strategies to generate high-quality summaries. Purpose: pyAutoSummarizer offers a comprehensive suite of extractive and abstractive summarization algorithms, constituting a highly versatile tool suitable for many text summarization tasks. Methods: The library integrates renowned algorithms such as TextRank, LexRank, LSA, and KL-Sum with cutting-edge deep-learning models like BART, T5, PEGASUS, and ChatGPT. In addition to its text summarization techniques, pyAutoSummarizer features robust preprocessing capabilities and employs widely recognized evaluation metrics, including Rouge-N, Rouge-L, Rouge-S, BLEU, and METEOR. Findings: Evaluating abstractive summaries solely based on traditional evaluation metrics may not accurately assess their quality. The abstractive models received lower scores across the evaluation metrics than extractive models like TextRank and LexRank. However, it is crucial to consider the nature of abstractive summarization, which involves generating novel sentences that capture the essence of the source text. Traditional evaluation metrics heavily rely on exact phrase and structure matching, which may not fully grasp abstractive summaries’ semantic and contextual accuracy. Originality: The significance of our library lies in its ability to bring together diverse summarization techniques, evaluation metrics, and efficient output generation, providing users with a comprehensive toolkit for automatic text summarization. pyAutoSummarizer is freely available at: https://bit.ly/43wR4Xe . Source code and demos are available at: https://bit.ly/4bnYP5I .