A Hybrid Extractive and Encoder-Decoder-Based Approach for Mitigating Hallucination in Automatic Text Summarization
摘要
Auto text summarization (ATS) revolutionized almost every area from social media to healthcare by giving the main information or summary of the document in a quick span of time. Extractive and abstractive summarization techniques are the pinnacle of advancement in ATS. Abstractive text summarization generates fluent and human-like summaries as new text, but this summary suffers from the problem of intrinsic or extrinsic hallucination. Filling this research gap is crucial because it undermines the factuality, faithfulness, and reliability of the generated summary by introducing plausible text that does not exist in the original text. While extractive techniques are fast and faithful, they lack fluency and coherence because the text is taken directly from the source material. To address the research gap of hallucination in generated summaries, this paper proposed an improved dual-stage hybrid approach by combining the functionalities of an extractive graph-based model and a transformer-based abstractive ATS model. The proposed approach is an improved approach for generating more factual summaries using hybridization. The proposed dual approach first extracts the important information from the main text and then processes this information using deep generative architecture like transformers. By ensuring the strength and semantics of selected content, the proposed approach generates a consistent summary. Hallucinations are detected based on the extra entities present in the generated summary with respect to the source text using the entity-level verification method named “entity extraction.” Entity matching is performed to determine the extra, missing entity. Further, to evaluate the performance of the proposed approach, it is tested for factuality and compared with the base model using precision, recall, and the F-measure of the ROUGE score on the CNN/DailyMail and XSum datasets. The proposed model has the highest recall 0.42 and 0.31 on CNN/DailyMail and XSum and the highest similarity score 0.87 as compared to the base model. The model is also evaluated on performance metrics like computational cost, latency, and model size. Hallucination is detected and reduced in generated summaries on the basis of extraneous, missing entities, and similarity scores with respect to source data using the proposed strategy. A statistical test is performed to check the superiority of the proposed model over the base model.