Text summarization is one of the crucial methods for NLP, especially for morphologically rich and resource-poor languages such as Hindi. In contrast to previous research, for languages such as Hindi, extractive summarization is both computationally efficient and linguistically strong as it identifies and selects the most significant lines from the source text. This study focuses on a general framework for extractive summarization in Hindi, bringing together statistical, linguistic, positional and semantic features. Using the corpus from the Indian Languages Corpora Initiative (ILCI), we compare the performance of TF-IDF, TextRank, and supervised learning algorithms by using both automatic and human metrics (ROUGE, BLEU). Through our results, it can be confirmed that hybrid and supervised models can give better summaries in terms of relevance, coherence, and readability. Although the dataset size is not large enough to do benchmarking for ML/DL at a large scale, these results provide foundations for feature-enriched frameworks for improving summarization quality in the Hindi language. In our future research, we will extend our benchmarking for neural and transformer based models also.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Feature-Enriched Extractive Text Summarization Framework for Hindi Language Documents

  • Atul Kumar,
  • Ashutosh Bajpai,
  • Ashish Baiswar,
  • Mahendra Kumar,
  • Archana Dixit

摘要

Text summarization is one of the crucial methods for NLP, especially for morphologically rich and resource-poor languages such as Hindi. In contrast to previous research, for languages such as Hindi, extractive summarization is both computationally efficient and linguistically strong as it identifies and selects the most significant lines from the source text. This study focuses on a general framework for extractive summarization in Hindi, bringing together statistical, linguistic, positional and semantic features. Using the corpus from the Indian Languages Corpora Initiative (ILCI), we compare the performance of TF-IDF, TextRank, and supervised learning algorithms by using both automatic and human metrics (ROUGE, BLEU). Through our results, it can be confirmed that hybrid and supervised models can give better summaries in terms of relevance, coherence, and readability. Although the dataset size is not large enough to do benchmarking for ML/DL at a large scale, these results provide foundations for feature-enriched frameworks for improving summarization quality in the Hindi language. In our future research, we will extend our benchmarking for neural and transformer based models also.