Topic modeling is an unsupervised machine learning technique used to discover hidden thematic structures within large collections of unstructured text. In an era of rapidly growing digital content, uncovering meaningful patterns from unstructured text has become increasingly vital. This study presents a topic modeling framework designed to extract latent themes and actionable insights from large-scale news data, using articles collected from “The Daily Star”. By applying transformer-based models, including BERTopic and BGE-Base-en-1.5 embeddings, the framework is evaluated across key metrics such as coherence, diversity, purity, and topic distribution. A comprehensive preprocessing pipeline ensures semantically rich and clean input for modeling. Experimental results demonstrate that BGE-Base-en-1.5 achieves the highest semantic coherence (0.69), while BERTopic delivers superior cluster purity (0.95), highlighting their complementary strengths. The findings reinforce the effectiveness of embedding-based topic models for automated, scalable, and interpretable analysis of news content.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Transformer-Based and Probabilistic Approaches for Topic Modeling in News Article Analysis

  • Rafin Abrar Rono,
  • Md. Sayem Kabir,
  • Sadman Samir Rafith,
  • Md. Bakibillah Rahat,
  • Tasnim Sultana Sintheia,
  • Kazi Tanvir

摘要

Topic modeling is an unsupervised machine learning technique used to discover hidden thematic structures within large collections of unstructured text. In an era of rapidly growing digital content, uncovering meaningful patterns from unstructured text has become increasingly vital. This study presents a topic modeling framework designed to extract latent themes and actionable insights from large-scale news data, using articles collected from “The Daily Star”. By applying transformer-based models, including BERTopic and BGE-Base-en-1.5 embeddings, the framework is evaluated across key metrics such as coherence, diversity, purity, and topic distribution. A comprehensive preprocessing pipeline ensures semantically rich and clean input for modeling. Experimental results demonstrate that BGE-Base-en-1.5 achieves the highest semantic coherence (0.69), while BERTopic delivers superior cluster purity (0.95), highlighting their complementary strengths. The findings reinforce the effectiveness of embedding-based topic models for automated, scalable, and interpretable analysis of news content.