Scholarlink

<p>Article reliability measures the truthfulness of news content, providing a&#xa0;direct assessment of factual accuracy and credibility. Organisations such as Ad Fontes Media manually assess article reliability as one of the core components for media bias evaluation. Despite its importance, automatic article reliability classification remains underexplored, partly due to the lack of datasets with suitable annotations. In this paper, we first release a&#xa0;reconstructed version of an existing dataset, consisting of 5270 articles with a&#xa0;new four-class annotation based on article reliability. Second, we propose HT-MAGPIE, a&#xa0;hierarchical transformer leveraging MAGPIE—a&#xa0;large-scale model pre-trained on bias-related tasks—to produce bias-aware representations. We demonstrate that HT-MAGPIE outperforms fine-tuned BERT by 5.02% in overall F1&#xa0;score and fine-tuned Longformer by 16% F1&#xa0;in the lowest-reliability class. We also explore the correlation between outlet-level and article-level reliability by comparing model performance with and without outlet metadata. Our findings indicate that including outlet metadata as an additional feature improves overall F1&#xa0;scores on fine-tuned BERT by 4.32% and BigBird by 2.62%.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

HT-MAGPIE: a Hierarchical Transformer for Article Reliability Classification

  • 期刊论文

  • 发表日期：

    2026年2月4日

查看全文
  • Stefan Adji,
  • Miryam de Lhoneux,
  • Timo Spinde

摘要

Article reliability measures the truthfulness of news content, providing a direct assessment of factual accuracy and credibility. Organisations such as Ad Fontes Media manually assess article reliability as one of the core components for media bias evaluation. Despite its importance, automatic article reliability classification remains underexplored, partly due to the lack of datasets with suitable annotations. In this paper, we first release a reconstructed version of an existing dataset, consisting of 5270 articles with a new four-class annotation based on article reliability. Second, we propose HT-MAGPIE, a hierarchical transformer leveraging MAGPIE—a large-scale model pre-trained on bias-related tasks—to produce bias-aware representations. We demonstrate that HT-MAGPIE outperforms fine-tuned BERT by 5.02% in overall F1 score and fine-tuned Longformer by 16% F1 in the lowest-reliability class. We also explore the correlation between outlet-level and article-level reliability by comparing model performance with and without outlet metadata. Our findings indicate that including outlet metadata as an additional feature improves overall F1 scores on fine-tuned BERT by 4.32% and BigBird by 2.62%.