Context attribution in retrieval-augmented generation (RAG) identifies and scores context sentences that support a model’s answer, thereby improving transparency and trustworthiness. However, existing attention-based methods often aggregate attention over all tokens in a sentence, which introduces noise and causes instability across varying model sizes. They also rely on manually tuned selection thresholds to determine supportive sentences, limiting scalability. This work addresses these issues with three key contributions. First, we systematically explore token selection strategies to enhance attention-based attribution scoring. Second, we introduce a think-twice mechanism that refines attention to capture previously overlooked sources. Third, we propose a greedy context ablation method that automatically determines the number of sources without relying on manual thresholds. Experiments on MESAQA, a multi-evidence dataset, show that our method achieves an improvement of up to 0.70 in F1-score and remains competitive at 0.68 even without any hyperparameter tuning, demonstrating its effectiveness and practicality for reliable context attribution.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing Attention-Based Context Attribution via Token Selection and Think-Twice Mechanism

  • Tz-Huan Hsu,
  • Sian-Yao Huang,
  • Che-Yu Lin,
  • Cheng-Lin Yang

摘要

Context attribution in retrieval-augmented generation (RAG) identifies and scores context sentences that support a model’s answer, thereby improving transparency and trustworthiness. However, existing attention-based methods often aggregate attention over all tokens in a sentence, which introduces noise and causes instability across varying model sizes. They also rely on manually tuned selection thresholds to determine supportive sentences, limiting scalability. This work addresses these issues with three key contributions. First, we systematically explore token selection strategies to enhance attention-based attribution scoring. Second, we introduce a think-twice mechanism that refines attention to capture previously overlooked sources. Third, we propose a greedy context ablation method that automatically determines the number of sources without relying on manual thresholds. Experiments on MESAQA, a multi-evidence dataset, show that our method achieves an improvement of up to 0.70 in F1-score and remains competitive at 0.68 even without any hyperparameter tuning, demonstrating its effectiveness and practicality for reliable context attribution.