Long text summarization is essential for processing large-scale text but remains challenging for LLMs like GPT and LLaMA due to limited training data and difficulties in modeling long-range context. To address this issue, we propose T3, a zero-shot transfer learning framework that trains an LLM on an assistant task that has richer data and structural or semantic similarity to summarization. We adopt T3 using question answering as the assistant task. Experiments on BBC Summary, NarraSum, FairytaleQA, and NLQuAD show that T3 outperforms strong LLM baselines, achieving improvements of up to 14% in ROUGE, 35% in BLEU, and 16% in FactScore, demonstrating its transfer effectiveness.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

T3: A Novel Zero-Shot Transfer Learning Framework Iteratively Training on an Assistant Task for a Target Task

  • Xindi Tong,
  • Yujin Zhu,
  • Shijian Fan,
  • Liang Xu

摘要

Long text summarization is essential for processing large-scale text but remains challenging for LLMs like GPT and LLaMA due to limited training data and difficulties in modeling long-range context. To address this issue, we propose T3, a zero-shot transfer learning framework that trains an LLM on an assistant task that has richer data and structural or semantic similarity to summarization. We adopt T3 using question answering as the assistant task. Experiments on BBC Summary, NarraSum, FairytaleQA, and NLQuAD show that T3 outperforms strong LLM baselines, achieving improvements of up to 14% in ROUGE, 35% in BLEU, and 16% in FactScore, demonstrating its transfer effectiveness.