Text summarization is an important problem in natural language processing. The advent of large-scale transformer-based models, such as BART, PEGASUS, and T5, has revolutionized abstractive summarization. However, all these models are centralized and require the entire text to be summarized at a single location, where the model operates and generates the summary. Centralized training of NLP models raises privacy concerns in sensitive domains such as healthcare, law, and finance. Another challenge concerning these models is their unfair representation of users with small corpus data. To mitigate this, we propose a privacy-preserving framework for abstractive summarization using federated learning, eliminating the need to transmit raw data to a centralized server. Our approach employs a pre-trained BART model, and to ensure privacy against inference attacks by the aggregating server, differential privacy is utilized. Our approach also incorporates a fairness-aware aggregation strategy, Q-fair, to ensure balanced client contributions. Experiments on the CNN/DailyMail dataset show that our FL-based method achieves competitive ROUGE scores while enhancing privacy and fairness, demonstrating its practicality for sensitive real-world applications.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Privacy-Preserving Fair Text Summarization Using Federated Learning

  • Aman Lachhiramka,
  • Nibhrant Vaishnav,
  • Dheeraj Kumar

摘要

Text summarization is an important problem in natural language processing. The advent of large-scale transformer-based models, such as BART, PEGASUS, and T5, has revolutionized abstractive summarization. However, all these models are centralized and require the entire text to be summarized at a single location, where the model operates and generates the summary. Centralized training of NLP models raises privacy concerns in sensitive domains such as healthcare, law, and finance. Another challenge concerning these models is their unfair representation of users with small corpus data. To mitigate this, we propose a privacy-preserving framework for abstractive summarization using federated learning, eliminating the need to transmit raw data to a centralized server. Our approach employs a pre-trained BART model, and to ensure privacy against inference attacks by the aggregating server, differential privacy is utilized. Our approach also incorporates a fairness-aware aggregation strategy, Q-fair, to ensure balanced client contributions. Experiments on the CNN/DailyMail dataset show that our FL-based method achieves competitive ROUGE scores while enhancing privacy and fairness, demonstrating its practicality for sensitive real-world applications.