Scholarlink

Federated Learning (FL) enables privacy-preserving distributed learning by uploading model parameters instead of raw data. With the rise of multimodal data, Multimodal Federated Learning (MFL) integrates FL with multimodal learning to collaboratively process heterogeneous data. However, existing MFL approaches face challenges from modality disparities and data heterogeneity, with random client selection hindering effective multimodal integration. This paper proposes a multimodal federated learning framework based on cross-modal contrastive learning and approximate Shapley value-driven client selection (FedCMAS). FedCMAS aligns modal representations locally and employs differential privacy for secure transmission. On the server side, it prioritizes high-contribution clients using Shapley values. Experiments on MSCOCO and Flickr30K demonstrate that FedCMAS surpasses baseline methods in cross-modal retrieval tasks, with ablation studies validating each component’s role in enhancing performance.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

FedCMAS: Cross-Modal Contrastive Learning and Shapley Value-Driven Client Selection in Multimodal Federated Learning

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Ruichun Gu,
  • Chaofeng Li,
  • Zhuolun Li

摘要

Federated Learning (FL) enables privacy-preserving distributed learning by uploading model parameters instead of raw data. With the rise of multimodal data, Multimodal Federated Learning (MFL) integrates FL with multimodal learning to collaboratively process heterogeneous data. However, existing MFL approaches face challenges from modality disparities and data heterogeneity, with random client selection hindering effective multimodal integration. This paper proposes a multimodal federated learning framework based on cross-modal contrastive learning and approximate Shapley value-driven client selection (FedCMAS). FedCMAS aligns modal representations locally and employs differential privacy for secure transmission. On the server side, it prioritizes high-contribution clients using Shapley values. Experiments on MSCOCO and Flickr30K demonstrate that FedCMAS surpasses baseline methods in cross-modal retrieval tasks, with ablation studies validating each component’s role in enhancing performance.