Scholarlink

<p>Federated learning has gained significant attention for enabling effective model training on distributed data. However, in practice, most client data is unlabeled, and reliable communication with a central server is often infeasible at scale. To address this, we consider a more realistic setting where clients hold only unlabeled data and can communicate only with neighbors and propose DeNAV, a novel decentralized self-supervised learning framework for handling such scenarios. DeNAV simultaneously pre-trains multiple lightweight transformer models across clients. To improve training, we design a navigator algorithm to smartly plan the training route of each model and adopt staleness-aware model aggregation to handle the discrepancy of training status between models. DeNAV eliminates the need for server coordination, offering both convergence and consensus guarantees. Extensive experiments show that DeNAV is comparable to state-of-the-art federated self-supervised learning baselines and also surpasses previous decentralized methods with equal communication efficiency.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

DeNAV: Decentralized Self-Supervised Learning with a Training Navigator

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年3月31日

查看全文
  • Xuanyu Chen,
  • Nan Yang,
  • Charles Z. Liu,
  • Dong Yuan

摘要

Federated learning has gained significant attention for enabling effective model training on distributed data. However, in practice, most client data is unlabeled, and reliable communication with a central server is often infeasible at scale. To address this, we consider a more realistic setting where clients hold only unlabeled data and can communicate only with neighbors and propose DeNAV, a novel decentralized self-supervised learning framework for handling such scenarios. DeNAV simultaneously pre-trains multiple lightweight transformer models across clients. To improve training, we design a navigator algorithm to smartly plan the training route of each model and adopt staleness-aware model aggregation to handle the discrepancy of training status between models. DeNAV eliminates the need for server coordination, offering both convergence and consensus guarantees. Extensive experiments show that DeNAV is comparable to state-of-the-art federated self-supervised learning baselines and also surpasses previous decentralized methods with equal communication efficiency.