Developing foundation models that generalize across diverse tasks is a long-standing goal in artificial intelligence. In recommendation systems, however, existing foundation models primarily focus on generative tasks while overlooking embedding-based tasks, and they often suffer from challenges in multi-task learning such as knowledge conflict and inconsistent convergence. To address these limitations, We propose RecFound, a unified generative and representation learning framework for recommendation foundation models. We construct the first comprehensive dataset covering both generative and embedding tasks across diverse scenarios. Building on this dataset, RecFound introduces a novel multi-task optimization scheme with three major components: a Task-wise Mixture of Low-rank Experts (TMoLE) to mitigate knowledge conflict and enhance sharing, a Step-wise Convergence-oriented Sample Scheduler (S2Sched) to balance convergence across tasks, and a Model Merge mechanism to further improve joint performance. Extensive experiments demonstrate that RecFound achieves state-of-the-art performance across a variety of recommendation tasks, significantly outperforming existing baselines. The dataset and code are available at https://github.com/JunkFood436/RecFound .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Generative Representational Learning of Foundation Models for Recommendation

  • Zheli Zhou,
  • Chenxu Zhu,
  • Jianghao Lin,
  • Bo Chen,
  • Ruiming Tang,
  • Yong Yu,
  • Weinan Zhang

摘要

Developing foundation models that generalize across diverse tasks is a long-standing goal in artificial intelligence. In recommendation systems, however, existing foundation models primarily focus on generative tasks while overlooking embedding-based tasks, and they often suffer from challenges in multi-task learning such as knowledge conflict and inconsistent convergence. To address these limitations, We propose RecFound, a unified generative and representation learning framework for recommendation foundation models. We construct the first comprehensive dataset covering both generative and embedding tasks across diverse scenarios. Building on this dataset, RecFound introduces a novel multi-task optimization scheme with three major components: a Task-wise Mixture of Low-rank Experts (TMoLE) to mitigate knowledge conflict and enhance sharing, a Step-wise Convergence-oriented Sample Scheduler (S2Sched) to balance convergence across tasks, and a Model Merge mechanism to further improve joint performance. Extensive experiments demonstrate that RecFound achieves state-of-the-art performance across a variety of recommendation tasks, significantly outperforming existing baselines. The dataset and code are available at https://github.com/JunkFood436/RecFound .