<p>Multimodal session-based recommendations aim to describe the preferences of anonymous users based on short sessions, requiring the fusion of data from different modalities. However, these modalities use distinct representation methods, which creates challenges in data fusion. Existing multimodal fusion methods primarily focus on integrating modal information, while neglecting the influence of complex factors and external environments. The key challenges are: (1) How can relevant semantics be extracted from heterogeneous descriptive information that may contain noise or be incomplete? (2) How can these heterogeneous descriptions be integrated fairly to address the suboptimal problem and provide a comprehensive inference of user interests? (3) How can we dynamically handle changes in multimodal data quality for each modality? To address these issues, we propose a novel Data Enhancement and Dynamic Fusion of multimodal Session-Based Recommendation (DEDFSBR) method. The method consists of four components for modeling: the feature extraction module, the multimodal fusion module, the balanced learning module, and the dynamic calibration strategy. Specifically, a pseudo-modal contrastive learning approach is designed to enhance the representation learning of descriptive information. Building on this, a hierarchical pivot transformer is proposed to integrate heterogeneous descriptive information, and a balanced learning module is introduced to optimize the balance between the modals, thereby improving recommendation accuracy. A dynamic calibration strategy is employed to address the uncertainty in modal data quality. Additionally, Gaussian distributions are used to represent numerical information, and a Wasserstein self-attention mechanism is designed to handle probabilistic influence models. Experiments on three real-world datasets demonstrate the effectiveness of the proposed method. Further analysis reveals that DEDFSBR can effectively alleviate the cold start problem in session-based recommendations.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A novel multimodal session-based recommendation method with data enhancement and dynamic fusion

  • Yongquan Fan,
  • Tao Chen,
  • Yajun Du,
  • Xianyong Li

摘要

Multimodal session-based recommendations aim to describe the preferences of anonymous users based on short sessions, requiring the fusion of data from different modalities. However, these modalities use distinct representation methods, which creates challenges in data fusion. Existing multimodal fusion methods primarily focus on integrating modal information, while neglecting the influence of complex factors and external environments. The key challenges are: (1) How can relevant semantics be extracted from heterogeneous descriptive information that may contain noise or be incomplete? (2) How can these heterogeneous descriptions be integrated fairly to address the suboptimal problem and provide a comprehensive inference of user interests? (3) How can we dynamically handle changes in multimodal data quality for each modality? To address these issues, we propose a novel Data Enhancement and Dynamic Fusion of multimodal Session-Based Recommendation (DEDFSBR) method. The method consists of four components for modeling: the feature extraction module, the multimodal fusion module, the balanced learning module, and the dynamic calibration strategy. Specifically, a pseudo-modal contrastive learning approach is designed to enhance the representation learning of descriptive information. Building on this, a hierarchical pivot transformer is proposed to integrate heterogeneous descriptive information, and a balanced learning module is introduced to optimize the balance between the modals, thereby improving recommendation accuracy. A dynamic calibration strategy is employed to address the uncertainty in modal data quality. Additionally, Gaussian distributions are used to represent numerical information, and a Wasserstein self-attention mechanism is designed to handle probabilistic influence models. Experiments on three real-world datasets demonstrate the effectiveness of the proposed method. Further analysis reveals that DEDFSBR can effectively alleviate the cold start problem in session-based recommendations.