Scholarlink

<p>It is a significant yet challenging task to represent categorical attributes in mixed-attribute data as low-dimensional compact numerical vectors through a two-stage categorical representation learning (i.e., feature extraction and representation learning). Most existing categorical representation learning methods either extract features from a single view or rely on the shallow structure-based representation learning. The former fails to fully capture the feature information of categorical data, while the latter cannot learn deep-level feature representations like deep learning. To address these issues, this paper proposes a BERT–BiLSTM-based Categorical Representation learning method (BBCR) for the mixed-attribute data classification. To be more precise, in the feature extraction stage, the intrinsic features of categorical values are comprehensively extracted from multiple different views, including the Intra-attribute, Inter-attribute, and Attribute-Class. In this stage, the problems of redundancy, noise, and high-dimensional features in the Inter-attribute view are alleviated by using the conditional information entropy extraction method and the symmetric uncertainty dimension reduction method. In the representation learning stage, inspired by the text representation technology in the natural language processing, we introduce the BERT–based deep learning to perform categorical representation learning on the multi-view features obtained in the above stage (i.e., the feature extraction stage). Additionally, BiLSTM is employed to enhance the interaction between long-distance categorical values, ultimately representing categorical values as low-dimensional compact numerical vectors. The representation learning stage of BBCR adopts an end-to-end deep representation learning architecture. Although this incurs certain computational overhead, its powerful representation capabilities make it particularly suitable for high-dimensional, large-scale, and complex mixed-attribute data scenarios, highlighting the inherent demand of this method for high-performance computing (HPC) resources. Extensive experiments on 26 mixed-attribute datasets with diverse characteristics demonstrate that BBCR significantly improves the representation performance compared to state-of-the-art baseline methods.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A BERT–BiLSTM-based categorical representation learning for mixed-attribute data

  • 期刊论文

  • 发表日期：

    2026年3月7日

查看全文
  • Qiude Li,
  • Yinglong Pan,
  • Shengfen Ji,
  • Sigui Hu,
  • Yang Yu,
  • Zhongwen Pan

摘要

It is a significant yet challenging task to represent categorical attributes in mixed-attribute data as low-dimensional compact numerical vectors through a two-stage categorical representation learning (i.e., feature extraction and representation learning). Most existing categorical representation learning methods either extract features from a single view or rely on the shallow structure-based representation learning. The former fails to fully capture the feature information of categorical data, while the latter cannot learn deep-level feature representations like deep learning. To address these issues, this paper proposes a BERT–BiLSTM-based Categorical Representation learning method (BBCR) for the mixed-attribute data classification. To be more precise, in the feature extraction stage, the intrinsic features of categorical values are comprehensively extracted from multiple different views, including the Intra-attribute, Inter-attribute, and Attribute-Class. In this stage, the problems of redundancy, noise, and high-dimensional features in the Inter-attribute view are alleviated by using the conditional information entropy extraction method and the symmetric uncertainty dimension reduction method. In the representation learning stage, inspired by the text representation technology in the natural language processing, we introduce the BERT–based deep learning to perform categorical representation learning on the multi-view features obtained in the above stage (i.e., the feature extraction stage). Additionally, BiLSTM is employed to enhance the interaction between long-distance categorical values, ultimately representing categorical values as low-dimensional compact numerical vectors. The representation learning stage of BBCR adopts an end-to-end deep representation learning architecture. Although this incurs certain computational overhead, its powerful representation capabilities make it particularly suitable for high-dimensional, large-scale, and complex mixed-attribute data scenarios, highlighting the inherent demand of this method for high-performance computing (HPC) resources. Extensive experiments on 26 mixed-attribute datasets with diverse characteristics demonstrate that BBCR significantly improves the representation performance compared to state-of-the-art baseline methods.