Scholarlink

<p>Non-intrusive personality detection identifies an individual’s personality traits without directly asking them questions or conducting personality tests. Research on personality prediction is growing in various applications such as career guidance, healthcare, education, social interactions, marketing, and advertising. Presently, the methods for non-intrusive personality detection are based on multiple modalities of video, audio, and text with limited accuracy as they don’t use cross-modality relations between features learned from each modality. This work explores a novel approach to learning cross-modality relations between the modalities using a cross-hierarchical multi-head attention and fusion network (CHMAFN). This architecture uses a multi-head attention mechanism to understand and capture the cross modality relations between modalities extract the relevant features at different levels of abstraction and fuse the attention to obtain the cross relation within and between the modalities. The proposed modal extracts deep learning features from multiple modalities of video, audio, and text. Then, video features are enhanced using modified DenseNet architecture, and text and audio features are enhanced using 1D CNN. Next, these features are enriched using cross-hierarchical attention and fusion networks to classify them to the big five personality traits using multivariate long short-term memory (LSTM.). The proposed network is experimented with CharLearn First Impression Dataset and the performance of the framework was found to be efficient with an accuracy of 93.97, which is 1.3% higher compared to existing works Cross Attention Transformer, MMMU-BA, Deep Multimodal Fusion, Multitask Transformer, ERINE and Cross Attention with Bilinear Transformation. The F1 score of CHMAFN increased at least by 1.32% with balanced and higher precision and recall values, which is at least 7% higher and this balance is due to the avoidance of learning bias through feature enrichment. The model CHMAFN is tested against three different datasets, VPTD (Video Dataset for Personality traits detection), UDIVA 0.5, and MATRICS (Multimodal-Task-Oriented Group Discussion), and recorded a 3 to 5% improvement in the result.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Deep cross-modal integration with hierarchical multi-head attention for big five personality prediction

  • 期刊论文

  • 发表日期：

    2025年7月12日

查看全文
  • Revathi S,
  • G. S. Nagaraja,
  • Farooque Azam

摘要

Non-intrusive personality detection identifies an individual’s personality traits without directly asking them questions or conducting personality tests. Research on personality prediction is growing in various applications such as career guidance, healthcare, education, social interactions, marketing, and advertising. Presently, the methods for non-intrusive personality detection are based on multiple modalities of video, audio, and text with limited accuracy as they don’t use cross-modality relations between features learned from each modality. This work explores a novel approach to learning cross-modality relations between the modalities using a cross-hierarchical multi-head attention and fusion network (CHMAFN). This architecture uses a multi-head attention mechanism to understand and capture the cross modality relations between modalities extract the relevant features at different levels of abstraction and fuse the attention to obtain the cross relation within and between the modalities. The proposed modal extracts deep learning features from multiple modalities of video, audio, and text. Then, video features are enhanced using modified DenseNet architecture, and text and audio features are enhanced using 1D CNN. Next, these features are enriched using cross-hierarchical attention and fusion networks to classify them to the big five personality traits using multivariate long short-term memory (LSTM.). The proposed network is experimented with CharLearn First Impression Dataset and the performance of the framework was found to be efficient with an accuracy of 93.97, which is 1.3% higher compared to existing works Cross Attention Transformer, MMMU-BA, Deep Multimodal Fusion, Multitask Transformer, ERINE and Cross Attention with Bilinear Transformation. The F1 score of CHMAFN increased at least by 1.32% with balanced and higher precision and recall values, which is at least 7% higher and this balance is due to the avoidance of learning bias through feature enrichment. The model CHMAFN is tested against three different datasets, VPTD (Video Dataset for Personality traits detection), UDIVA 0.5, and MATRICS (Multimodal-Task-Oriented Group Discussion), and recorded a 3 to 5% improvement in the result.