To estimate the Big Five personality traits (Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism), this work combines face expressions, voice analysis, and demographic data in a novel multi-modal technique. Our system analyses facial features from video frames using a three-pronged approach: (1) a vision-based model using VGG16 as a feature extractor; (2) speech and text analysis using BERT-based models; and (3) demographic analysis including age and gender data using Random Forest Regression. We evaluate our approach using the VPTD dataset and show that over single-modality approaches, integration of numerous modalities significantly raises prediction accuracy. With a test loss of 0.00011, our visual model exhibits good prediction ability. We also use real-time stress detection with eyebrow movement tracking and emotion identification to provide a whole personality evaluation framework. Experimental evidence shows that our multi-modal approach outperforms single-modality baselines, so the decision-level fusion of several modalities produces more accurate and strong personality predictions than any single component. This study develops the growing topic of computational personality evaluation by offering a full, real-time personality detection system with applications in psychology, human-computer interaction, and personalised services.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multi-modal Personality Trait Detection: Integrating Facial, Vocal, and Demographic Features for Big Five Personality Prediction

  • Nikhil Singh

摘要

To estimate the Big Five personality traits (Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism), this work combines face expressions, voice analysis, and demographic data in a novel multi-modal technique. Our system analyses facial features from video frames using a three-pronged approach: (1) a vision-based model using VGG16 as a feature extractor; (2) speech and text analysis using BERT-based models; and (3) demographic analysis including age and gender data using Random Forest Regression. We evaluate our approach using the VPTD dataset and show that over single-modality approaches, integration of numerous modalities significantly raises prediction accuracy. With a test loss of 0.00011, our visual model exhibits good prediction ability. We also use real-time stress detection with eyebrow movement tracking and emotion identification to provide a whole personality evaluation framework. Experimental evidence shows that our multi-modal approach outperforms single-modality baselines, so the decision-level fusion of several modalities produces more accurate and strong personality predictions than any single component. This study develops the growing topic of computational personality evaluation by offering a full, real-time personality detection system with applications in psychology, human-computer interaction, and personalised services.