<p>Automatic detection of personality traits from text has potential applications in recruitment, psychological assessment, and human–computer interaction. However, progress in this field has been limited by reliance on a single dataset, the lack of standardized evaluation protocols, and the absence of systematic comparisons involving recent large language models (LLMs). This study addresses these gaps by systematically comparing traditional machine learning (ML), deep learning (DL), and LLMs, including BERT, GPT−3.5, and GPT-4, for detecting the Big Five personality traits from text. We evaluate all models on the Stream of Consciousness Essays (SoCE) dataset under a unified experimental protocol with threshold-independent metrics (ROC curves and AUC scores), enabling fair comparison across model families. To examine cross-dataset applicability, we introduce the Behavioral Interview Data (BID) dataset, comprising 94 responses to behavioral interview questions with personality labels derived from the IPIP-50 psychometric test. On SoCE, all model families achieved modest performance, with LLMs slightly outperforming ML and DL approaches. When applied to BID, no clear performance degradation was observed, though the small sample size prevents definitive conclusions. Among all models, GPT-4 emerged as the top performer, particularly on BID, where contextually structured interview responses appeared to provide stronger predictive signals than free-form essays. Despite these findings, current performance levels remain insufficient for practical deployment, and further improvements on larger, more diverse datasets are needed.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Advancing text-based personality detection: a comparative study of machine learning models on behavioral interview data and stream of consciousness essays

  • Wael Khreich,
  • Rima Wehbe

摘要

Automatic detection of personality traits from text has potential applications in recruitment, psychological assessment, and human–computer interaction. However, progress in this field has been limited by reliance on a single dataset, the lack of standardized evaluation protocols, and the absence of systematic comparisons involving recent large language models (LLMs). This study addresses these gaps by systematically comparing traditional machine learning (ML), deep learning (DL), and LLMs, including BERT, GPT−3.5, and GPT-4, for detecting the Big Five personality traits from text. We evaluate all models on the Stream of Consciousness Essays (SoCE) dataset under a unified experimental protocol with threshold-independent metrics (ROC curves and AUC scores), enabling fair comparison across model families. To examine cross-dataset applicability, we introduce the Behavioral Interview Data (BID) dataset, comprising 94 responses to behavioral interview questions with personality labels derived from the IPIP-50 psychometric test. On SoCE, all model families achieved modest performance, with LLMs slightly outperforming ML and DL approaches. When applied to BID, no clear performance degradation was observed, though the small sample size prevents definitive conclusions. Among all models, GPT-4 emerged as the top performer, particularly on BID, where contextually structured interview responses appeared to provide stronger predictive signals than free-form essays. Despite these findings, current performance levels remain insufficient for practical deployment, and further improvements on larger, more diverse datasets are needed.