<p>Enriching multimedia content with affective metadata unlocks opportunities for innovative and enhanced user experiences in recommendation systems, multimedia retrieval, and many other applications. However, it is really challenging to accurately decode human affect. On the one hand, manual procedures for affective annotation are labour-intensive and hardly scalable. On the other hand, content-based approaches rely on the assumption that affective experiences (1) are directly related to the content, (2) are homogeneous for a particular content, and (3) ignore the subjective nature of affective responses. Recent advancements in brain-computer interfacing (BCI) signifies the prospect of partially automating affective annotation by decoding natural affective experiences toward the content. We consider affective annotation of videos based on <i>brainsourcing</i>: crowdsourced affective reactions from brain signals, recorded while participants were watching videos. Our experiments are based on two popular datasets (DEAP and SEED) and three crowdsourcing models. Crowdsourcing models can support affective annotation for all videos in the SEED dataset and most videos in the DEAP dataset. For both datasets and crowdsourcing models, the performance of affective annotation increases with the crowd size and shows increased confidence of the classifiers with larger crowd sizes. For example, the average classification accuracy for binary valence in DEAP is less than 60% for the individual predictions, but increases up to about 70% for a crowd size of six participants and get to about 80% for fourteen participants. Our findings open avenues for utilizing data captured via BCI for understanding and annotating content according to its users’ affective experiences.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Affective annotation of videos from EEG-based crowdsourcing

  • Yoelvis Moreno-Alcayde,
  • Tuukka Ruotsalo,
  • Luis A. Leiva,
  • V. Javier Traver

摘要

Enriching multimedia content with affective metadata unlocks opportunities for innovative and enhanced user experiences in recommendation systems, multimedia retrieval, and many other applications. However, it is really challenging to accurately decode human affect. On the one hand, manual procedures for affective annotation are labour-intensive and hardly scalable. On the other hand, content-based approaches rely on the assumption that affective experiences (1) are directly related to the content, (2) are homogeneous for a particular content, and (3) ignore the subjective nature of affective responses. Recent advancements in brain-computer interfacing (BCI) signifies the prospect of partially automating affective annotation by decoding natural affective experiences toward the content. We consider affective annotation of videos based on brainsourcing: crowdsourced affective reactions from brain signals, recorded while participants were watching videos. Our experiments are based on two popular datasets (DEAP and SEED) and three crowdsourcing models. Crowdsourcing models can support affective annotation for all videos in the SEED dataset and most videos in the DEAP dataset. For both datasets and crowdsourcing models, the performance of affective annotation increases with the crowd size and shows increased confidence of the classifiers with larger crowd sizes. For example, the average classification accuracy for binary valence in DEAP is less than 60% for the individual predictions, but increases up to about 70% for a crowd size of six participants and get to about 80% for fourteen participants. Our findings open avenues for utilizing data captured via BCI for understanding and annotating content according to its users’ affective experiences.