Emotional reactivity and sensory saturation in subtitled multimedia: effects of high-fidelity vibrokinetic stimulation
摘要
Haptic technologies are being increasingly used to augment traditional multimedia experiences. While there is growing evidence on the benefits of haptic stimulation to improve the quality of experience (QoE), our understanding of how additional stimulation can influence multimedia experiences that already involve multiple streams of audiovisual (AV) sensory information is limited. To address this gap, a within-subject laboratory experiment was conducted to assess the effect of high-fidelity vibrokinetic (HFVK) stimulation delivered by a cinema seat, in accordance with the non-verbal auditory stimuli of subtitled 2D cinematic opera, on the audience’s viewing experience. Subjects’ visual attention to subtitles, emotional reactivity, and QoE of subjects were measured via fixation time in the subtitle zone, electrodermal activity (EDA), automatic facial expression (AFE) based discrete emotions frequency, Likert scales, and semi-structured interviews. Our results suggest that, although HFVK stimulation and subtitle presence individually enhance emotional reactivity, their combination does not produce an additive effect. Overall, this study raises the potential issue of sensory processing saturation by adding HFVK stimulation to AV experiences that already involve multiple streams of sensory information. Our study further discusses guidelines for assessing implicit emotional reactivity to subtitled multimedia.