This study examines whether targeted feedback can lead to improvements in nonverbal communication, specifically in speech delivery, by analyzing 3-min videos of three management consultants. The analysis focused on facial expressions, utilizing the Facial Action Coding System (FACS) and vocal characteristics, as represented by Mel-Frequency Cepstral Coefficients (MFCCs). For five months, the authors provided individualized feedback focusing on facial expression and speech delivery, which involves key elements such as speaking pace, intonation, and pitch variation. While the sample size was limited to three individuals, the study leveraged a year’s worth of accumulated video data. The findings indicate that feedback contributed to observable improvements in facial expressions, whereas vocal features derived from MFCCs remained largely unaffected. Furthermore, this study suggests that combining FACS with real-time speech content extraction offers a deeper understanding of how message content and emotional expression interact, and using the T5 model to generate feedback on speech content automatically improves the speech.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Speech Improvement by Multimodal Analysis

  • Kazunori Minetaki,
  • I.-Hsien Ting

摘要

This study examines whether targeted feedback can lead to improvements in nonverbal communication, specifically in speech delivery, by analyzing 3-min videos of three management consultants. The analysis focused on facial expressions, utilizing the Facial Action Coding System (FACS) and vocal characteristics, as represented by Mel-Frequency Cepstral Coefficients (MFCCs). For five months, the authors provided individualized feedback focusing on facial expression and speech delivery, which involves key elements such as speaking pace, intonation, and pitch variation. While the sample size was limited to three individuals, the study leveraged a year’s worth of accumulated video data. The findings indicate that feedback contributed to observable improvements in facial expressions, whereas vocal features derived from MFCCs remained largely unaffected. Furthermore, this study suggests that combining FACS with real-time speech content extraction offers a deeper understanding of how message content and emotional expression interact, and using the T5 model to generate feedback on speech content automatically improves the speech.