The Phonetic Modification and Facial Movements Made During Mandarin Vowel and Tone Production in Noise
摘要
In noisy environments, speakers usually adopt “Lombard speech” to improve speech intelligibility. This is characterized by acoustic expansion of vowel and tonal spaces, as well as exaggerated facial movements. However, no study has simultaneously examined acoustic and visual enhancement, or their potential interaction in noise. Furthermore, the visibility of interlocutors may influence the degree of enhancement. Mandarin Chinese, a tonal language that utilizes both vowels and tones to convey meaning, can offer a comprehensive perspective on speech production. Ten participants described animations using seven disyllabic Mandarin words containing three target vowels (/a/, /i/, /u/) and four lexical tones. Listeners reconstructed animations based on these descriptions. Interactions were recorded under four conditions with varying noise and visibility. Acoustic and visual data were analyzed using Praat and OpenFace, respectively. Results demonstrated enhanced facial movements in noise. Movement patterns vary across different interactive conditions: Vowel production showed distinct lip movements but minimal head motion, while Tone 3 production involved unique eyebrow and head movements. When interlocutors were invisible, speakers produced greater facial movements without concomitant expansion of acoustic vowel or tonal space, suggesting partial complementarity between acoustic and visual enhancement. In summary, this study showed that acoustic realization and facial movements co-occur and interact in Lombard speech. Furthermore, acoustic and visual enhancements were complementary to some extent, varying with different interaction conditions.