This entry explores the role of multimodal corpora in language research and teaching, by highlighting their potential for enhancing data-driven learning (DDL). A defining feature of multimodal corpora is the alignment of diverse data streams—such as video, audio, speech transcriptions, and annotations of gesture, prosody, and spatial relations—which together can offer a rich understanding of communication. By incorporating multimodality, DDL shifts from an exclusive focus on written text to a broader range of semiotic resources, thereby enabling learners to observe language use in authentic, real-world contexts. In this way, multimodal DDL supports contextually grounded language learning and allows for the development of language skills based on multiple modes, including speech, visual cues, and body language. The entry also addresses the practical and methodological challenges of building multimodal corpora, including the interpretive nature of transcription, the variable depth of annotation, and the need for tools to support data collection and analysis. It discusses the use of concordances—both form- and meaning-oriented—as a means of analysing multimodal data, each with distinct implications for analytical depth and time investment. Finally, the entry compares the use of ready-made versus custom-built multimodal corpora, weighing the trade-offs between accessibility and pedagogical relevance, and shows how recent advances in artificial intelligence can assist teachers with little technical expertise in constructing and annotating their own multimodal corpora.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multimodal Tools and Corpora for Data-Driven Learning

  • Tony Berber Sardinha

摘要

This entry explores the role of multimodal corpora in language research and teaching, by highlighting their potential for enhancing data-driven learning (DDL). A defining feature of multimodal corpora is the alignment of diverse data streams—such as video, audio, speech transcriptions, and annotations of gesture, prosody, and spatial relations—which together can offer a rich understanding of communication. By incorporating multimodality, DDL shifts from an exclusive focus on written text to a broader range of semiotic resources, thereby enabling learners to observe language use in authentic, real-world contexts. In this way, multimodal DDL supports contextually grounded language learning and allows for the development of language skills based on multiple modes, including speech, visual cues, and body language. The entry also addresses the practical and methodological challenges of building multimodal corpora, including the interpretive nature of transcription, the variable depth of annotation, and the need for tools to support data collection and analysis. It discusses the use of concordances—both form- and meaning-oriented—as a means of analysing multimodal data, each with distinct implications for analytical depth and time investment. Finally, the entry compares the use of ready-made versus custom-built multimodal corpora, weighing the trade-offs between accessibility and pedagogical relevance, and shows how recent advances in artificial intelligence can assist teachers with little technical expertise in constructing and annotating their own multimodal corpora.