We present a novel approach for automating character recognition and dialogue extraction in comics for dubbing applications. Our system operates in four main phases: character detection with gender identification, speech balloon localization with text extraction, text correction, and voice dubbing. We leverage YOLOv11 for object detection and OCR for text retrieval, followed by transformer-based text correction. This paper focuses on the first two phases, presenting effective solutions for extracting characters, dialog balloon and their associated dialogues. Our experimental results demonstrate the practical applicability of the proposed methodology with high accuracy rates in character-dialogue mapping for the Doraemon comic series. We have obtained positive results that can be applied in practice.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Speech Balloon and Character Association for Comics

  • Trung Thong Nguyen,
  • Hong Quang Ngo,
  • Van Tuan Doan,
  • Thi Bich Thuy Nguyen,
  • Van Tri Vu,
  • Minh Thuy Ta

摘要

We present a novel approach for automating character recognition and dialogue extraction in comics for dubbing applications. Our system operates in four main phases: character detection with gender identification, speech balloon localization with text extraction, text correction, and voice dubbing. We leverage YOLOv11 for object detection and OCR for text retrieval, followed by transformer-based text correction. This paper focuses on the first two phases, presenting effective solutions for extracting characters, dialog balloon and their associated dialogues. Our experimental results demonstrate the practical applicability of the proposed methodology with high accuracy rates in character-dialogue mapping for the Doraemon comic series. We have obtained positive results that can be applied in practice.