<p>Conversational gestures are essential for enabling natural and intuitive interactions with digital humans. However, the development and application of such gestures remain constrained by the scarcity of high-quality, scalable motion data. In this paper, we introduce a novel approach for controllable data acquisition and generation that combines human design, motion capture, and software synthesis to significantly reduce production costs, accelerate production speed, and ensure high-quality, precisely controlled training data. Building on this foundation, we propose an efficient method for synthesizing high-quality conversational gestures. Our framework integrates a motion feature extraction network extracted from a Transformer-based vector quantized variational autoencoder (VQ-VAE), and a cascaded gesture generation network based on Gate Recurrent Unit (GRU) with adversarial training. To further improve multi-modal data alignment and optimize data utilization, we introduce a dynamic replay buffering technique. Extensive experiments demonstrate the effectiveness of our methodology, showcasing its potential to advance both academic research and practical applications of conversational gestures for digital humans.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhanced data techniques and optimization in conversational gesture generation

  • Xiang Wang,
  • Yifeng Peng,
  • Zhaoxiang Liu,
  • Kai Wang,
  • Shiguo Lian

摘要

Conversational gestures are essential for enabling natural and intuitive interactions with digital humans. However, the development and application of such gestures remain constrained by the scarcity of high-quality, scalable motion data. In this paper, we introduce a novel approach for controllable data acquisition and generation that combines human design, motion capture, and software synthesis to significantly reduce production costs, accelerate production speed, and ensure high-quality, precisely controlled training data. Building on this foundation, we propose an efficient method for synthesizing high-quality conversational gestures. Our framework integrates a motion feature extraction network extracted from a Transformer-based vector quantized variational autoencoder (VQ-VAE), and a cascaded gesture generation network based on Gate Recurrent Unit (GRU) with adversarial training. To further improve multi-modal data alignment and optimize data utilization, we introduce a dynamic replay buffering technique. Extensive experiments demonstrate the effectiveness of our methodology, showcasing its potential to advance both academic research and practical applications of conversational gestures for digital humans.