DreamDancer: Music-Driven Dance Video Intelligent Generation
摘要
Music-driven dance generation technology converts audio signals into expressive body movements and is widely used in virtual content creation and intelligent performing arts. However, this task faces challenges in cross-modal alignment, temporal consistency, and visual fidelity, especially in accurately capturing the rhythm of music and mapping it to complex movements. To address these issues, we propose DreamDancer, a cross-modal framework that integrates motion generation and high-quality video synthesis. First, we design a Music-to-Pose module that combines Graph Convolutional Networks with temporal attention to generate rhythm-consistent and stylistically coherent dance sequences. Subsequently, we employ a diffusion-based video synthesis pipeline that is conditioned on both skeletal sequences and a reference image, enhancing the realism and visual quality of generated movements. To mitigate facial identity distortion, we incorporate a learnable face enhancement module based on ArcFace, which applies adaptive feature fusion to improve facial consistency across video frames. Additionally, to increase the model’s ability to generalize across different dance styles, we build a multi-ethnic, multi-modal dance dataset, which enriches cultural and stylistic diversity. Experimental results and user studies show that DreamDancer outperforms existing methods in dance quality, music synchronization, and facial consistency, demonstrating strong practical potential.