Scholarlink

<p>Synthesising appropriate choreographies from music remains an open problem, due to the need to align musical semantics with subjective human motions. We introduce MDLT, a novel approach that frames the choreography generation problem as a translation task. Our method leverages the AIST++ and PhantomDance data sets to learn to translate sequences of audio into corresponding dance poses. We present two variants of MDLT: MDLT-T based on the Transformer architecture, and MDLT-M based on the Mamba architecture, for strong long-horizon sequence modeling. Trained on these data sets, our method enables a robotic arm to dance, and can generalize to humanoid robots through its architecture-agnostic design. Evaluation metrics, including Average Joint Error and Fréchet Inception Distance, consistently demonstrate that, when given a piece of music, MDLT excels at producing realistic and high-quality choreography. The code will be made available upon acceptance.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Music to dance as language translation using sequence models

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月5日

查看全文
  • André Correia,
  • Luís A. Alexandre

摘要

Synthesising appropriate choreographies from music remains an open problem, due to the need to align musical semantics with subjective human motions. We introduce MDLT, a novel approach that frames the choreography generation problem as a translation task. Our method leverages the AIST++ and PhantomDance data sets to learn to translate sequences of audio into corresponding dance poses. We present two variants of MDLT: MDLT-T based on the Transformer architecture, and MDLT-M based on the Mamba architecture, for strong long-horizon sequence modeling. Trained on these data sets, our method enables a robotic arm to dance, and can generalize to humanoid robots through its architecture-agnostic design. Evaluation metrics, including Average Joint Error and Fréchet Inception Distance, consistently demonstrate that, when given a piece of music, MDLT excels at producing realistic and high-quality choreography. The code will be made available upon acceptance.