Text-to-music generation is a task that creates musical compositions from textual descriptions, allowing users to easily and intuitively generate music using natural language. There are two main approaches: text-to-audio-music and text-to-symbolic-music; the latter has the advantage of easy editing of musical elements. This paper proposes a text-to-MIDI generation system that uses musical attributes as a bridge. In contrast to a large single-stage model, two compact models are proposed: text-to-attribute understanding and attribute-to-MIDI generation. For text-to-attribute, BERT is used to extract musical attribute values from plain text in multiple languages. For attribute-to-MIDI, GPT-2 combined with Low-Rank Adaptation is used to optimize training cost and efficiency. Furthermore, a post-processing method based on music theory principles is proposed to ensure the accuracy and integrity of the generated music. With approximately 200 million parameters, the proposed model shows a 9.84% improvement in Average Sample-wise Accuracy and a 2.39 times faster token output speed compared to the MuseCoCo model trained on the same dataset. The source code is available and maintained in the GitHub repository ( https://github.com/HCMUS-Thesis-AI-for-Music-Composition/Brainstorming ).

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

An Enhanced Solution for Multilingual Text-to-MIDI Generation

  • Phi-Hung Ngo,
  • Quoc-Vuong Pham,
  • Duy-Hoang Tran

摘要

Text-to-music generation is a task that creates musical compositions from textual descriptions, allowing users to easily and intuitively generate music using natural language. There are two main approaches: text-to-audio-music and text-to-symbolic-music; the latter has the advantage of easy editing of musical elements. This paper proposes a text-to-MIDI generation system that uses musical attributes as a bridge. In contrast to a large single-stage model, two compact models are proposed: text-to-attribute understanding and attribute-to-MIDI generation. For text-to-attribute, BERT is used to extract musical attribute values from plain text in multiple languages. For attribute-to-MIDI, GPT-2 combined with Low-Rank Adaptation is used to optimize training cost and efficiency. Furthermore, a post-processing method based on music theory principles is proposed to ensure the accuracy and integrity of the generated music. With approximately 200 million parameters, the proposed model shows a 9.84% improvement in Average Sample-wise Accuracy and a 2.39 times faster token output speed compared to the MuseCoCo model trained on the same dataset. The source code is available and maintained in the GitHub repository ( https://github.com/HCMUS-Thesis-AI-for-Music-Composition/Brainstorming ).