Vocal2Instrument: Instrumental Sound Synthesis from Vocals Using a Hybrid Digital Signal Processing-Convolutional Neural Network Architecture
摘要
Music creation depends heavily on Digital Audio Workstations; the complexity and steep learning curves of these workstations can break the creative momentum of music creation. This paper introduces Vocal2Instrument, an end-to-end system that converts hummed vocals into instrumental arrangements without manual annotation or expertise in Digital Audio Workstations. In contrast to prior approaches, Vocal2Instrument uniquely applies Digital Signal Processing techniques for pre-processing, specifically using Log-Mel spectrograms and onset-based segmentation to address Convolutional Neural Network’s poor temporal alignment abilities, followed smoothly by pitch classification and Musical Instrument Digital Interface (MIDI) synthesis in a single pipeline. Using the HUMTRANS dataset, this architecture produces clean note boundaries and achieves an average pitch error of 4.78 MIDI units, surpassing both Digital Signal Processing-only and Convolutional Neural Network-only baselines. The model decreases manual labor by an estimate of 60% and generates full MIDI arrangements as well as instrumental wav files, Vocal2Instrument accelerates and simplifies vocal-to-instrument conversions, making the conversion faster and more accessible for musicians without technical backgrounds.