Research on Generalization of U-Net Intermediate Block Design for Music Source Separation
摘要
Music source separation (MSS) refers to the separation of all the single all the sources from the mixture of multiple audio sources. An U-Net architecture is widely used in this issue, and a query-based conditioned U-Net can deal with the performance degradation and inefficient memory usage problem of generating multiple outputs, which also enables zero-shot MSS. In this paper, we perform a series of experiments modifying the convolution layers design (both 1-D and 2-D blocks) of intermediate blocks in the U-Net architecture. The 1-D blocks only captures the frequency characteristics and the 2-D blocks captures both temporal and frequency characteristics. For the dataset, we divide it into two categories, and some instruments only appear in the test set to test the generalization ability of the model Results shows that for the instruments “seen” in the training process, the 2-D blocks has the best result and its the overall performance is also the best. However for the “unseen” instruments, the 1-D blocks performs best. The results suggest that compared to the temporal characteristics, the frequency characteristics are more universal among different musical instruments.