From Vision to Sound: A Review of Deep Learning Methods for 2D Image Processing to Audio Generation
摘要
The convergence of image processing and audio synthesis has fueled growing interest in generating sound from visual inputs using deep learning techniques. This review explores recent advancements in deep generative models, particularly Generative Adversarial Networks (GANs), for converting 2D images into temporally coherent and semantically aligned audio signals. A central theme is the use of intermediate representations, such as spectrograms, which serve as structured, two-dimensional proxies for audio wave-forms. These spectrograms bridge the visual and auditory domains by enabling deep models to translate extracted image features typically derived from CNNs or vision transformers into plausible audio representations. The surveyed approaches are categorized into three major groups: GAN-based models, diffusion-based architectures, and encoder-decoder frameworks. Their performance is analyzed across diverse applications such as sound effect generation, music synthesis, and ambient scene reconstruction. We highlight key challenges in this domain, including cross-modal alignment, limited multimodal datasets, and the demand for more generalizable representation learning. Additionally, we examine the role of spectrogram inversion techniques and vocoder-based audio synthesis in producing high-fidelity waveforms from visual cues. Emerging directions, including multimodal contrastive learning, cross-modal transformers, and vision-conditioned audio generation, are also discussed. This survey offers a structured taxonomy of 2D image-to-audio generation models and outlines future research pathways for advancing spectrogram-guided synthesis using GANs and related deep learning frameworks.