Scholarlink

<p>Visual generation and understanding have rapidly advanced in recent years, propelled by deep learning and the growing availability of large-scale multimodal datasets. Among the most transformative developments are diffusion models, which have achieved state-of-the-art results in image and video generation through a stochastic denoising process. Beyond high-fidelity synthesis, diffusion models have shown remarkable flexibility for controllable generation, multi-modal conditioning, and even visual understanding tasks such as depth estimation and segmentation. In this survey, we provide a comprehensive overview of recent progress at the intersection of diffusion-based modeling and multi-modal visual intelligence. We structure our discussion around five key themes: (1) diffusion-based image and video generation, (2) controllable diffusion via structured conditions, (3) diffusion for visual understanding, (4) multi-modal generation beyond RGB, and (5) unified frameworks for generation and understanding. By reviewing advances across these domains, we aim to outline emerging trends, identify open challenges, and inspire future research toward more general and unified models of visual perception and synthesis.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A survey: advances in multi-modal visual understanding and generation

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月28日

查看全文
  • Dianbing Xi,
  • Zhaoqi Zhou,
  • Youming Wang,
  • Jiepeng Wang,
  • Chi Zhang

摘要

Visual generation and understanding have rapidly advanced in recent years, propelled by deep learning and the growing availability of large-scale multimodal datasets. Among the most transformative developments are diffusion models, which have achieved state-of-the-art results in image and video generation through a stochastic denoising process. Beyond high-fidelity synthesis, diffusion models have shown remarkable flexibility for controllable generation, multi-modal conditioning, and even visual understanding tasks such as depth estimation and segmentation. In this survey, we provide a comprehensive overview of recent progress at the intersection of diffusion-based modeling and multi-modal visual intelligence. We structure our discussion around five key themes: (1) diffusion-based image and video generation, (2) controllable diffusion via structured conditions, (3) diffusion for visual understanding, (4) multi-modal generation beyond RGB, and (5) unified frameworks for generation and understanding. By reviewing advances across these domains, we aim to outline emerging trends, identify open challenges, and inspire future research toward more general and unified models of visual perception and synthesis.