Scholarlink

We introduce InternVideo2, a new family of video foundation models (ViFM) that achieve the state-of-the-art results in video recognition, video-text tasks, and video-centric dialogue. Our core design is a progressive training approach that unifies the masked video modeling, crossmodal contrastive learning, and next token prediction, scaling up the video encoder size to 6B parameters. At the data level, we prioritize spatiotemporal consistency by semantically segmenting videos and generating video-audio-speech captions. This improves the alignment between video and text. Through extensive experiments, we validate our designs and demonstrate superior performance on over 60 video and audio tasks. Notably, our model outperforms others on various video-related dialogue and long video understanding benchmarks, highlighting its ability to reason and comprehend longer contexts.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

InternVideo2: Scaling Foundation Models for Multimodal Video Understanding

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Yi Wang,
  • Kunchang Li,
  • Xinhao Li,
  • Jiashuo Yu,
  • Yinan He,
  • Guo Chen,
  • Baoqi Pei,
  • Rongkun Zheng,
  • Zun Wang,
  • Yansong Shi,
  • Tianxiang Jiang,
  • Songze Li,
  • Jilan Xu,
  • Hongjie Zhang,
  • Yifei Huang,
  • Yu Qiao,
  • Yali Wang,
  • Limin Wang

摘要

We introduce InternVideo2, a new family of video foundation models (ViFM) that achieve the state-of-the-art results in video recognition, video-text tasks, and video-centric dialogue. Our core design is a progressive training approach that unifies the masked video modeling, crossmodal contrastive learning, and next token prediction, scaling up the video encoder size to 6B parameters. At the data level, we prioritize spatiotemporal consistency by semantically segmenting videos and generating video-audio-speech captions. This improves the alignment between video and text. Through extensive experiments, we validate our designs and demonstrate superior performance on over 60 video and audio tasks. Notably, our model outperforms others on various video-related dialogue and long video understanding benchmarks, highlighting its ability to reason and comprehend longer contexts.