Scholarlink

This paper presents a novel multi-stage framework for automatic video understanding and storytelling. The proposed system first identifies keyframes using a temporal segmentation strategy to extract the most informative moments from the input video. Each selected keyframe is then processed through the YOLO object detection model to identify and classify relevant objects within the scene. Finally, a large language model (GPT-4o) is employed to generate coherent and context-aware natural language descriptions, transforming sequences of visual data into a structured narrative. One of the key strengths of this approach is its computational efficiency: by focusing only on representative keyframes rather than processing the entire video, the system significantly reduces resource consumption without compromising narrative quality. This integrated approach bridges the gap between low-level visual perception and high-level semantic interpretation, enabling applications in video summarization, assistive technologies, and multimedia content generation. Experimental results on a benchmark video dataset demonstrate the potential of the system to produce meaningful and fluent storylines that reflect the visual content.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

From Keyframes to Narrative: A Multi-stage AI Pipeline for Scene Understanding Using Object Detection and Large Language Models

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Valentín Calzada-Ledesma,
  • Victor Castillo-Olivetto,
  • Emmanuel Márquez-Robles,
  • Carlos Orozco-Solis,
  • Faustino Neri-Larios

摘要

This paper presents a novel multi-stage framework for automatic video understanding and storytelling. The proposed system first identifies keyframes using a temporal segmentation strategy to extract the most informative moments from the input video. Each selected keyframe is then processed through the YOLO object detection model to identify and classify relevant objects within the scene. Finally, a large language model (GPT-4o) is employed to generate coherent and context-aware natural language descriptions, transforming sequences of visual data into a structured narrative. One of the key strengths of this approach is its computational efficiency: by focusing only on representative keyframes rather than processing the entire video, the system significantly reduces resource consumption without compromising narrative quality. This integrated approach bridges the gap between low-level visual perception and high-level semantic interpretation, enabling applications in video summarization, assistive technologies, and multimedia content generation. Experimental results on a benchmark video dataset demonstrate the potential of the system to produce meaningful and fluent storylines that reflect the visual content.