Efficient video captioning annotation: a semi-automated framework via LLM-driven heuristics
摘要
The rapid advancement of Multimodal Large Language Models (MLLMs) has revolutionized video understanding and caption generation. However, traditional dataset construction remains a bottleneck. It relies heavily on manual annotation, which is not only prohibitively expensive but also prone to inconsistency due to the linguistic subjectivity of different annotators. To address these challenges, this study proposes a semi-automated video captioning framework based on LLM-driven heuristic reasoning. By integrating the generative capabilities of Large Language Models (LLMs) with structured prompt constraints, we achieve high-quality caption generation with minimal human intervention. Specifically, the framework utilizes GPT-4o to generate initial annotations, which are then refined using a human-guided strategy, where a small set of manually annotated seed videos provides standard instructions and heuristic rules. These constraints guide the model to perform iterative semantic rewriting and syntactic alignment, significantly narrowing the quality gap between automatic and manual annotations while drastically reducing labor costs. Using this framework, we construct Dense-HVC, a multi-source heterogeneous dataset comprising 1,041 videos and 20,820 parallel annotations, covering diverse scenarios. We validate the dataset using the VLM baseline. Experimental results demonstrate that our iteratively optimized dataset yields performance comparable to fully manually annotated datasets. This research presents a scalable, cost-effective alternative to traditional annotation methods, offering new perspectives for efficient multimodal semantic understanding.