An Effective Image Captioning Using Vision Language Models
摘要
Image captioning utilizes natural language processing and computer vision to generate textual descriptions of images. This makes it easier to tag media and makes it more accessible. This work assesses contemporary Vision-Language Models (VLMs) including Qwen2.5-VL (7B) and LLaMA 3.2 Vision (11B), which exemplify a novel generation of architectures designed for enhanced multimodal comprehension. We used QLoRA, a memory- and parameter-efficient method that allows for efficient adaptation of large-scale models through 4-bit quantization, to fine-tune the VLMs. The models were trained on the Flickr30k dataset using tokenized and padded captions to set a baseline for how well they worked. We used a wide range of evaluation metrics to rate the quality of the captions. These included BLEU-1/2, ROUGE-1/2/L, BERT Score, METEOR, CIDEr, SPICE, and Distinct-1/2. The results show that VLM-based models do better on most metrics, which shows that large, pre-trained multimodal models are becoming more useful for complex vision-language tasks.