ViT-LSTM: Image Caption Generation
摘要
This paper presents a novel image captioning approach that integrates vision transformer (ViT) for feature extraction with long short-term memory (LSTM) networks for caption generation. Leveraging ViT’s self-attention mechanisms, the model effectively captures complex visual dependencies, while the LSTM decoder translates these features into coherent and contextually accurate captions. Using the Flickr8k dataset, the model demonstrates strong performance, achieving high BLEU scores (BLEU-1: 0.9961, BLEU-4: 0.9901), indicating near-perfect alignment with human-annotated captions. This approach surpasses traditional CNN-RNN architectures by generating more fluent and precise descriptions, bridging visual content with natural language. Future improvements may focus on enhancing caption diversity and exploring additional evaluation metrics.