ViT-GPT2: Vision Transformer Based Automatic Image Captioning
摘要
Automatic image caption generation using deep learning is rapidly bridging computer vision with natural language processing. Envision a system that might be able to carefully study a picture and then describe it in natural language. This technology has huge potential for totally revolutionizing the way interactions with images are undertaken, greatly improving accessibility for visually challenged users, more accurate image search, and streamlined image organization. It makes use of Vision Transformers, especially the ViT-GPT2 model for the extraction of image features from Flickr8k dataset. Features extracted in this stage are further fed into the Vision Encoder Decoder Model for processing to output grammatically correct and contextually relevant captions. For proposed model, Standard evaluation metrics used are BLEU, METEOR, GLEU, and ROUGE-L. The experimental results show a great improvement over the present approaches. Obtained results on Flickr8k dataset are BLEU-1 score 0.7024, METEOR score 0.5127, GLEU score 0.2431 and ROUGE-L score 0.5491. It is a model that underscores important progress to be achieved in AI systems for sense-making and describing visual content in the natural language.