In computer vision, automatic image captioning is a challenging task that requires a combination of image and natural language processing techniques. The goal is to generate accurate and semantically consistent textual descriptions for a given image. Deep learning models have recently achieved significant results, particularly those employing an encoder–decoder architecture with Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). This study presents a new image captioning model based on the EfficientNetB7-LSTM architecture. We use EfficientNetB7, a high-capacity convolutional neural network (CNN) known for its strong performance in image recognition tasks, to extract detailed features from images. These features are then inputted into an LSTM network, effectively capturing long-term dependencies in sequential data and enabling the generation of descriptive and coherent captions. Our model is trained and evaluated on the Flickr30k benchmark dataset comprising 31,783 images, each with five human-written captions. We assess the performance of our model using widely recognized metrics such as BLEU, METEOR, and CIDEr. The EfficientNetB7-LSTM model achieves a BLEU-4 score of 19.64, a METEOR score of 41.41, and a CIDEr score of 47.91, outperforming all other variants tested in our experiments. This research contributes to advancing the study of image captioning by offering a high-performing and scalable architecture.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Integrating EfficientNet and Long Short-Term Memory Network for Effective Image Captioning

  • Anh Kim Su,
  • Loi Quoc Duong,
  • Hai Thanh Nguyen

摘要

In computer vision, automatic image captioning is a challenging task that requires a combination of image and natural language processing techniques. The goal is to generate accurate and semantically consistent textual descriptions for a given image. Deep learning models have recently achieved significant results, particularly those employing an encoder–decoder architecture with Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). This study presents a new image captioning model based on the EfficientNetB7-LSTM architecture. We use EfficientNetB7, a high-capacity convolutional neural network (CNN) known for its strong performance in image recognition tasks, to extract detailed features from images. These features are then inputted into an LSTM network, effectively capturing long-term dependencies in sequential data and enabling the generation of descriptive and coherent captions. Our model is trained and evaluated on the Flickr30k benchmark dataset comprising 31,783 images, each with five human-written captions. We assess the performance of our model using widely recognized metrics such as BLEU, METEOR, and CIDEr. The EfficientNetB7-LSTM model achieves a BLEU-4 score of 19.64, a METEOR score of 41.41, and a CIDEr score of 47.91, outperforming all other variants tested in our experiments. This research contributes to advancing the study of image captioning by offering a high-performing and scalable architecture.