This paper explores the integration of sentiment analysis into image caption generation. Image captioning utilizes deep learning techniques like CNNs and RNNs to generate descriptive captions for images. Sentiment analysis, on the other hand, focuses on determining emotions and opinions expressed in text. The proposed system combines these two techniques to improve image captioning by ensuring emotionally appropriate captions. The proposed system is evaluated using the Flickr8k dataset for image captioning and the Sentiment140 dataset for sentiment analysis. It utilizes an end-to-end approach, extracting visual features from images with a pre-trained VGG16 model and generating captions using an LSTM decoder. For sentiment analysis, \(TF-IDF\) vectorization and word embeddings are used as feature extraction methods. Experimental results demonstrate that the integrated sentiment analysis enhances the emotional appropriateness of the captions that are produced. The BLEU score evaluates the image caption generator, while F1-score and accuracy assess the sentiment analysis models.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Sentiment Analysis in Image Caption Generation

  • Amit Kumar Trivedi,
  • Kashish Singla,
  • Sachit Dhiman

摘要

This paper explores the integration of sentiment analysis into image caption generation. Image captioning utilizes deep learning techniques like CNNs and RNNs to generate descriptive captions for images. Sentiment analysis, on the other hand, focuses on determining emotions and opinions expressed in text. The proposed system combines these two techniques to improve image captioning by ensuring emotionally appropriate captions. The proposed system is evaluated using the Flickr8k dataset for image captioning and the Sentiment140 dataset for sentiment analysis. It utilizes an end-to-end approach, extracting visual features from images with a pre-trained VGG16 model and generating captions using an LSTM decoder. For sentiment analysis, \(TF-IDF\) vectorization and word embeddings are used as feature extraction methods. Experimental results demonstrate that the integrated sentiment analysis enhances the emotional appropriateness of the captions that are produced. The BLEU score evaluates the image caption generator, while F1-score and accuracy assess the sentiment analysis models.