Satellite image captioning integrates image processing and language processing to generate descriptions for satellite images, supporting applications in disaster management, environmental monitoring, and urban planning. Using the DeepGlobe Land Cover dataset, this study employs a U-net model to segment images into land cover types and extract their proportions. These proportions are incorporated into predefined template sentences, forming rule-based sentences, which are paraphrased using the T5 Language model. Sentence-based and neighbourhood-based strategies are employed to fill in the blank spaces inside paraphrased template sentences. Fine-tuning and utilising ChatGPT further enhances descriptive text generation, enabling an efficient approach to satellite image captioning. The study evaluates outputs using ROUGE, BERT similarity scores, and human assessments. While ChatGPT excelled in generating novel and detailed descriptions achieving ROUGE-1 (0.23), ROUGE-2 (0.01), ROUGE-L (0.22), and BERT (0.78) scores, fine-tuned T5 showed moderate improvement but had limitations. Future work involves fine-tuning ChatGPT, comparing its performance with models like LSTM for comprehensive evaluation, and applying the pipeline to larger satellite image datasets for enhanced results.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Satellite Image Captioning

  • Roxana Bagheri

摘要

Satellite image captioning integrates image processing and language processing to generate descriptions for satellite images, supporting applications in disaster management, environmental monitoring, and urban planning. Using the DeepGlobe Land Cover dataset, this study employs a U-net model to segment images into land cover types and extract their proportions. These proportions are incorporated into predefined template sentences, forming rule-based sentences, which are paraphrased using the T5 Language model. Sentence-based and neighbourhood-based strategies are employed to fill in the blank spaces inside paraphrased template sentences. Fine-tuning and utilising ChatGPT further enhances descriptive text generation, enabling an efficient approach to satellite image captioning. The study evaluates outputs using ROUGE, BERT similarity scores, and human assessments. While ChatGPT excelled in generating novel and detailed descriptions achieving ROUGE-1 (0.23), ROUGE-2 (0.01), ROUGE-L (0.22), and BERT (0.78) scores, fine-tuned T5 showed moderate improvement but had limitations. Future work involves fine-tuning ChatGPT, comparing its performance with models like LSTM for comprehensive evaluation, and applying the pipeline to larger satellite image datasets for enhanced results.