Image-based geolocalization is a crucial task with many applications, ranging from environmental monitoring and disaster response to navigation through fighting disinformation; however, assigning a precise geographical location to an image based solely on visual content involves overcoming multiple significant challenges. The first one lies in the variety of landscapes ranging from urban areas to remote zones, such as deserts or forests, each with distinct visual characteristics. Additionally, the same place could differ in weather, seasons, lighting, and angles thus further complicating the task. Machine learning tackled this problem by trying to locate images in a specific region, or biome, of interest. A global approach has been attempted from just a handful of studies, such as [8] or [6]. Usually, the training of these models is a very challenging and expensive task, so since it is a common trend in computer science to use Large Language Models (LLMs from now on) to solve tasks that previously required specific training, we tried a new approach based on these new generative models. Moreover, after the advent of multimodal models, such as ChatGPT-4o [10] or LLava [7], it is possible to extend the analysis to images. In this paper, our goal is to evaluate how these models perform in global geolocalization. We extracted 2,000 images from a pre-existing dataset [3] composed of 10,000 Google tool Street View images along the coordinates of the location where they were taken. These images were mainly taken in remote areas from different biomes, such as deserts, forests, and other similar environments, as shown in Fig. 2, but also include some urban scenes. We limited the dataset to 2,000 images, similarly to [8], where the same number of images was utilized. To process the dataset we supplied both ChatGPT-4o and LLava with the prompt, shown in Fig. 1, and one of the images in our data. The output of our models gave us three predictions containing both the approximate coordinates of the location and the state they were in, then we chose just the most accurate one. Each image is analyzed separately from the other to avoid context contamination between different iterations of the process. Due to computational performance and unstable outputs, we narrowed down the dataset used for LLava to 250 images.  Our results, especially for ChatGPT-4o are promising. We found that ChatGPT-4o correctly guessed the state approximately 76.5 \(\%\) of the time. This high success rate is reinforced by the fact that in the 23.5 \(\%\) of cases where the model guessed the state incorrectly, 60.5 \(\%\) of these incorrect guesses were within 750 km of the correct state. This indicates that even in those cases where the state is mistaken, it often provides a geographically accurate prediction. Contrary LLava performed quite poorly on this task: we guessed correctly the state only in the 20.9 \(\%\) of the cases and only 24.5 \(\% \) of the incorrect guesses were within 750 km of the correct state. Due to the different methods used, we needed to adapt our metrics to compare our results to the state-of-the-art. The first metric followed the existing literature and used the distance between real and computed coordinates as a proxy for different levels of geographical accuracy (750 km is the radius threshold for the state), the second checked if the country of the input coordinates matches the state obtained as input and the third is the union of the two previous cases. As shown in Table 1, we obtained results similar to the state-of-the-art since our results are better than [6] in one metric out of the three, and better than [5] in two out of three. Finally, it is also important to underline the effect that these results will have. Given the usage simplicity, its versatility, and its efficiency this type of geolocation can impact many different fields. It can easily increase situational awareness during crises, enabling emergency services, law enforcement, and professionals to identify where incidents are occurring to respond more effectively. For example, these models could be used as threat mitigation in scenarios, such as those described in [1], or during natural disasters [11]. Within realms like open-source intelligence (OSINT), this type of geolocation can help track and analyze threats, gather information, and improve the accuracy of analyzed data. Moreover, it can help verify the authenticity of information found on social media, as the usage of unaltered images in a new, but false context is one of the most prevalent methods to mislead the public. By cross-referencing the claimed location of an event with actual geolocation data, anyone can confirm whether the information is credible thus reducing the impact of out-of-context images. This result is especially helpful as online coordinated behaviour has already proven to spread disinformation during the 2019 UK election [9] and the 2020 United States election [12], and while the disinformation can easily spread on social media, such as telegram [2] and twitter [4]. However, there are notable drawbacks to consider, concerning potential violation of privacy. Geolocation data can be misused to harm individuals, revealing their private information and location without consent. This can lead to dangerous activities such as swatting, where, due to false reports the victim is raided by police, but also it can put individuals participating in witness protection programs at great risk (Fig. 3).

Geo-Localization Using Multimodal Large Language Models

会议论文
发表日期：

2026年

查看全文

Lorenzo Alvisi

摘要

Image-based geolocalization is a crucial task with many applications, ranging from environmental monitoring and disaster response to navigation through fighting disinformation; however, assigning a precise geographical location to an image based solely on visual content involves overcoming multiple significant challenges. The first one lies in the variety of landscapes ranging from urban areas to remote zones, such as deserts or forests, each with distinct visual characteristics. Additionally, the same place could differ in weather, seasons, lighting, and angles thus further complicating the task. Machine learning tackled this problem by trying to locate images in a specific region, or biome, of interest. A global approach has been attempted from just a handful of studies, such as [8] or [6]. Usually, the training of these models is a very challenging and expensive task, so since it is a common trend in computer science to use Large Language Models (LLMs from now on) to solve tasks that previously required specific training, we tried a new approach based on these new generative models. Moreover, after the advent of multimodal models, such as ChatGPT-4o [10] or LLava [7], it is possible to extend the analysis to images. In this paper, our goal is to evaluate how these models perform in global geolocalization. We extracted 2,000 images from a pre-existing dataset [3] composed of 10,000 Google tool Street View images along the coordinates of the location where they were taken. These images were mainly taken in remote areas from different biomes, such as deserts, forests, and other similar environments, as shown in Fig. 2, but also include some urban scenes. We limited the dataset to 2,000 images, similarly to [8], where the same number of images was utilized. To process the dataset we supplied both ChatGPT-4o and LLava with the prompt, shown in Fig. 1, and one of the images in our data. The output of our models gave us three predictions containing both the approximate coordinates of the location and the state they were in, then we chose just the most accurate one. Each image is analyzed separately from the other to avoid context contamination between different iterations of the process. Due to computational performance and unstable outputs, we narrowed down the dataset used for LLava to 250 images. Our results, especially for ChatGPT-4o are promising. We found that ChatGPT-4o correctly guessed the state approximately 76.5 \(\%\) of the time. This high success rate is reinforced by the fact that in the 23.5 \(\%\) of cases where the model guessed the state incorrectly, 60.5 \(\%\) of these incorrect guesses were within 750 km of the correct state. This indicates that even in those cases where the state is mistaken, it often provides a geographically accurate prediction. Contrary LLava performed quite poorly on this task: we guessed correctly the state only in the 20.9 \(\%\) of the cases and only 24.5 \(\% \) of the incorrect guesses were within 750 km of the correct state. Due to the different methods used, we needed to adapt our metrics to compare our results to the state-of-the-art. The first metric followed the existing literature and used the distance between real and computed coordinates as a proxy for different levels of geographical accuracy (750 km is the radius threshold for the state), the second checked if the country of the input coordinates matches the state obtained as input and the third is the union of the two previous cases. As shown in Table 1, we obtained results similar to the state-of-the-art since our results are better than [6] in one metric out of the three, and better than [5] in two out of three. Finally, it is also important to underline the effect that these results will have. Given the usage simplicity, its versatility, and its efficiency this type of geolocation can impact many different fields. It can easily increase situational awareness during crises, enabling emergency services, law enforcement, and professionals to identify where incidents are occurring to respond more effectively. For example, these models could be used as threat mitigation in scenarios, such as those described in [1], or during natural disasters [11]. Within realms like open-source intelligence (OSINT), this type of geolocation can help track and analyze threats, gather information, and improve the accuracy of analyzed data. Moreover, it can help verify the authenticity of information found on social media, as the usage of unaltered images in a new, but false context is one of the most prevalent methods to mislead the public. By cross-referencing the claimed location of an event with actual geolocation data, anyone can confirm whether the information is credible thus reducing the impact of out-of-context images. This result is especially helpful as online coordinated behaviour has already proven to spread disinformation during the 2019 UK election [9] and the 2020 United States election [12], and while the disinformation can easily spread on social media, such as telegram [2] and twitter [4]. However, there are notable drawbacks to consider, concerning potential violation of privacy. Geolocation data can be misused to harm individuals, revealing their private information and location without consent. This can lead to dangerous activities such as swatting, where, due to false reports the victim is raided by police, but also it can put individuals participating in witness protection programs at great risk (Fig. 3).