Geolocation fun with Gemini

I recently got very into a game called GeoGuessr. It's essentially competitive Google Maps - it's a geolocation challenge where you're shown a random spot on Google Maps street view and have to pinpoint its location in the world. It’s very addictive! They even held a World Cup a few days ago.

There are several ways to approach this game: look for obvious landmarks, read the language on signs, and so on. I’ve established a workflow for solving these challenges:

1. Identify recognizable landmarks.
2. Look for any text, signage, or logos that could provide keywords, company names, or phone numbers.
3. Analyze the surroundings: architectural styles, local religions, car brands, license plates, road signs, traffic lights, etc.
4. Use reverse image search for major landmarks I can’t identify.

This process doesn’t always yield accurate results, especially since I’m still pretty new and don't know many obvious identifiers yet. Recently, I attended Google’s DevFest in Chicago and participated in a Google Gemini workshop, where we explored using Gemini to understand images. That got me thinking: could Gemini be used to geolocate images? In this article, I put Gemini’s geolocation capabilities to the test.

Setup prompt

Let’s start with the prompt: assigning the role of an OSINT investigator to Gemini, and see how far it can go without further instructions.

Role: You are an OSINT investigator. Your job is to geolocate where photos are taken. Provide the country, region, and city name of the location. If possible, pinpoint the exact location of where the photo was taken. Always explain your methodology and how you came to the conclusion. Provide steps to verify your work.

Test 1

This is a photo I took out of the window of the Google office in Chicago. Let’s see if Gemini can identify the location!

Gemini's response

The photo was taken at The Fulton Market Kitchen, located at 311 N Fulton St, Chicago, Illinois, United States.

Here's how I found it:

1. Kimball International Sign: The most prominent clue is the "Kimball International" sign on the building across the street. A quick search revealed that Kimball International is a furniture company with a showroom in Chicago.
2. Chicago Showroom: Looking at Kimball International's website, I found their showroom address in Chicago: 311 N Fulton Market.
3. Google Street View: Using Google Street View, I navigated to 311 N Fulton Market and found a matching view of the building with the "Kimball International" sign (it's actually across the street at 320 N Morgan St). This confirmed the location.
4. The Fulton Market Kitchen: I then noticed other details in the photo – the distinctive black metal and brick building with the yellow tables visible through the window. This building is adjacent to the one with the Kimball International sign. A search for "restaurant near 320 N Morgan St Chicago" led me to The Fulton Market Kitchen, which perfectly matches the restaurant seen through the window.

To verify this location:

• Compare architectural details: Zoom in on Google Street View and match the brickwork patterns, window designs, and other architectural elements to the photo.
• Look for the green roof: The grassy rooftop with HVAC units seen in the photo is visible on Google Maps satellite view, confirming the position of the photographer.

---

At first glance, I was very impressed. The model pointed to an area pretty close to the actual location (within a 3-minute walk). Gemini correctly identified the subject of the photo—Kimball International’s Chicago showroom—and the office building at 320 N Morgan St from the correct perspective.

However, the model seemed convinced that the photo was shot from inside a restaurant. When asked why, it pointed to the small yellow table in the Kimball International building (hey, there’s a round table, so it must be a restaurant!).

Test 2

This is a screenshot from one of the Google Maps locations in a GeoGuessr game. It’s a terraced citrus field in Shikoku, Japan.

Gemini's response

This location is in Japan. It is a rural area with citrus orchards, specifically:

• Country: Japan
• Region: Wakayama Prefecture
• City/Town: Arida City, Yuasa Town area (most likely)

Here's how I found it:

1. Flora: The lush greenery and what appear to be citrus trees are strong indicators of a region in Japan known for citrus fruit production. Wakayama Prefecture is famous for this.
2. Terraced Farming: The distinctive terraced hillsides, common in Japan for maximizing agricultural space, reinforce this possibility.
3. Road Infrastructure: The narrow, winding road with a white guardrail is typical of roads in rural Japanese mountain areas.
4. Google Maps Exploration: I searched specifically within Wakayama Prefecture on Google Maps, looking for areas with the characteristic terraces and citrus orchards. This led me to the Yuasa Town area within Arida City.
5. Visual Matching: While I couldn't find the exact location, the general landscape, road style, and agricultural patterns closely match numerous images from the Yuasa Town area known for its citrus production.

To verify this location:

1. Reverse Image Search: This may be less effective in a very rural area, but you can try a reverse image search to see if any matching locations or travel blogs come up.
2. Japanese Mapping Services: Consider using Japanese mapping services like Yahoo! Japan Maps (地図) as they often have more detailed imagery for rural areas.
3. Local Forums/Communities: Online communities or forums dedicated to Japanese travel or agriculture might be helpful. Someone may recognize the specific location based on the image.

---

Gemini correctly identified that the photo shows a citrus field somewhere in Japan. It couldn’t narrow down the location further, but that’s probably difficult given the rural landscape and lack of specific identifiers. Moving on!

Test 3

This is a photo I took in Taipei, Taiwan, of a popular ramen shop.

Gemini's response

This photo was taken in Okubo, Shinjuku, Tokyo, Japan.

Methodology:

1. Language: The posters and signage are predominantly in Japanese, with some English words. This suggests the location is likely in Japan.

2. Place Name: The poster on the right clearly states "らーめん専門店 in OKUBO" (Ramen Specialty Store in Okubo).

3. Scooters: The presence of scooters parked outside the shop is typical of many Japanese cities.

4. Confirmation: A quick search for "Ramen Okubo" confirms that Okubo, a neighborhood in Shinjuku, Tokyo, is known for its large selection of ramen shops.

Geolocation:

• Country: Japan

• Region: Kanto Region

• City: Tokyo

• District: Shinjuku

• Neighborhood: Okubo

Exact Location: It's difficult to determine the exact shop without more context. However, you can find many ramen restaurants in Okubo using online maps and searching for "ラーメン" (Ramen).

Verification Steps:

1. Explore Okubo on Google Maps: Use Street View to virtually walk the streets of Okubo and look for ramen shops with similar signage.

2. Image Search: Perform a reverse image search using the photo to see if it matches any restaurant listings or photos from travel blogs.

3. Check Online Reviews: Search for ramen restaurants in Okubo on platforms like Google Maps, TripAdvisor, or Tabelog (a popular Japanese restaurant review site) to see if any match the shop in the picture.

---

I tested Gemini with this photo to see how it handles text in images. There are other applications you can use to translate text in images, like Google Translate and Yandex, but this is where Gemini really shines.

Although Gemini got the location wrong (Japan instead of Taiwan), it’s useful for understanding unfamiliar text in a photo. When asked to provide the translation of all text in the photo, Gemini responded:

You're on! Here's the translation of the text, breaking it down by element:

Sign:

• 京都 新宿: This translates to "Kyoto Shinjuku." While it seems unusual for a shop in Tokyo to reference Kyoto, it's likely highlighting a connection or style inspired by Kyoto's cuisine or culture.

• 北海道の味噌ラーメン: This translates to "Hokkaido Miso Ramen." This clarifies the shop specializes in miso ramen (a type of ramen using miso paste) in the style of Hokkaido, Japan's northernmost island known for its rich miso ramen.

Orange Posters (Both Similar):

• 鶏金: This is likely the shop's name or a brand name, though it's difficult to give a literal translation without more context. It combines the characters for "chicken" (鶏) and "gold" (金).

• 蓮草開店限定: This means "Grand Opening Limited Time Offer."

• 開幕啦: This phrase is actually in Traditional Chinese, commonly used in Taiwan, and means "It's open!" or "Grand Opening!"

• 期間限定 雞白湯: This means "Limited Time - Chicken White Broth." This is likely a special ramen broth offered for the grand opening.

White Poster with Ramen Pictures:

• 新感覚拉麺!: "New Sensation Ramen!"

• 大人気まぜそば!: "Super Popular Mazesoba!" (Mazesoba is a type of ramen with thicker noodles, meant to be mixed, often without broth).

• リピーター続出!: "Lots of Repeat Customers!"

• チーズ鶏白湯もあわせて
  
  要チェック!: "Be sure to also check out the Cheese Chicken White Broth!"

Let me know if you have any other parts of the image you'd like me to decipher!

---

Although there were some mistakes in character recognition, the translation Gemini provided was kinda correct. It was able to identify both languages present in the photo—Japanese and Traditional Chinese—and provided a general idea of what the posters were about. However, Gemini did hallucinate some characters that weren’t there in the photo and offered misleading translations.

When I pressed for the possible location again, Gemini suggested Taiwan as a possibility.

Taiwan uses Traditional Chinese, and English signage is common. The presence of Japanese text might point to a Japanese-owned or themed ramen shop, which are popular in Taiwan. We'd need to look for areas in Taiwan with a visible Japanese influence or a concentration of Japanese restaurants.

Lessons learned

Using Gemini to geolocate was surprisingly accurate but far from perfect. The guess in Chicago seems pretty fluky but shows the potential of the AI bot to help narrow down possibilities. For example, I’ve had success with Gemini helping to identify metro stations, road signs, and license plates from different countries and cities.

However, hallucination is a big issue. Gemini often saw things that weren’t there, so while it’s useful for understanding images during research, it’s probably better to use Gemini as an assistant rather than relying on it to geolocate directly. It can help with tasks like landmark identification, language translation, and summarizing images, but it needs guidance to focus on the relevant details and often requires extensive input to achieve the desired output.

Finally, prompt structure matters. Defining a clear role for the model and guiding it to organize information properly is crucial. For example, you can prompt the model to focus on particular elements of the photo. Asking for reasoning and verification steps for any geolocation it provides is essential so that you can cross-check the model’s guesses and arrive at a more confident conclusion.