ChatGPT now performs well at GeoGuesser

Original Article ↗ Hacker News Discussion ↗

Performance on GeoGuessr and “Solved Problem” Debate

  • Many commenters report o3/o4 models doing strikingly well: often identifying cities, specific neighborhoods, or being within tens of kilometers, even with EXIF stripped and VPNs used.
  • Others test with vacation photos or random street scenes and see large errors (wrong country or thousands of km off), especially for generic suburbs, dense city streets, or countryside.
  • Benchmarks across multiple models show strong but imperfect scores; some humans and specialized GeoGuessr systems still outperform LLMs.
  • Several argue the title’s “solved” claim is overstated; the models are impressive but not reliably superhuman, especially on obscure or non–Street View locations.

Data Sources, Leakage, and Behavior

  • Concerns that results may rely on EXIF, IP geolocation, or user memory; some experiments confirm it reads EXIF by default and performs worse when memory is disabled.
  • Mixed evidence on IP use: some see guesses biased toward their current city; others get correct remote locations.
  • The model sometimes claims things like “I’ve seen this exact house on Street View,” widely regarded as hallucination, though many assume training on large web image corpora including Street View–like data.
  • Reasoning traces show it cropping and focusing on road markings, architecture, mountains, signage, and “vibes” (e.g., “Scandinavian,” “Seattle”).

Privacy and Threat Models

  • Central thread: the threat model shifts from “skilled, motivated OSINT analyst” to “any stalker with $20/month.”
  • This is seen as a substantial change even if accuracy is imperfect; a 40–100km radius still hugely narrows search for human analysts or hostile actors.
  • Some argue GPS EXIF and existing tools (e.g., Google Lens) were already major privacy holes; others counter that ubiquity and ease of use are what make this qualitatively new.
  • Common advice: don’t upload private or identifying outdoor photos at all; stripping metadata helps but isn’t sufficient long term.

Beyond Games: Military, Forensics, and Intelligence

  • Commenters discuss military/infosec applications: triaging imagery, focusing analysts, possibly supporting targeting (though precision remains an issue).
  • Debate over whether militaries already have more specialized, powerful geo-location models versus being “caught flat-footed” like everyone else.
  • Separate thread on image-forensics: camera sensor noise as a fingerprint that could, in principle, link intimate photos to social profiles once paired with large datasets and AI.

Intelligence, Brain, and Philosophy Tangent

  • Some note the odd profile of abilities: strong at GeoGuessr-like inference but weak at simple tasks like counting rocks, reinforcing that this is a non-human form of intelligence.
  • A long side discussion explores whether brains are “organic computers,” religious vs scientific views of soul/consciousness, and how such paradigms color reactions to AI capabilities.