geo-sleuth

geo-sleuth (v2)

Goal: find a location that holds up under checking. Every conclusion must point to the clue it used, the command it ran and the file it produced.

${CLAUDE_SKILL_DIR} in commands means the directory containing this SKILL.md (the same for commands in references). Claude Code substitutes it automatically; in other agents, first export CLAUDE_SKILL_DIR=<absolute path of this directory> before running (if the shell doesn't keep variables between commands, prefix every command with it), or replace it with that path directly.

The work is split into three layers. Remember this first, then read the flow:

The method comes from breaking down 14 photo-geolocation (网络迷踪) creator videos and 22 puzzles, plus several rounds of blind-test comparisons; the breakdown notes are not published with the repo. The lesson of v1: rules written as prose don't get executed, and two runs of the same skill version gave very different results; so v2 puts every rule that can be written as code into board.py.

Hard rules (always in force; ★ = enforced by board.py, just follow it)

1. If the user says the photo isn't theirs, or the image shows a private residence or minors, ask once what it's for before continuing; when the prompt already states the source or purpose (an evaluation puzzle, a puzzle setter's hint, the user says they took it), don't ask, and go to the finest level as usual.
2. Don't fabricate verification. "Measured on the map", "matched street view", "±10 m" must correspond to commands actually run and files actually produced in this session. If you didn't run it, write "unverified".
3. Precision needs a source: an error radius ≤100 m requires the intersection of two independent constraints, or a ground-level match on ≥3 invariant features. Before reporting a camera position, run a self-check: pose.py project --horizon <sea-horizon row> computes what the pitch should be and each feature's "expected row" for the current camera position; reconcile that with the photo (the depression angle to a target ≠ the camera's pitch; look up the camera position's elevation with terrain.py elev — if the height difference doesn't match the depression angles in the image, the camera position is wrong).
4. Once you have a unique anchor, close the loop: all later search and geometry starts from the anchor; don't go back to generic features like "blue-green lake water" or "tropical park" and pick the most famous place of that kind.
5. Compute bearings first, then identify structures: which block in satellite imagery is the tower or chimney in the photo is itself an interpretation. First compute its true bearing in the image with sun.py compass or a confirmed landmark, then use geo.py bearings to see which outline around the candidate camera position falls on that bearing; when nothing fits, first try both templates ("original/mirrored", "sunrise/sunset"); if not verified, only lower the tier, don't exclude.
6. ★ List the whole category before inferring: when inferring a region from "hill city", "tropical", "IP in a municipality", use board.py children to add all subordinate admin divisions as candidates, then rank them with evidence. Don't default to the main urban district, don't pick by population.
7. Metadata and hints are hypotheses: when EXIF, IP location, location tags or the puzzle setter's words conflict with the image, the image wins; don't invent a story to reconcile them. IP says country A but the hint or the regulation designs (plates, signs) point to another continent: intersect with clues.py lookup territories <country A> --continent <continent>.
8. Self-consistency: redo each key judgment once with a different crop or keyword set; if the two results differ by tens of kilometers or more, lower the tier.
9. ★ If you can't pin a point, give a range + what information is missing; when the candidates are a few discrete ones, don't take the midpoint: the main answer of board.py report is always the top-ranked candidate; the rest go into alternatives with discriminating tests.
10. ★ Exclusion and confirmation use the same standard: board.py exclude only accepts read/computed clues + a computed file (output of geo.py frame, terrain.py, etc.); observed and inferred clues can only down-weight via evidence --against, with the likelihood ratio clamped to 1/3–3 (inferred) or 1/5–5 (observed). Generating candidates from "there's an X nearby" is also filtering, and X must be something confirmed in the image. Fine-level candidates such as campuses, residential compounds and street segments must go on the board too: add a batch at once with board.py add --from <poi.py --out / osm.py geom output> --level area/road, don't hand-pick; when you drop one, write evidence --against with a comparison image, and check lists candidates without a single piece of evidence (meaning nobody looked at them); labels from third-party data (tree species in a municipal inventory, an exact school-name match) can only down-weight, never justify exclusion (in two cases the ground truth was skipped by hand exactly this way). The excluded extent must be ≤ the evidence extent: for candidates with extent, such as districts, areas and roads, exclude needs --covers lat,lon[:lat,lon] stating which stretch the evidence covers; the script rejects coverage below half (excluding a whole road after looking at one point on it failed in two cases).
11. ★ Population and fame are not evidence; scan order follows "share ÷ pages" (board.py next): finish the small districts first, put large districts last with a page cap.

Flow

You don't have to go through every step: if reverse image search in step 1 hits directly, jump to steps 6 and 7 to confirm. Every step's output goes on the candidate board.

Step 1: steps 0–3 in one command

uv run ${CLAUDE_SKILL_DIR}/scripts/intake.py photo.jpg --out-dir intake/ [--box x0,y0,x1,y1 ...]
uv run ${CLAUDE_SKILL_DIR}/scripts/board.py init --photo photo.jpg

intake.py usually takes 1–2 minutes, longer on the first run because it installs dependencies: give the command a generous timeout (10+ minutes) or run it in the background. If a command timeout interrupts it, the reverse-image-search subprocesses may still be writing into rev/, but intake.md won't be generated; don't treat it as finished.

intake.md contains: metadata, OCR text (lines read from upscaled/tiled passes are marked pass=up/tile and are hypotheses), Baidu similar images (source-site counts + numbered contact sheet), reverse-image-search labels with tiered vote counts, likely residential compound/development names, the list of edge crops, and failed items. Then you do four things:

• Look at the image: go through every crop in edges/ (four edges, four corners); run through the checklist in references/observe.md; log each clue with board.py clue "<text>" --kind <kind> --status observed|read|inferred|computed --file <zoomed crop>. Be honest about status: text you read is read; "the building is probably 8 floors", "the road goes uphill" are inferred.
• Lookup tables: plates, area codes, calling codes, driving side, overseas territories → clues.py lookup <kind> <value>; for those that resolve to an admin division, use board.py apply --kind plate --value 渝G --file <zoomed crop> directly (adds candidates and evidence automatically; the other candidates at the same level are only down-weighted, not excluded).
• Reverse-image-search results: first open rev/<name>_baidu_similar.jpg (the top-left tile is the query image) and look for near-duplicates of the same object or the same scene; if there are any, go by number to similar[i].from in the JSON to see the source page; compound, development or hotel names in the labels → poi.py "<name>" --city <city> --out pois.json to get coordinates; for same-name hits (several campuses, several branches) put them all on the board with board.py add --from pois.json --level area, then check; always open the screenshots, and once a post hits, look through the rest of its photo set. Choose where to search by object type (references/search.md).
• Hints and metadata: log each as an inferred clue and state how credible it is; IP location only says where the person was when posting, and the posting time is not the capture time.

Step 2: list all candidates, score evidence, check the next step

uv run ${CLAUDE_SKILL_DIR}/scripts/board.py children <parent admin area>          # municipality → all districts; country → first-level admin divisions (gazetteer.py queries OSM, with bbox)
uv run ${CLAUDE_SKILL_DIR}/scripts/board.py evidence --clue K1 --for <candidate A>:5 --for <candidate B>:2 --against <candidate C>:0.3 --why "…" --file <comparison image>
uv run ${CLAUDE_SKILL_DIR}/scripts/board.py rank
uv run ${CLAUDE_SKILL_DIR}/scripts/board.py next

next only ever says one of two things:

• Can't separate: run discriminating tests from cheap to expensive, each one on all candidates at once — lookup tables → terrain (plain vs. hill city, tiles.py fetch --zoom 13 or terrain.py) → vehicle livery (revimg.py --query "<城市> <颜色> 公交" (Chinese query: "<city> <color> bus"); read route signs from the result images, compare the stripe along the bus rear by district) → street fixtures (baidu_pano.py sample --bbox <built-up area> --n 24) → river/road-network templates. Still can't separate after all of them: don't stop; scan in the "share ÷ pages" order it gives.
• Ready to narrow: first narrow the extent to the built-up area with board.py urban <candidate> (or give it by hand with scan-bbox), then write a falsification condition with board.py falsify <candidate> --text "…", then go to step 3.

The coarse environmental-location rules still apply: terrain before river width and building color; phenology must be paired with the month; take the intersection of rare facility combinations; recognizable species are only an exclusion tool; when all you have is a land type, first narrow to that type of land with a land-cover map (references/clues/).

Step 3: pick a branch to narrow down

Step 4: finding the spot in satellite imagery — the machine ranks first

uv run ${CLAUDE_SKILL_DIR}/scripts/sat_scan.py grid --bbox <scan_bbox> --zoom 17 --preset track --multi-scale --top 30 --out sat.json --sheet sat_top.jpg --heat heat.jpg
uv run ${CLAUDE_SKILL_DIR}/scripts/poi.py "<district> 学校" --city <prefecture-level city> --out schools.json      # Chinese query "<district> school"; seeds: --seeds schools.json boosts nearby cells
uv run ${CLAUDE_SKILL_DIR}/scripts/sat_scan.py points --points big.json --preset factory --out r.json --sheet r.jpg   # rank candidate points from osm.py buildings / poi.py

• Presets: track (running tracks), stadium, factory, silo, dam, bridge, quarry, solar, greenhouse, port; custom --query. In tests: across 300-odd cells of an urban area, more than half of the OSM-mapped running tracks made the top 30; in an old town in China where OSM is blank, a school's sports field still ranked first. It ranks, it doesn't decide: look at the thumbnails of the top 20–30 cells, then check them against the bearing and shape in the photo.
• Translate the image description into top-down features before looking: curved buildings, octagonal pavilion roofs, sports courts, parking spaces perpendicular to a railway; for tall buildings look at the base; imagery has a date.
• With many candidates, make a candidate table: one row per point, one column per criterion directly visible in the photo.
• Nothing in the top 30 matches: first go back to the "candidates down-weighted by inference" listed by board.py check and to the falsification conditions, then switch preset or go to z18, and only then widen the area.

Step 5: confirmation — street view is ranked first too

uv run ${CLAUDE_SKILL_DIR}/scripts/baidu_pano.py scan <lat,lon> --radius 300 --out panos.json                     # China; outside China use gsv.py
uv run ${CLAUDE_SKILL_DIR}/scripts/match.py rank --query photo.jpg --panos panos.json --toward <landmark lat,lon> --spread 15 --refine sift --top 10 --out m.json --sheet m.jpg
uv run ${CLAUDE_SKILL_DIR}/scripts/match.py rank --query photo.jpg --items around.index.json --render gsv --spread-headings -30,0,30 --out m.json --sheet m.jpg

• match.py coarse-ranks by DINOv2 global similarity and fine-ranks by SIFT inliers; in tests, ground-truth street views of the same place from different years all made the top 4. Open only the top 10 and compare invariant features (building outline, window positions, balconies, pole positions, curbs, ridgelines), not vehicles, signs or foliage. ≥2 features for road level, ≥3 for building level. Not a single image with ≥15 inliers doesn't mean it's wrong: with a change of season, an old capture, or the photo taken from the sidewalk while street view was shot from the middle of the road, the ground truth had only 5 and 0–8 inliers in tests (two cases); first open the top 10 and compare invariant features, and only if none fit change --spread-headings or widen --within. The global score gets dominated by season (a blossom-season capture ranks high wherever it is), so compare against a historical capture from the same season as the photo when you can (gsv.py sheet --date).
• With many panorama points, group by date and road; sheet --road <road name> --spread 60 shows one road only; older captures have a more open view.
• No street view doesn't mean you can't confirm: compare ridgelines with terrain.py view --photo; find a name for the candidate facility (OSM name, nearby place name + the facility-type word in the local language) and search news, encyclopedias and official-site photos to compare facade details.
• When street view is many years older than the photo, go by old buildings and permanent structures.

Step 6: fix the camera position and write the output

• Look back: at the matching street-view point, turn 180° and see what's on the photographer's side.
• A camera position needs two independent constraints (geo.py intersect sight-line intersection, geo.py line alignment line, pose.py multi-point solve, camera height, reverse street view); with only one, building-level confidence is at most "medium".
• When there's no measurable shadow, use lit faces for the heading: sun.py facing --lit left --shaded camera. For photos taken from a car, boat or train, state the direction of travel.

uv run ${CLAUDE_SKILL_DIR}/scripts/board.py check                      # before concluding: do exclusions have files, does the main answer have verified evidence, which clues went unused
uv run ${CLAUDE_SKILL_DIR}/scripts/board.py report --merge result.json # main answer = top-ranked; alternatives, exclusions and unused clues are written into result.json automatically
uv run ${CLAUDE_SKILL_DIR}/scripts/evidence.py spec.json --out evidence.jpg

Output:

1. One-sentence conclusion: place + camera position + heading (+ direction of travel and capture time, where applicable)
2. Coordinates: WGS84 and GCJ-02 (geo.py convert), with error radius
3. Evidence image (satellite image marking the camera position and heading wedge + comparison images)
4. Reasoning chain: clue → inference → commands actually run and files produced → extent
5. Tiered confidence (table below). Rate each level separately; the tier you report is the finest level rated "medium" or above: a low building level doesn't drag down a high area level
6. Excluded candidates and reasons, alternatives and discriminating tests, clues not used or not resolved (generated by board.py report)
7. When you can't pin a point: which level is settled + what information is still needed

Use English for tool messages, report headings and generated labels. Preserve source text (OCR, place names, service responses) verbatim as evidence; explain or translate it for the reader. Write the agent's final report in the user's language. In Chinese, use 高/中/低 for high/medium/low and 城市/片区/路/楼/楼层 for city/area/road/building/floor.

Budget and stopping conditions

• intake.py counts as one call; if reverse image search with 3 different keyword sets finds nothing, stop and go back to the checklist for other clues.
• While candidates still can't be separated, don't scan any point one by one (when board.py next says "can't separate", do the discriminating tests first); if they still can't be separated after the cheap tests, scan in the order it gives, the first 3 pages of each candidate's built-up area first.
• Scanning and confirmation always rank first: look at the top 30 cells from sat_scan.py and the top 10 images from match.py; widen only when all of the top ones are wrong, instead of paging through.
• Run osm.py coverage before enumerating candidates with OSM: districts with sparse data get silently excluded; switch to sat_scan.py grid and say so in the conclusion.
• ≤500 panorama points per area; stop when 3 areas don't match, and use board.py report to report which level is settled.
• Before stopping, board.py check: go back through the top 20 candidates, in ranking order, that were down-weighted by inference but not excluded.

Runtime environment

• Python 3.10+, uv, and curl. Always use uv run ${CLAUDE_SKILL_DIR}/scripts/xxx.py; each script declares its dependencies. scripts/ in references is relative to this skill directory.
• On first setup or after a runtime failure, run uv run ${CLAUDE_SKILL_DIR}/scripts/doctor.py; add --network to check service reachability. Read the English checks and fixes before starting an expensive scan. It uploads no photos and does not load ML models.
• Reverse image search uses local Google Chrome, with automatic fallback to Playwright Chromium (uvx playwright install chromium). If neither starts, use intake.py --no-rev and report the skipped search. OCR prefers Apple Vision on macOS and uses RapidOCR elsewhere or as a fallback.
• match.py and sat_scan.py install ML dependencies and download model weights on first use. Allow extra time and disk space.
• Cache goes to .geo-cache/ in the current directory; the candidate board is board.json in the current directory. Script list and data sources: references/data-sources.md.
• macOS has no timeout command; in zsh $var doesn't word-split, so use bash -c or ${=var} in loops. A province-wide Overpass query can take several minutes; run it in the background.