diff --git a/README.md b/README.md index c27a515..20695cb 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,115 @@ -# openclaw-dnd-city-visualization-skill +# Map-true town and city renders for tabletop campaigns +An agent skill plus a small toolkit for turning a hand-drawn town map into photoreal, *geometrically consistent* +pictures: the first view from the road, the gate, an aerial of the market block, a shopfront, the taproom at +midnight when it is packed, the same street in a blizzard. Every picture of a place agrees with every other picture +of it, because they all descend from one 3D block model traced from your map. + +Developed for a Norse-themed D&D campaign by Rick Falkvinge, with the agents that ran the render lanes. Shared so other +tables can use it with their own maps, their own renderer accounts and their own agent setups. + +## What it does + +1. **Segments your map** into roofs, trees and water. +2. **Builds a 2.5D block model** of the town (footprints from the map, heights proposed by wealth zone, landmarks + modeled by hand with named dimensions, people as true-scale mannequins). +3. **Ray-marches a template plate per camera**: an untextured render from exactly where the final picture will be taken. +4. **Hands the plate to an image-edit model** ("re-surface IMAGE 1; move nothing") with a prompt built from blocks that + measurably held, plus material references from another camera. +5. **Gates the result against the plate**: drift, zoom, landmark fidelity, invented skyline structures, figure scale, + text and watermarks. Numbers, not vibes. +6. **Derives the matrices** every table needs: Day/Sunset/Night x Dry/Rain/Snow, three snow depths with route + clearance classes, and a population x time-of-day matrix for every venue with people in it. +7. **Sizes the whole effort by tier**: how likely is this place to come into focus at the table? That decides the + deliverables, the budget, which agent model runs it, and what the GM is asked to look at. + +Why it works: words cannot carry an aspect ratio, a distance or a height, and a map given as a reference moves the +layout but not the shape. When the geometry arrives as pixels from the target camera, the renderer only has to paint +surface, and a correction is one edit to the model followed by a re-render of every camera. + +## Requirements + +- Python 3.10+, `pip install -r requirements.txt` (numpy, OpenCV, Pillow, requests). CPU only; a 2048x1152 plate takes + 1-15 minutes on a many-core machine. No GPU needed. +- An image-edit API that accepts several reference images and a prompt and returns a 16:9 image. The method was built + and measured on one hosted model (`gpt-image-2.5-flare`, OpenAI-compatible `/images/edits`); what made it work was + that it treats the first image as the scene to re-surface and holds its layout, framing and figure positions well + enough to gate against. Anything with that property will do; see `scripts/adapters/README.md` for the adapter point. +- A map export at a known scale (meters per pixel). Calibrate on something of known size. +- An agent runtime that can read `SKILL.md` (OpenClaw, Claude Code, or any harness that loads AgentSkills), or a + patient human following it by hand. + +## Quick start + +```bash +pip install -r requirements.txt +python scripts/smoke_test.py # synthetic town: segment -> mass -> plates -> jobs -> gates, no API calls + +cp examples/*.json my-town/ # citykit.json, views.json, scene.json, prompts.json; edit every value +export IMAGE_API_KEY=... # never in a file that goes into version control +cd scripts +python segment_map.py ../my-town/map.png --mpp 0.5 --out ../my-town/seg.npz --look ../my-town/look.jpg +python massing.py ../my-town/seg.npz --config ../my-town/citykit.json --out ../my-town/town.npz +python render_plate.py A_first_view C_gate_approach --config ../my-town/citykit.json +python build_jobs.py day --config ../my-town/citykit.json --prompts ../my-town/prompts.json --takes 3 --out ../my-town/jobs_day.json +python render_batch.py ../my-town/jobs_day.json --out ../my-town/out_day --start 4 --max 8 +python gate.py ../my-town/out_day --config ../my-town/citykit.json +python cost.py ../my-town/out_day/results.jsonl --config ../my-town/citykit.json +``` + +Then pick, relight (sunset/night as light-only edits of the day pick), build the matrices with `matrix_jobs.py`, and +deliver per `references/naming.md`. `SKILL.md` is the ordered procedure with a completion criterion per step. + +## A worked example (method only) + +A capital of about 29,000 people on a whole-city map at 0.53 m/px. The party rides in from the west and nobody had a +picture of the valley road, a gate, or the first view of the city; every earlier overview had invented the city from +words (river on the wrong side, a cathedral, a metropolis). + +- **Model:** 8,500 house pieces auto-massed from the bitmap in three wealth zones (turf-roofed outskirts, timber south, + stone-and-slate north); the castle traced from the map with proposed heights; the library at its canon dimensions; + the valley, the road beyond the map's edge and a timber toll gate all PROPOSED as named constants, because the map + draws no terrain and no gate. +- **Cameras:** a wide first view from a lookout knoll beside the road (the road itself ran in a cutting there), a long + lens and a telephoto from the same spot, the valley road at eye level, the gate approach at 52 m, through the gate at + 13 m. Travelers, carts and a pack horse placed on the road as proxies. +- **Re-surface:** three to four takes per view, ~120 calls, 0 refusals. +- **Gates:** wide views held ECC scale 1.003-1.014 after a framing lock (the first round zoomed in 4-10 %); landmark + correlation 0.44-0.72, one take rejected at 0.16 for redrawing the castle as a palace; figures +5-13 % after the + eye-line clause (first round +25-28 %); sunset relights within 0.15 % scale and 1.8 px of their day pick after the + "this is the eastern sky, no glow" clause (two rounds had painted a sunset in the east). +- **Failures fixed in the model, not the prompt:** a false lake from a blurred river class; the road in a trench; the + gate's roof as a solid block; the toll bench standing on the road; a sun angle that put the gate in its own shadow. +- **Delivered:** 14 frames (7 views x day + sunset) with alternates, QC sheets, a README with 12 PROPOSED items each + carrying "if you rule otherwise, change X and re-render Y", and a Cost line. + +## What to expect in cost + +Measured in October 2026 on the hosted model used: about $0.06 per call with one reference and $0.09 with four, at +2048x1152 high quality. A tier-1 place (full matrix, rooms, population cells) runs 100-200 calls, $10-20; a tier-2 shop +20-40 calls; a tier-3 pub under $1. Volume is what costs: 54-cell matrices times takes times re-rolls. Over three weeks +of heavy use one campaign spent about $800 on 11,000 calls, 37 % of it on weather matrices. Every run writes a Cost +line so you can see where yours goes. The honest way to cut it is fewer takes and tighter candidate sets, not a weaker +renderer: the retakes and the review time cost more than the unit price. + +The other cost is the GM's attention. The tier scheme exists to spend tokens and renders so that fewer, better frames +reach a human, and to put gated work straight into the campaign's canon folder with a one-line note instead of a +review pile. + +## Layout + +``` +SKILL.md the procedure, for agents (and humans) +references/ pipeline, gates, prompting, matrices, effort tiers, naming, failure modes +scripts/ the toolkit (see SKILL.md "Reference"); smoke_test.py proves it end to end +scripts/adapters/ renderer adapter point +examples/ config, cameras, scene, prompts to copy and edit +``` + +## Status and license + +The pipeline and gates are proven in use (hundreds of plates, thousands of renderer calls). The effort-tier scheme is +newly adopted and on trial; `references/effort-tiers.md` says what to evaluate. Contributions that add a renderer +adapter, a map-style segmentation preset, or measured gate bands for another model are welcome. + +License: see `LICENSE`. diff --git a/SKILL.md b/SKILL.md new file mode 100644 index 0000000..3585cdd --- /dev/null +++ b/SKILL.md @@ -0,0 +1,90 @@ +--- +name: dnd-city-visualization +description: "Render map-true towns, cities, approaches, establishments and their weather/population matrices for a tabletop campaign: 3D block model from the map, template plates per camera, re-surface with an image-edit model, gate, deliver by effort tier. Use when a GM asks for a view of a mapped place, a first view from the road, an aerial, a gate, a shopfront, a tavern at a given hour, or a weather matrix." +license: see LICENSE +--- + +# City render workflow + +The method: **geometry arrives as pixels, never as words.** Trace the map into a block model, ray-march a template +plate per camera, hand the plate to an image-edit model as IMAGE 1 with "re-surface this; move nothing", then gate the +frame against the plate. A correction is one edit to the model and a re-render of every camera. Details in +`references/pipeline.md`; run `scripts/smoke_test.py` once to see every tool work on a synthetic town. + +## Before you start + +- Read `references/effort-tiers.md` and assign the place a tier (= the chance it comes into focus at the table). The + tier fixes the deliverables, the budget and the matrix sizes. **Never exceed the tier's ceiling without the GM.** +- Copy `examples/citykit.example.json`, `views.example.json`, `scene.example.json`, `prompts.example.json` next to + the map and fill every value. Credentials only in the environment (`IMAGE_API_KEY`, `IMAGE_API_BASE`). +- Write down what is canon (on the map) and what you are about to PROPOSE (terrain, heights, walls, gates, the hour). +- Check the lane folder for a fresh REPORT.md: the same brief dispatched twice is re-verified, never re-rendered. +- **Done when:** config files exist, the tier and budget are in the brief, and REPORT.md exists with a Problem line. + +## Steps + +1. **Segment the map.** `segment_map.py MAP --mpp M --out seg.npz --look look.jpg`. Eyeball `look.jpg`. + **Done when:** roofs, trees and water are right where you will put a camera; false trees removed. +2. **Mass the town.** `massing.py seg.npz --config citykit.json --out town.npz`. Heights come from wealth zones + (proposed); hand-modeled landmarks are cut out and rebuilt in `scene.json` with every dimension named. + **Done when:** separate houses are separate cells, no slivers, landmarks have their canon dimensions. +3. **Place cameras and proxies.** Cameras in `views.json`: eye-level views at ground + 1.65 m with a lens shift + instead of pitch; the camera that *presents* the defining feature; found from the heightfield, never by eye. + Proxies (1.60-1.80 m, carts, animals) in the scene, 10 m or more from an eye-level camera, off the frame edge. + **Done when:** every view has a visibility list (what it sees, what is occluded) written for step 5. +4. **Render plates.** `render_plate.py V1 V2 --config citykit.json` (parallel processes; 1-15 min each at 2048). + Gate the plate against the map's checkable facts (`references/gates.md` section 0) and fix the model, not the + prompt. **Done when:** every plate passes its facts by eye and the untextured plate shows no artifact. +5. **Write the prompts.** Fill `prompts.json` per `references/prompting.md`: resurface premise, per-view intro / + keep / FRAMING LOCK with measured band positions, the people block, the eye-line clause for eye-level views, the + setting and town blocks (with the big things the model invents for a town of this kind, negated), the light with + the sun relative to the camera, no-modern and no-text. Material references from a different camera than the plate. + **Done when:** `build_jobs.py day --config citykit.json --prompts prompts.json --takes 3 --out jobs_day.json` + writes the jobs and you have read `jobs_day_PROMPTS.md` for contradictions. +6. **Re-surface.** `render_batch.py jobs_day.json --out out_day --start 4 --max 8`. Three to four takes per view. + **Done when:** `out_day/calibration.md` reports the run and no quota or refusal stopped it (a refusal stops the + lane; report it, do not switch renderers). +7. **Gate and pick.** `gate.py out_day --config citykit.json`, then `qc_sheet.py`. Reject reframes (ECC scale off + 1, shifts beyond a few px), low landmark correlation, any invented skyline structure on wide views, figures off + their brackets, anything in the corner sweep. Pick one plus an alternate per view into `picks.json`. + **Done when:** every pick has its gate numbers in REPORT.md and the plate-vs-pick sheet exists. +8. **Relight.** `build_jobs.py relight --light sunset V=out_day/V_day_t2.png ... --out jobs_sunset.json`, render, + gate with `--keepers`. Sunset, night and morning are light-only edits of the day pick; the sky in frame is named + (east or west). **Done when:** relights register to their day pick within 0.2 % scale and 2 px. +9. **Matrices (tier 1 and 2).** `matrix_jobs.py weather --tier T V=keeper ...` for Day/Sunset/Night x Dry/Rain/ + Snow (light) and any storm or deep-snow cells; `matrix_jobs.py population --tier T V=empty_master ...` for every + venue with people. Chained cells (Sunset/Night of a weather) get their reference after the Day cell is picked. Gate + storms on the wall band, deep snow above the snow line, route clearance by class (`references/matrices.md`). + **Done when:** the matrix is complete for the tier, each cell's gate rung is recorded, fixed cells are disclosed. +10. **Character pass (several exteriors on one wealth rung).** Thumbnail the batch; if the houses are + interchangeable, one-change edits adding each house's emblem sign and visible trade, gated against their own source. + **Done when:** the contact sheet reads as different places. +11. **Deliver.** Canon file names per `references/naming.md`; picks plus alternates; the QC sheets; a README for + the GM (what to show when, PROPOSED items with "if you rule otherwise", open questions, known flaws). Gated work + goes straight to canon with a one-line note; a human review entry only for a story-critical face, a permanent + bake, or on request. Verify by fetching back and hashing. **Done when:** REPORT.md has Problem, two Whys, what + was built, picks with numbers, failures and fixes, PROPOSED, open questions, limits, **Cost** (`cost.py`), and the + re-render chain; the hand-off is its path and one headline line. + +## Non-negotiables + +- Fix geometry in the model, never by wording. The renderer photographs the plate's mistakes as faithfully as its truths. +- Per camera, from a visibility list; occluded features disowned positively. +- Overviews and wide aerials are gated for invented large buildings against the model's landmark list. +- Figures are proxies in the plate, gated by brackets (+-25 %). +- Signs are emblems, never lettering; a corner and edge sweep on every frame. +- Never re-export the map to fix a fit; move a pin or reuse an existing building. +- Every run reports its Cost. Cut spend by volume, never by switching to a weaker renderer. +- Play promotes a place's tier; never demote; nothing rendered is deleted, superseded files are moved aside. + +## Reference + +- `references/pipeline.md` — the method step by step, with the measured reasons. +- `references/gates.md` — every gate, its bands, and what it is blind to. +- `references/prompting.md` — prompt blocks, framing locks, scale by relation, wealth-tier vocabulary, glass. +- `references/matrices.md` — the 9-square, snow levels and route clearance, foliage, population x time, matrix traps. +- `references/effort-tiers.md` — tiers, deliverables, budgets, model-cost tiering, delivery policy (on trial). +- `references/naming.md` — file and folder names, the report shape. +- `references/failure-modes.md` — symptom, cause, fix, one line each. +- `scripts/` — `segment_map.py`, `massing.py`, `scene_solids.py`, `render_plate.py`, `build_jobs.py`, + `render_batch.py` (+ `adapters/`), `gate.py`, `qc_sheet.py`, `matrix_jobs.py`, `cost.py`, `smoke_test.py`.