Files

11 KiB

The pipeline in detail

The problem this solves: a renderer given a prompt plus the map as a reference puts the right things in the frame and still invents the town: a square drawn as a broad plaza when the map draws it long and narrow, a river on the wrong side, a metropolis with a cathedral for a town of twenty thousand. Why 1: a reference image moves the layout but the proportions come from the model's idea of "a town square", and words cannot carry an aspect ratio. Why 2: "camera from the south" asks the model to invent a 3D scene from a 2D plan, and every renderer fills that gap with its prior. So geometry has to arrive as pixels, from the target camera. Splitting geometry (a block model, ray-marched) from surface (a re-surfacing edit) removes the trade-off, and a correction becomes one edit to the model plus a re-render of every camera. Across angles, every frame shows the same building by construction.

0. Inputs (write all of them down before step 1)

  1. The map bitmap at native export resolution, and its scale in meters per pixel, calibrated on something of known size (a canon building, a bridge span, a template-matched patch of a sheet whose scale you already know). Two sheets of one town at different scales: template-match five footprint patches between them (NCC 0.92-0.96 in practice) and resample one to the other, so one segmentation recipe applies to both.
  2. Never re-export the map to fix a fit. Map tools rarely reproduce the exact position and zoom, and a re-export that differs ever so slightly moves every pin and breaks every model registered on the sheet. Move a pin to a better-fitting building that already exists, or paste a small patch bitmap onto the existing sheet and pixel-diff the result so the change stays inside the patch box.
  3. The subject's extent in map px. For a square, street or yard the extent is the open ground, never the open ground plus its frontages.
  4. Ridge directions of the buildings that matter, read from a drawn ridge highlight at 12x, else from the shingle course direction at 10x, never from proportion or shading.
  5. The checkable facts the frame must show (five or six: elongation, a closing range, a saw-tooth of gables, the ways in, the tracks, open ground clear of buildings). The plate gate scores these.
  6. Target cameras. Pick the camera that presents the defining feature: a saw-tooth of gable ends reads from across the square, not along it. Find streets from an occupancy top-down of the heightfield, never by eye on the map (a first approach camera landed inside a house row).
  7. What is canon and what you are about to PROPOSE. Terrain is almost never on a map; walls and gates may not be drawn; heights are never drawn. Everything not on the map is a proposal and lives as a named constant.

0.5. District sheet (optional, before rendering establishments)

For a large city, register each district bitmap to the whole-city map before choosing render subjects. Template-match several distinctive roof patches, fit one scale-and-translation transform, record the residuals, and project the whole-city pins through it. Keep every pin on its real drawn building. If a pin falls outside the template's placeable window or under its veil, omit it or make another sheet; never move the pin to improve the layout.

Populate the district template with the established pins first. Then use empty slots for living-city background: proposed establishments with a name, one-line tagline, wealth tier, and an actual unoccupied roof. Mark every invented field PROPOSED in both the sheet and its companion table. Route leaders without crossings, foreign-icon hits, or legend overlap, and verify label-to-pin identity after any slot permutation.

Assign render effort separately from wealth. Most padded places are T3 or T4: T3 may get one exterior when it becomes relevant; T4 remains a pin and a short fact until play promotes it. Established or likely destinations may be T1/T2. See docs/walkthrough-district-map.md for the illustrated registration → pins → boundary → populated-sheet chain.

Done when: registration residuals are recorded; established and proposed rows are distinguishable; every pin is on a real map building; label routes pass; and each row has both wealth and effort tiers.

1. Segment (segment_map.py)

Roof, tree and water masks. Trees are the cauliflower glyphs (dense short dark strokes in compact blobs); roofs are darker than the parchment and not a tree, with the cast-shadow halo stripped by a 1 px erosion; water is the teal. Eyeball the check image. Known misses: two "trees" on a square that were building-edge detail; a big tree glyph at a gable read as roof; a sheet whose tree glyphs cannot be separated from roofs within ~50 m of the subject (replace that patch with houses fitted to the roof mask minus hand-placed tree glyphs).

2. Mass (massing.py)

Footprints straightened (polygon approximation at 1 px), slivers opened away with a 2 m disc, pieces under 40 m2 dropped (slivers become ruins when re-surfaced), merged blocks split into 8-15 m houses with a 1-cell gap (separate houses must be separate in pixels or the renderer re-slices them into an 11-gable "caterpillar"), a gable over each fitted rectangle (fill > 0.70, ridge on the long axis) else a capped hip from the distance transform, trees as lumpy domes, roof components over water as bridge decks. Heights are proposed by wealth zone (the map gives footprints, not heights). Landmarks you model by hand are cut out of the auto-massing. If a frozen earlier tool has a known offset bug in its roof heights, fix it in your copy, not by prompt.

A landmark with canon dimensions goes into scene.json as analytic solids: prisms traced from the map, boxes, cones, gables. Watch for a convex split of an L-shaped wing that puts battlements on the internal seam ("a wall across the roof"); check the untextured plate for it.

3. Terrain and roads (optional, all PROPOSED)

A plateau = the town's drawn extent closed and grown by a margin; a valley floor along the river; walls rising with distance from the floor (a glacial trough profile: gentle toe, faces steepening to ~30 degrees); ridged noise. Roads benched in afterwards, at a grade cap (10 %), on a 1 m layer that cuts a broad shelf (no trench). Model where a built feature ENDS: if a quay, wall or paving just stops, give the far side its own visibly different geometry, or the renderer continues the built feature to the frame edge. When shaping terrain to match a painting's skyline, scale the massif about the painter's eye, never shape terrain by sight lines. The renderer draws its own switchbacks, so gate roads tolerantly.

4. Proxies (scene.json people, carts, horses)

True-scale mannequins (1.60-1.80 m, varied), carts, pack horses, placed where the story needs them. They are the scale anchors: the renderer dresses each where it stands, so the proxies BECOME the population and the headcount per cell is how many proxies that cell's plate carries. Keep proxies 10 m or more from an eye-level camera and off the frame edge (a mannequin cut by the edge becomes a stone post with a ball finial, 4/4). Never a proxy mostly hidden behind another without its occluder.

5. Plates (render_plate.py)

One pinhole camera per view with an optional vertical lens shift (render level and shift, never pitch up: a pitched-up plate tilted the verticals 1-2.6 degrees in every take). A 2048x1152 plate takes 1-15 minutes on 32 cores; run views in parallel. The plate carries a window/door rhythm on the walls (the renderer builds real shuttered houses from it) and sun shadows from the hour you will name in the prompt. Outputs: the plate, depth, a solid-id buffer, a class map, the camera, the projected people table, the projected landmarks.

Plate gate before any renderer call (references/gates.md section 0).

6. Re-surface (build_jobs.py, render_batch.py)

IMAGE 1 = the plate; IMAGE 2.. = material and light references from a different camera (a register of the town's materials, a canon landmark photo), each with a stated purpose and an "ignore its camera and composition" clause. The prompt blocks are in references/prompting.md. Three to four takes per view, for choice not control (the model has no seed and no strength dial; wording barely moves geometry). Concurrency 4-8 in flight was smooth; 16-20 with several 4 MB references hit upload timeouts. Measured cost per call: about $0.06 with one reference and $0.09 with four, at 2048x1152 high quality on the hosted model used.

Day frames are fresh re-surfaces. Sunset, night and morning are relights (light-only edits) of the picked day frame, which hold geometry to within 0.15 % scale and 2 px.

7. Gate (gate.py, qc_sheet.py)

references/gates.md. In one line: drift and ECC against the plate; landmark-region correlation; skyline fit and the invented-structure check on wide views; figure brackets; the corner sweep; then the eye on the contact sheet for what instruments cannot see (occlusion, water, extra people, signs).

8. Matrices (matrix_jobs.py)

references/matrices.md: the time x weather 9-square per camera, snow levels and route clearance, and the population x time matrix for every venue with people in it, sized by tier.

9. Interiors

The same method indoors: a convex-solid room model (box, oriented box, gable prism, walls with openings, mannequins), a plate per camera with a color key (orange = fire, pale warm = sunlit opening, pale blue = sky opening, black = dark doorway), IMAGE 1 = "the set, not a style reference; same tripod". 51/51 held exits, hearth wall, counter and stair. The only failure class: windows invented in an internal partition the prompt could read as an exterior wall. Multiple angles of one room agree by construction, and the same model takes proxies for scenes. The empty master is the keeper that every population cell and scene descends from.

10. Deliver

Picks and alternates, the QC sheets, a README for the GM (what to show when, the PROPOSED items with "if you rule otherwise", the open questions, the known flaws), the REPORT with its Cost line. Verify delivery by fetching back and hashing. Promote gated work straight to canon; a human review entry is the expensive unit (references/effort-tiers.md).

How to re-render after a ruling

Edit the named constant (in scene.json, the zones, the terrain, the camera) -> re-render the affected plates -> re-run the day jobs for those views -> gate -> relight from the new picks -> re-run the matrix cells that descend from them. Because every downstream frame descends from a plate, a ruling is one edit and a known chain, never a redraw.