146 lines
11 KiB
Markdown
146 lines
11 KiB
Markdown
# The pipeline in detail
|
|
|
|
The problem this solves: a renderer given a prompt plus the map as a reference puts the right *things* in the frame
|
|
and still invents the town: a square drawn as a broad plaza when the map draws it long and narrow, a river on the
|
|
wrong side, a metropolis with a cathedral for a town of twenty thousand. **Why 1:** a reference image moves the layout
|
|
but the proportions come from the model's idea of "a town square", and words cannot carry an aspect ratio. **Why 2:**
|
|
"camera from the south" asks the model to invent a 3D scene from a 2D plan, and every renderer fills that gap with
|
|
its prior. So geometry has to arrive as pixels, from the target camera. Splitting geometry (a block model, ray-marched)
|
|
from surface (a re-surfacing edit) removes the trade-off, and a correction becomes one edit to the model plus a
|
|
re-render of every camera. Across angles, every frame shows the same building by construction.
|
|
|
|
## 0. Inputs (write all of them down before step 1)
|
|
|
|
1. **The map bitmap** at native export resolution, and its **scale** in meters per pixel, calibrated on something of
|
|
known size (a canon building, a bridge span, a template-matched patch of a sheet whose scale you already know).
|
|
Two sheets of one town at different scales: template-match five footprint patches between them (NCC 0.92-0.96 in
|
|
practice) and resample one to the other, so one segmentation recipe applies to both.
|
|
2. **Never re-export the map to fix a fit.** Map tools rarely reproduce the exact position and zoom, and a re-export
|
|
that differs ever so slightly moves every pin and breaks every model registered on the sheet. Move a pin to a
|
|
better-fitting building that already exists, or paste a small patch bitmap onto the existing sheet and pixel-diff
|
|
the result so the change stays inside the patch box.
|
|
3. **The subject's extent** in map px. For a square, street or yard the extent is the open ground, never the open
|
|
ground plus its frontages.
|
|
4. **Ridge directions** of the buildings that matter, read from a drawn ridge highlight at 12x, else from the shingle
|
|
course direction at 10x, never from proportion or shading.
|
|
5. **The checkable facts** the frame must show (five or six: elongation, a closing range, a saw-tooth of gables, the
|
|
ways in, the tracks, open ground clear of buildings). The plate gate scores these.
|
|
6. **Target cameras.** Pick the camera that *presents* the defining feature: a saw-tooth of gable ends reads from
|
|
across the square, not along it. Find streets from an occupancy top-down of the heightfield, never by eye on the map
|
|
(a first approach camera landed inside a house row).
|
|
7. **What is canon and what you are about to PROPOSE.** Terrain is almost never on a map; walls and gates may not be
|
|
drawn; heights are never drawn. Everything not on the map is a proposal and lives as a named constant.
|
|
|
|
## 0.5. District sheet (optional, before rendering establishments)
|
|
|
|
For a large city, register each district bitmap to the whole-city map before choosing render subjects. Template-match
|
|
several distinctive roof patches, fit one scale-and-translation transform, record the residuals, and project the
|
|
whole-city pins through it. Keep every pin on its real drawn building. If a pin falls outside the template's placeable
|
|
window or under its veil, omit it or make another sheet; never move the pin to improve the layout.
|
|
|
|
Populate the district template with the established pins first. Then use empty slots for living-city background:
|
|
proposed establishments with a name, one-line tagline, wealth tier, and an actual unoccupied roof. Mark every invented
|
|
field **PROPOSED** in both the sheet and its companion table. Route leaders without crossings, foreign-icon hits, or
|
|
legend overlap, and verify label-to-pin identity after any slot permutation.
|
|
|
|
Assign render effort separately from wealth. Most padded places are T3 or T4: T3 may get one exterior when it becomes
|
|
relevant; T4 remains a pin and a short fact until play promotes it. Established or likely destinations may be T1/T2.
|
|
See `docs/walkthrough-district-map.md` for the illustrated registration → pins → boundary → populated-sheet chain.
|
|
|
|
**Done when:** registration residuals are recorded; established and proposed rows are distinguishable; every pin is
|
|
on a real map building; label routes pass; and each row has both wealth and effort tiers.
|
|
|
|
## 1. Segment (`segment_map.py`)
|
|
|
|
Roof, tree and water masks. Trees are the cauliflower glyphs (dense short dark strokes in compact blobs); roofs are
|
|
darker than the parchment and not a tree, with the cast-shadow halo stripped by a 1 px erosion; water is the teal.
|
|
**Eyeball the check image.** Known misses: two "trees" on a square that were building-edge detail; a big tree glyph at
|
|
a gable read as roof; a sheet whose tree glyphs cannot be separated from roofs within ~50 m of the subject (replace
|
|
that patch with houses fitted to the roof mask minus hand-placed tree glyphs).
|
|
|
|
## 2. Mass (`massing.py`)
|
|
|
|
Footprints straightened (polygon approximation at 1 px), slivers opened away with a 2 m disc, pieces under 40 m2
|
|
dropped (slivers become ruins when re-surfaced), merged blocks split into 8-15 m houses with a 1-cell gap (separate
|
|
houses must be separate in pixels or the renderer re-slices them into an 11-gable "caterpillar"), a gable over each
|
|
fitted rectangle (fill > 0.70, ridge on the long axis) else a capped hip from the distance transform, trees as lumpy
|
|
domes, roof components over water as bridge decks. **Heights are proposed by wealth zone** (the map gives footprints,
|
|
not heights). Landmarks you model by hand are cut out of the auto-massing. If a frozen earlier tool has a known
|
|
offset bug in its roof heights, fix it in your copy, not by prompt.
|
|
|
|
A landmark with canon dimensions goes into `scene.json` as analytic solids: prisms traced from the map, boxes, cones,
|
|
gables. Watch for a convex split of an L-shaped wing that puts battlements on the internal seam ("a wall across the
|
|
roof"); check the untextured plate for it.
|
|
|
|
## 3. Terrain and roads (optional, all PROPOSED)
|
|
|
|
A plateau = the town's drawn extent closed and grown by a margin; a valley floor along the river; walls rising with
|
|
distance from the floor (a glacial trough profile: gentle toe, faces steepening to ~30 degrees); ridged noise. Roads
|
|
benched in afterwards, at a grade cap (10 %), on a 1 m layer that cuts a broad shelf (no trench). Model where a built
|
|
feature ENDS: if a quay, wall or paving just stops, give the far side its own visibly different geometry, or the renderer
|
|
continues the built feature to the frame edge. When shaping terrain to match a painting's skyline, scale the massif
|
|
about the painter's eye, never shape terrain by sight lines. The renderer draws its own switchbacks, so gate roads
|
|
tolerantly.
|
|
|
|
## 4. Proxies (`scene.json` people, carts, horses)
|
|
|
|
True-scale mannequins (1.60-1.80 m, varied), carts, pack horses, placed where the story needs them. They are the scale
|
|
anchors: the renderer dresses each where it stands, so the proxies BECOME the population and the headcount per cell is
|
|
how many proxies that cell's plate carries. Keep proxies 10 m or more from an eye-level camera and off the frame edge (a
|
|
mannequin cut by the edge becomes a stone post with a ball finial, 4/4). Never a proxy mostly hidden behind another
|
|
without its occluder.
|
|
|
|
## 5. Plates (`render_plate.py`)
|
|
|
|
One pinhole camera per view with an optional vertical lens shift (render level and shift, never pitch up: a pitched-up
|
|
plate tilted the verticals 1-2.6 degrees in every take). A 2048x1152 plate takes 1-15 minutes on 32 cores; run views
|
|
in parallel. The plate carries a window/door rhythm on the walls (the renderer builds real shuttered houses from it)
|
|
and sun shadows from the hour you will name in the prompt. Outputs: the plate, depth, a solid-id buffer, a class map,
|
|
the camera, the projected people table, the projected landmarks.
|
|
|
|
**Plate gate before any renderer call** (references/gates.md section 0).
|
|
|
|
## 6. Re-surface (`build_jobs.py`, `render_batch.py`)
|
|
|
|
IMAGE 1 = the plate; IMAGE 2.. = material and light references from a *different* camera (a register of the town's
|
|
materials, a canon landmark photo), each with a stated purpose and an "ignore its camera and composition" clause. The
|
|
prompt blocks are in references/prompting.md. Three to four takes per view, for choice not control (the model has no
|
|
seed and no strength dial; wording barely moves geometry). Concurrency 4-8 in flight was smooth; 16-20 with several
|
|
4 MB references hit upload timeouts. Measured cost per call: about $0.06 with one reference and $0.09 with four, at
|
|
2048x1152 high quality on the hosted model used.
|
|
|
|
Day frames are fresh re-surfaces. **Sunset, night and morning are relights** (light-only edits) of the picked day
|
|
frame, which hold geometry to within 0.15 % scale and 2 px.
|
|
|
|
## 7. Gate (`gate.py`, `qc_sheet.py`)
|
|
|
|
references/gates.md. In one line: drift and ECC against the plate; landmark-region correlation; skyline fit and the
|
|
invented-structure check on wide views; figure brackets; the corner sweep; then the eye on the contact sheet for what
|
|
instruments cannot see (occlusion, water, extra people, signs).
|
|
|
|
## 8. Matrices (`matrix_jobs.py`)
|
|
|
|
references/matrices.md: the time x weather 9-square per camera, snow levels and route clearance, and the population x
|
|
time matrix for every venue with people in it, sized by tier.
|
|
|
|
## 9. Interiors
|
|
|
|
The same method indoors: a convex-solid room model (box, oriented box, gable prism, walls with openings, mannequins),
|
|
a plate per camera with a color key (orange = fire, pale warm = sunlit opening, pale blue = sky opening, black = dark
|
|
doorway), IMAGE 1 = "the set, not a style reference; same tripod". 51/51 held exits, hearth wall, counter and stair.
|
|
The only failure class: windows invented in an internal partition the prompt could read as an exterior wall. Multiple
|
|
angles of one room agree by construction, and the same model takes proxies for scenes. The empty master is the keeper
|
|
that every population cell and scene descends from.
|
|
|
|
## 10. Deliver
|
|
|
|
Picks and alternates, the QC sheets, a README for the GM (what to show when, the PROPOSED items with "if you rule
|
|
otherwise", the open questions, the known flaws), the REPORT with its Cost line. Verify delivery by fetching back
|
|
and hashing. Promote gated work straight to canon; a human review entry is the expensive unit (references/effort-tiers.md).
|
|
|
|
## How to re-render after a ruling
|
|
|
|
Edit the named constant (in `scene.json`, the zones, the terrain, the camera) -> re-render the affected plates ->
|
|
re-run the day jobs for those views -> gate -> relight from the new picks -> re-run the matrix cells that descend from
|
|
them. Because every downstream frame descends from a plate, a ruling is one edit and a known chain, never a redraw.
|