skill: codify live Wealthy packaging lessons and harden canonicalization
BetterLeaks / scan (push) Successful in 57s

Document 2026-03-28 live-run findings (sync gates, TransitionScene folding, wildcard collision handling, DAV behavior, canonical runhex form) in SKILL docs. Harden finalize/preflight to normalize legacy single-underscore runhex names, support stage-root reruns, and match single-stage files with scene-tag/legacy suffix variations so valid Magnus/Torsten-style assets are packaged automatically.
This commit is contained in:
Dolon (Agent)
2026-03-28 15:26:58 +00:00
parent e1120039bb
commit 34ea251aaf
4 changed files with 339 additions and 43 deletions
+8
View File
@@ -82,6 +82,14 @@ Prerequisite: The human user has selected favorites across all dress stages and
**Preflight requirement (packaging):** before packaging, **fail fast** if any non-Arrival stage files still include stage letters (e.g., `_A__`, `_B__`, `_A_E__`, or `TeasingMoreC__`). These must be stripped before packaging.
**Live packaging hardening lessons (2026-03-28, Wealthy run):**
- Wait for full file sync before preflight/canonicalization. Half-synced folders can cause `ENOENT` during two-phase renames and leave `.tmpnorm/.tmpcanon` residue.
- Keep canonical filename form `...__<runhex>.jpg`. Legacy single-underscore forms (`..._(2)_abc123.jpg`) are now treated as ingest-only and normalized to canonical `__`.
- Treat single `TransitionScene` files in **Topless Plus** as the same sequence family as `Name_Topless[...]`; normalize into the same numbered chain before packaging.
- Merge scene-suffix variants under one wildcard base before finalize (e.g., `_reclined`, `_intertwined`, `_faredge_relaxed`) to avoid duplicate/mixed-index collisions.
- Single-stage matching must tolerate scene tags and legacy tails (e.g., `_marblebath_Teasing__...`, `_Dressed_v2_E__...`) plus short-name Topless fallbacks.
- Run finalize in background and send progress pings; on DAV/WebDAV mounts, expect churny sync behavior and slower apparent writes.
## References
- **references/workflow.md** - the full process notes, prompt clauses, failure modes, yields, and timing.
+9 -1
View File
@@ -246,10 +246,18 @@ Run **A-G** variants and **try each single up to 7 moderation rejects** before f
**Performance note:** edits are slower than pure generations; increase concurrency if needed. Default for `grok-imagine-image` (standard): **25 workers**. `grok-imagine-image-pro` is limited (~30 images/min), so keep concurrency lower as needed.
**ArrivalAllSingles packaging (tier root):** place the overview images **directly in the tier folder**, named `Arrival (Lobby type A/B/C).jpg` (and `Arrival (Lobby type X, with stairs).jpg` as applicable). Also create **empty order markers** in the tier root to show lineup order: `01-Name1.txt`, `02-Name2.txt`, etc., matching the order used in the ArrivalAllSingles prompt (default: alphabetic singles, include only one twin unless configured otherwise).
**ArrivalAllSingles packaging (tier root):** place the overview images **directly in the tier folder**, named `Arrival (Lobby type A/B/C).jpg` (and `Arrival (Lobby type X, with stairs).jpg` as applicable). Do **not** create `01-Name.txt` marker files; use passport thumbnails instead.
**Post-packaging cleanup:** create a folder named **"Stages and Raw Files"** under the tier, and move **all folders at the tier root except "Per Courtesan"** into it (Arrival/Dressed/Teasing/Teasing More/Topless/Topless Plus/Underwear/Undressed/etc.) to reduce DM-seat noise.
**Live run hardening notes (2026-03-28):**
- Confirm sync completion before preflight/canonicalization; partial sync causes rename `ENOENT` and temp artifacts.
- Normalize legacy single-underscore runhex names to canonical `__` before packaging.
- In **Topless Plus**, merge single `TransitionScene` files into `Name_Topless[n]` sequence before finalize.
- Collapse scene-suffix variants (`_reclined`, `_intertwined`, etc.) under one base sequence to prevent wildcard collisions.
- For singles, match stage files robustly: allow scene tags before stage names and legacy tails after stage names.
- Run long finalize jobs in background and send progress pings; DAV/WebDAV mounts can look "stuck" due to sync churn.
## 8) Scripts (current operational set)
- `scripts/rerender_courtesans_doublepairs_fullynaked_smile.py`
+297 -15
View File
@@ -16,8 +16,18 @@ import argparse
import re
import shutil
import sys
from collections import defaultdict
BRACKET_RE=re.compile(r'[\[\{\(\<]\s*(\d{1,2})\s*[\]\}\)\>]')
MARKER_SUFFIX_RE=re.compile(r'^(?P<base>.*?)(?P<sep>[ _-]*)[\[\{\(\<]\s*(?P<token>[^\]\}\)\>]+?)\s*[\]\}\)\>]$')
# Legacy filename support: one underscore before runhex, e.g. ..._(0b)_7857649.jpg
# Canonical form remains ...__7857649.jpg and this regex is only for ingest/normalization.
LEGACY_SINGLESEP_RUNHEX_RE=re.compile(r'^(?P<prefix>.+?)(?<!_)_(?P<runhex>[0-9a-fA-F]{6,16})\.jpg$')
NUM_TOKEN_RE=re.compile(r'^\d{1,3}$')
NUM_SUFFIX_TOKEN_RE=re.compile(r'^(\d{1,3})([a-z]+)$', re.IGNORECASE)
LAST_TOKEN_RE=re.compile(r'^last$', re.IGNORECASE)
LAST_NUM_TOKEN_RE=re.compile(r'^last-(\d+)$', re.IGNORECASE)
LAST_N_TOKEN_RE=re.compile(r'^last-n$', re.IGNORECASE)
def pick_latest(files):
@@ -46,7 +56,10 @@ def bracketed_variants(stage_dir: Path, base_pattern: str):
m=BRACKET_RE.search(f.name)
if not m:
continue
out[int(m.group(1))]=f
idx=int(m.group(1))
if idx in out:
raise RuntimeError(f"Conflict: duplicate variant index [{idx}] in {stage_dir.name} for pattern {base_pattern}")
out[idx]=f
return out
@@ -57,6 +70,209 @@ def has_unbracketed(stage_dir: Path, base_pattern: str):
return False
def _suffix_rank(s: str):
if not s:
return 0
v=0
for ch in s.lower():
if not ('a' <= ch <= 'z'):
return None
v=v*27 + (ord(ch)-96)
return v
def _parse_variant_token(token: str):
t=(token or '').strip().lower().replace(' ', '')
if NUM_TOKEN_RE.fullmatch(t):
return ('num', int(t), None)
m=NUM_SUFFIX_TOKEN_RE.fullmatch(t)
if m:
return ('numsuffix', int(m.group(1)), m.group(2).lower())
if LAST_TOKEN_RE.fullmatch(t):
return ('last', 0, None)
m=LAST_NUM_TOKEN_RE.fullmatch(t)
if m:
return ('last', int(m.group(1)), None)
if LAST_N_TOKEN_RE.fullmatch(t):
return ('lastn', None, None)
return ('unknown', t, None)
def _manual_order_prompt(stage_name: str, base: str, entries, reason: str):
if not sys.stdin.isatty():
sample='; '.join(e['path'].name for e in entries[:6])
raise RuntimeError(
f"Preflight ambiguous in {stage_name} for '{base}': {reason}. "
f"Run interactively and choose order, or rename manually. Files: {sample}"
)
print(f"\nAMBIGUOUS ORDER in {stage_name}: {base}")
print(f"Reason: {reason}")
for i, e in enumerate(entries, start=1):
tok=e.get('token') or 'unmarked'
print(f" {i}) {e['path'].name} marker={tok}")
print("Enter desired order as item numbers, earliest -> latest (e.g. 2,1,3).")
while True:
raw=input('order> ').strip()
if not raw:
raise RuntimeError(f"Aborted by user while resolving {stage_name}/{base}")
parts=[p.strip() for p in raw.split(',') if p.strip()]
try:
chosen=[int(p) for p in parts]
except ValueError:
print('Please enter only numbers separated by commas.')
continue
if len(chosen) != len(entries):
print(f"Need exactly {len(entries)} numbers.")
continue
if sorted(chosen) != list(range(1, len(entries)+1)):
print('Use each item number exactly once.')
continue
return [entries[i-1] for i in chosen]
def _compute_order(stage_name: str, base: str, entries):
parsed=[]
for e in entries:
kind,a,b=_parse_variant_token(e['token'])
parsed.append((e,kind,a,b))
unknown=[e for (e,kind,_,_) in parsed if kind=='unknown']
if unknown:
return _manual_order_prompt(stage_name, base, entries, f"unrecognized marker(s): {', '.join(sorted({u['token'] for u in unknown}))}")
non_last=[]
last=[]
lastn=[]
for e,kind,a,b in parsed:
if kind=='num':
non_last.append((a,0,e))
elif kind=='numsuffix':
sr=_suffix_rank(b or '')
if sr is None:
return _manual_order_prompt(stage_name, base, entries, f"invalid numeric suffix marker: {e['token']}")
non_last.append((a,sr,e))
elif kind=='last':
last.append((a,e))
elif kind=='lastn':
lastn.append(e)
# Detect duplicate numeric/provisional slots (ambiguous intent)
seen_slots=defaultdict(list)
for n,s,e in non_last:
seen_slots[(n,s)].append(e)
dup_slots=[k for k,v in seen_slots.items() if len(v)>1]
if dup_slots:
return _manual_order_prompt(stage_name, base, entries, f"duplicate provisional slots: {dup_slots}")
seen_last=defaultdict(list)
for d,e in last:
seen_last[d].append(e)
dup_last=[k for k,v in seen_last.items() if len(v)>1]
if dup_last:
return _manual_order_prompt(stage_name, base, entries, f"duplicate last-markers: {dup_last}")
if len(lastn) > 1:
return _manual_order_prompt(stage_name, base, entries, "multiple 'last-n' placeholders")
# Infer one 'last-n' as earliest among last-group.
if len(lastn) == 1:
inferred=(max([d for d,_ in last], default=0) + 1)
last.append((inferred, lastn[0]))
ordered=[]
if non_last:
ordered.extend([e for _,_,e in sorted(non_last, key=lambda t: (t[0], t[1]))])
if last:
# last-2, last-1, last (distance descending)
ordered.extend([e for _,e in sorted(last, key=lambda t: t[0], reverse=True)])
if not ordered:
return entries
if len(ordered) != len(entries):
return _manual_order_prompt(stage_name, base, entries, "could not establish full ordering")
return ordered
def normalize_provisional_markers(stage_dir: Path):
"""Normalize provisional markers like (3b)/(3c)/(last-2)/(last) to canonical [1]..[n]."""
if not stage_dir.exists():
return []
by_base=defaultdict(list)
for f in stage_dir.glob('*.jpg'):
name=f.name
if '__' in name:
prefix, suffix = name.split('__', 1)
else:
# Legacy fallback: single underscore before runhex, normalize to canonical __ on write.
m_legacy=LEGACY_SINGLESEP_RUNHEX_RE.match(name)
if not m_legacy:
continue
prefix=m_legacy.group('prefix')
suffix=f"{m_legacy.group('runhex')}.jpg"
m=MARKER_SUFFIX_RE.match(prefix)
if m:
base=m.group('base')
sep=m.group('sep') or ''
token=(m.group('token') or '').strip()
marked=True
else:
base=prefix
sep=''
token=None
marked=False
by_base[base].append({
'path': f,
'base': base,
'sep': sep,
'token': token,
'suffix': suffix,
'marked': marked,
})
renames=[]
for base, entries in sorted(by_base.items()):
marked=[e for e in entries if e['marked']]
if not marked:
continue
if len(marked) != len(entries):
raise RuntimeError(
f"Preflight failed in {stage_dir.name}: mixed marked/unmarked variants for '{base}'. "
"Either mark all variants or keep exactly one unmarked file."
)
ordered=_compute_order(stage_dir.name, base, marked)
for idx, e in enumerate(ordered, start=1):
new_prefix=f"{e['base']}{e['sep']}[{idx}]"
new_name=f"{new_prefix}__{e['suffix']}"
if new_name != e['path'].name:
renames.append((e['path'], e['path'].with_name(new_name)))
# two-phase rename to avoid collisions
if renames:
tmp_moves=[]
for i, (src, dst) in enumerate(renames, start=1):
tmp=src.with_name(src.name + f".tmpnorm{i}")
src.rename(tmp)
tmp_moves.append((tmp, dst))
for tmp, dst in tmp_moves:
if dst.exists():
raise RuntimeError(f"Cannot normalize markers; target already exists: {dst.name}")
tmp.rename(dst)
return renames
def copy_stage_with_variants(stage_dir: Path, base_pattern: str, dst_folder: Path, label: str):
variants=bracketed_variants(stage_dir, base_pattern)
unbracketed=has_unbracketed(stage_dir, base_pattern)
@@ -113,20 +329,33 @@ def extract_single_name(base: str):
# Pattern: Name_Name_Single_Courtesan_Tier or Name_Name_Courtesan_Tier
# We want just: Name_Name
parts = base.split('_')
# Find where the suffix starts (Single, Courtesan, or tier names)
suffix_markers = ['Single', 'Courtesan', 'Comfortable', 'Modest', 'Wealthy', 'Elite', 'Poor']
name_parts = []
for part in parts:
if part in suffix_markers:
break
name_parts.append(part)
return '_'.join(name_parts) if name_parts else base
def extract_single_base_from_dressed_filename(filename: str):
"""Return the single base from a dressed filename, tolerant of legacy suffixes.
Examples:
- Helga_Sunspire_Dressed__abcd.jpg -> Helga_Sunspire
- Magnus_Ironbrook_Single_Courtesan_Wealthy_Dressed_v2_E__abcd.jpg -> Magnus_Ironbrook_Single_Courtesan_Wealthy
"""
prefix = filename.split('__', 1)[0]
# Strip trailing _Dressed plus optional legacy tail segments (_v2_E, etc.).
return re.sub(r'_Dressed(?:_[^_]+)*$', '', prefix)
def copy_arrival(arrival_dir: Path, base_pattern: str, dst_folder: Path, is_single: bool):
copied=False
for L in ['A','B','C']:
pat = f"{base_pattern}_Arrival{L}" if is_single else f"{base_pattern}_{L}"
files = list(arrival_dir.glob(f"{pat}*__*.jpg"))
@@ -137,34 +366,81 @@ def copy_arrival(arrival_dir: Path, base_pattern: str, dst_folder: Path, is_sing
if stairs:
f=pick_latest(stairs)
copy_to(f, dst_folder/f"Arrival (Lobby type {L}, with stairs).jpg")
copied=True
if nonstairs:
f=pick_latest(nonstairs)
copy_to(f, dst_folder/f"Arrival (Lobby type {L}).jpg")
copied=True
return copied
def _pick_latest_globs(stage_dir: Path, patterns):
files=[]
for pat in patterns:
files.extend(stage_dir.glob(pat))
if not files:
return None
uniq={str(p): p for p in files}
return pick_latest(list(uniq.values()))
def copy_stages_single(base: str, dirs, dst_folder: Path, final_stage: str):
dressed_dir, teasing_dir, teasing_more_dir, topless_dir, topless_plus_dir, underwear_l_dir, underwear_s_dir, undressed_dir, arrival_dir = dirs
f=pick_latest(list(dressed_dir.glob(f"{base}_Dressed__*.jpg")))
name_only=extract_single_name(base)
# Allow suffixes after stage names (e.g., _Dressed_v2_E) so legacy keeper names still resolve.
f=_pick_latest_globs(dressed_dir, [
f"{base}_Dressed*__*.jpg",
f"{name_only}_Dressed*__*.jpg",
])
if f: copy_to(f, dst_folder/'Dressed.jpg')
f=pick_latest(list(teasing_dir.glob(f"{base}_Teasing__*.jpg")))
# Some singles include scene tokens before stage names (e.g., _marblebath_Teasing__...)
f=_pick_latest_globs(teasing_dir, [
f"{base}_Teasing*__*.jpg",
f"{base}_*Teasing__*.jpg",
f"{name_only}_Teasing*__*.jpg",
f"{name_only}_*Teasing__*.jpg",
])
if f: copy_to(f, dst_folder/'Teasing.jpg')
f=pick_latest(list(teasing_more_dir.glob(f"{base}_TeasingMore__*.jpg")))
f=_pick_latest_globs(teasing_more_dir, [
f"{base}_TeasingMore*__*.jpg",
f"{base}_*TeasingMore__*.jpg",
f"{name_only}_TeasingMore*__*.jpg",
f"{name_only}_*TeasingMore__*.jpg",
])
if f: copy_to(f, dst_folder/'Teasing More.jpg')
copy_stage_with_variants(topless_dir, f"{base}_Topless*", dst_folder, 'Topless')
# Topless may be keyed by long single base or short name-only base.
has_topless=copy_stage_with_variants(topless_dir, f"{base}_Topless*", dst_folder, 'Topless')
if not has_topless and name_only != base:
copy_stage_with_variants(topless_dir, f"{name_only}_Topless*", dst_folder, 'Topless')
if final_stage == 'Underwear L':
f=pick_latest(list(underwear_l_dir.glob(f"{base}_Underwear*__*.jpg")))
f=_pick_latest_globs(underwear_l_dir, [
f"{base}_Underwear*__*.jpg",
f"{name_only}_Underwear*__*.jpg",
])
if f: copy_to(f, dst_folder/'Underwear L.jpg')
elif final_stage == 'Underwear S':
f=pick_latest(list(underwear_s_dir.glob(f"{base}_Underwear*__*.jpg")))
f=_pick_latest_globs(underwear_s_dir, [
f"{base}_Underwear*__*.jpg",
f"{name_only}_Underwear*__*.jpg",
])
if f: copy_to(f, dst_folder/'Underwear S.jpg')
else:
copy_stage_with_variants(topless_plus_dir, f"{base}_Topless*", dst_folder, 'Topless Plus')
copy_stage_with_variants(undressed_dir, f"{base}*Undressed*", dst_folder, 'Undressed')
has_topless_plus=copy_stage_with_variants(topless_plus_dir, f"{base}_Topless*", dst_folder, 'Topless Plus')
if not has_topless_plus and name_only != base:
copy_stage_with_variants(topless_plus_dir, f"{name_only}_Topless*", dst_folder, 'Topless Plus')
copy_arrival(arrival_dir, base, dst_folder, True)
has_undressed=copy_stage_with_variants(undressed_dir, f"{base}*Undressed*", dst_folder, 'Undressed')
if not has_undressed and name_only != base:
copy_stage_with_variants(undressed_dir, f"{name_only}*Undressed*", dst_folder, 'Undressed')
if not copy_arrival(arrival_dir, base, dst_folder, True) and name_only != base:
copy_arrival(arrival_dir, name_only, dst_folder, True)
def copy_stages_double(base: str, dirs, dst_folder: Path, final_stage: str):
@@ -301,7 +577,13 @@ def main():
undressed_dir=stage_root/'Undressed'
arrival_dir=stage_root/'Arrival'
# preflight: no stage-letter tokens
# preflight: normalize provisional variant markers to canonical [1..n]
for folder in (topless_dir, topless_plus_dir, undressed_dir):
changes=normalize_provisional_markers(folder)
if changes:
print(f"Normalized {len(changes)} provisional marker(s) in {folder.name}")
# preflight: no stage-letter tokens in finalized folders
stage_pat=re.compile(r'_(?:[A-E])(?:_[A-E])?__|(?:TeasingMore|Teasing|Topless|ToplessPlus|UnderwearL|UnderwearS)[A-E](?:_[A-E])?__')
check_folders=[teasing_dir, teasing_more_dir, topless_plus_dir, underwear_l_dir, underwear_s_dir]
for folder in check_folders:
@@ -325,7 +607,7 @@ def main():
elif name.startswith('Triple_'):
triples.append(name.split('__')[0])
else:
singles.append(name.split('_Dressed__')[0])
singles.append(extract_single_base_from_dressed_filename(name))
# Deduplicate singles by extracted name (prefer shorter bases)
seen_names = {}
+25 -27
View File
@@ -8,7 +8,8 @@ import argparse
import re
import sys
from pathlib import Path
from collections import defaultdict
from finalize_courtesan_tier import normalize_provisional_markers
BASE_DIR = Path("/home/rick/output/to-dnd-session/Grok-NPCs/Courtesans")
@@ -20,53 +21,50 @@ def preflight_check(tier_dir: Path):
"""Run preflight checks on finalized stages only."""
issues = []
warnings = []
# Re-run packaging support: if cleanup already happened, inputs live under Stages and Raw Files.
stage_root = tier_dir / "Stages and Raw Files" if (tier_dir / "Stages and Raw Files").exists() else tier_dir
if stage_root != tier_dir:
print(f"INFO: using stage root {stage_root}")
# Check 1: No stage letters remaining in finalized stages
for stage in FINALIZED_STAGES:
stage_dir = tier_dir / stage
stage_dir = stage_root / stage
if not stage_dir.exists():
warnings.append(f"WARN: {stage} folder missing")
continue
for f in stage_dir.glob("*__*.jpg"):
# Check for stage letter patterns (but not Arrival which uses A1-C6 numbering)
if stage == "Arrival":
continue # Arrival uses letter+number, not just letter
# Pattern: _StageName + single letter + __
if re.search(r"_(Dressed|Teasing|TeasingMore)[A-Z]__", f.name):
issues.append(f"FAIL: Stage letter not stripped: {stage}/{f.name}")
# Check 2: Topless/Topless Plus/Undressed mixed bracketed/unnumbered
# Check 2: Normalize provisional markers in multi-variant stages.
# Supports markers like (3b), (3c), (last-2), (last-1), (last), etc.
for stage in ["Topless", "Topless Plus", "Undressed"]:
stage_dir = tier_dir / stage
stage_dir = stage_root / stage
if not stage_dir.exists():
continue
by_base = defaultdict(list)
for f in stage_dir.glob("*__*.jpg"):
base = re.sub(r'\[[^\]]+\]__', '__', f.name)
base = re.sub(r'__.*', '', base)
has_bracket = bool(re.search(r'\[[^\]]+\]', f.name))
by_base[base].append((f.name, has_bracket))
for base, files in by_base.items():
has_numbered = any(has_bracket for _, has_bracket in files)
has_unnumbered = any(not has_bracket for _, has_bracket in files)
if has_numbered and has_unnumbered:
issues.append(f"FAIL: Mixed bracketed/unnumbered in {stage}: {base}")
try:
changes = normalize_provisional_markers(stage_dir)
if changes:
print(f"INFO: normalized {len(changes)} provisional marker(s) in {stage}")
except Exception as e:
issues.append(f"FAIL: {stage} marker normalization: {e}")
# Check 3: Count finalized stage files
for stage in FINALIZED_STAGES:
stage_dir = tier_dir / stage
stage_dir = stage_root / stage
if stage_dir.exists():
count = len(list(stage_dir.glob("*__*.jpg")))
print(f"INFO: {count} {stage} files")
else:
warnings.append(f"WARN: {stage} folder missing")
# Check 4: ArrivalAll composites
arrival_dir = tier_dir / "Arrival"
arrival_dir = stage_root / "Arrival"
if arrival_dir.exists():
arrival_all_singles = list(arrival_dir.glob("ArrivalAllSingles_*.jpg"))
arrival_all_men = list(arrival_dir.glob("ArrivalAllMen_*.jpg"))