fix(preload): the eager-preload existed — it just disabled the safety net it replaced

THE SILENT ORBITS WERE A TWO-PART INTERACTION, AND PART TWO WAS OURS.

Symptom, for two evenings: orbits went silent mid-set. Which ones changed every
boot. The boot log looked perfect — every bank registered with the right file
count — and 19741 `Buffer UGen: no buffer data` messages piled up in a single 6h
session where no performer would ever see them.

Part one is upstream design. `~dirt.doNotReadYet = true` registers banks from WAV
HEADERS ONLY; the audio is read on first use. That read is fired asynchronously by
`readFileIfNecessary` (DirtSoundLibrary.sc:268-274), which returns immediately — so
the synth for the very event that triggered the read gets a bufnum with no samples
in it. A correct file count in the log proves nothing about playability, because
the count comes from the header.

Part two is ours, and it is the one that turned a first-hit glitch into a dead
orbit. `preload.scd` — our own generated warm-up — set `~dirt.doNotReadYet = false`
and NEVER SET IT BACK. `readFileIfNecessary` has exactly one caller, and that call
site is gated on `doNotReadYet` (DirtSoundLibrary.sc:259-261). With the flag left
false, a bank registered header-only at boot is never read. Not read late: never.
So every bank the whitelist happened to MISS was permanently silent, forever.

And the whitelist missed banks routinely, because gig-up.sh built it from the N
most-recently-EDITED tracks (`--last`). That is a guess about what will be played.
the_revolution_will_be_sampled had not been edited recently, so `like_sugar` and
`the_revolution` were never warmed — which is precisely the set of banks feeding
the six orbits that were dead. Whitelist-by-mtime also explains the symptom that
made this look haunted: the misses changed as files were touched, so a different
set of orbits died on each boot.

Neither half is fatal alone. A miss with the flag restored is a slow first hit; a
flipped flag with a complete whitelist is invisible. Together they are a
silent-orbit generator.

THE FIX, in the generator that emits preload.scd:
  - Restore `doNotReadYet = true` after warming. This is the load-bearing line: it
    demotes the whitelist from load-bearing to a mere optimisation, which is the
    only thing it is safe to be on stage.
  - Drive the whitelist from an explicit setlist file (setlist_opal2026.txt) rather
    than from mtimes. Warm what will be PLAYED. `--last` stays for jamming.
  - Emit a per-bank COUNT ASSERTION. Re-loading a bank REPLACES it (addBuffer
    defaults appendToExisting = false, DirtSoundLibrary.sc:42-46), so this is
    index-stable and `like_sugar:21` keeps its meaning; the assertion guards the one
    case that would break that, a whitelist gone stale against the folders.
  - gig-up.sh writes via a temp file, because `> preload.scd` truncates BEFORE the
    generator runs — a failure used to leave an empty preload the boot read as
    "nothing to warm".

The assertion earned its keep on its first run, failing `jbk_kick: expected 508,
got 254`. Exactly 2x, and the tool was wrong, not the rig: that folder symlinks
into the rhadamanthe pack, which carries an AppleDouble `._X.wav` beside every real
`X.wav`. `glob('*.wav')` matches both; SuperDirt's `pathMatch("*")` skips dotfiles;
`ls` hides them so the folder looks clean. Counting now skips dotfiles.

VALIDATION — instrumental, no ears (PLN at work), two timepoints, per the rule that
short windows make a time-varying fault look static:
  BEFORE: the_revolution_will_be_sampled, d4/d5/d9/d10/d11/d12 all -inf at source.
  AFTER : all 11 declared orbits carry signal, and STILL carry it on a second probe
          ~95s later with no re-eval (levels stable to ~0.2 dB).
  no buffer data since the eval: 0  (was 19741 in 6h)
  Preload: 47/47 banks pass the count assertion, 2352 files, 2.2 s, 1.2 GB.
  Also fully green: perfect 12/12 orbits, gimme_acid 10/10, mafia_sans_serif 7/7,
  wap 5/5. Bare `s "like_sugar*4"` / `"the_revolution*2"` with no helpers: audible.

STILL BROKEN, SEPARATELY: vague_de_crime is silent at source with ZERO buffer
errors, and every one of its dead orbits routes through the ^NN helpers
(gMute2/gM1/gM3/gF1/gF3). That is #61 — an untouched `^NN` evaluates to `silence`,
not 0 — not this bug. Fixing the buffers removed the noise that was hiding it.

Refs #26, #63. Surfaces #61. Note for #44: the readiness gate passes BEFORE the
preload finishes, since "listening to Tidal" is printed by ~dirt.start well ahead
of the warm-up.

gig-up.sh

@@ -62,12 +62,34 @@ ensure_perf(){
 # --- generate the set-specific sample preload the boot will warm ---
 gen_preload(){
   [ "${PRELOAD_TRACKS}" = 0 ] && { info "preload: disabled (PRELOAD_TRACKS=0)."; return; }
-  if python3 "$DIR/tools/setlist_samples.py" --last "$PRELOAD_TRACKS" --emit-sc > "$DIR/preload.scd" 2>/dev/null; then
-    ok "preload: $(grep -c loadSoundFiles "$DIR/preload.scd" 2>/dev/null || echo 0) sample folders queued from your last $PRELOAD_TRACKS tracks."
+
+  # Prefer the explicit SETLIST over --last N. `--last` guesses the set from file
+  # mtimes, and on 2026-07-28 that guess silently omitted a set track's banks — back
+  # when a preload miss meant permanent silence rather than a slow first hit. Derive
+  # the warm-up from what will be PLAYED. --last stays as the fallback for jamming.
+  local SETLIST="${SETLIST:-$DIR/setlist_opal2026.txt}"
+  local args desc
+  if [ -f "$SETLIST" ]; then
+    args="--setlist $SETLIST"; desc="setlist $(basename "$SETLIST")"
+  else
+    args="--last $PRELOAD_TRACKS"; desc="your last $PRELOAD_TRACKS edited tracks"
+    warn "preload: no setlist at $SETLIST — falling back to mtime guessing."
+  fi
+
+  # Generate to a temp file and move it into place: `> preload.scd` truncates before
+  # the generator runs, so a failure would otherwise leave an EMPTY preload.scd that
+  # the boot happily sources as "nothing to warm".
+  local tmp; tmp="$(mktemp)"
+  if python3 "$DIR/tools/setlist_samples.py" $args --emit-sc > "$tmp" 2>"$tmp.err"; then
+    mv "$tmp" "$DIR/preload.scd"
+    ok "preload: $(grep -c '^\s*\[ \\' "$DIR/preload.scd" 2>/dev/null || echo 0) banks queued from $desc."
+    [ -s "$tmp.err" ] && warn "preload: generator notes — $(head -3 "$tmp.err" | tr '\n' ' ')"
   else
-    warn "preload: generation failed (python3 / tool?) — samples will lazy-load."
-    rm -f "$DIR/preload.scd"
+    warn "preload: generation failed — samples will lazy-load on demand (not fatal)."
+    [ -s "$tmp.err" ] && warn "preload: $(head -3 "$tmp.err" | tr '\n' ' ')"
+    rm -f "$tmp" "$DIR/preload.scd"
   fi
+  rm -f "$tmp.err"
 }
 
 # --- terminal for the sclang post window ---

setlist_opal2026.txt

+# OPAL 2026 — the tracks to warm at boot (gig-up.sh reads this).
+#
+# One track per line: a path under the repo, or a bare name resolved under live/.
+# `#` comments and blank lines are ignored.
+#
+# WHY THIS FILE EXISTS
+# The preload whitelist used to be derived from the N most-recently-EDITED tracks
+# (`setlist_samples.py --last`). That is a guess, and on 2026-07-28 it was wrong in
+# the worst way: the_revolution_will_be_sampled had not been edited recently, so its
+# banks (like_sugar, the_revolution) were left out of the whitelist — and back then a
+# miss meant PERMANENT silence, not a slow first hit. Six orbits were dead on stage.
+# Derive the warm-up from what will be PLAYED, not from what was last touched.
+#
+# Order is irrelevant here (it is a set, not a sequence) — keep it in show order
+# anyway so this doubles as a readable setlist. Source: backlog.md "## OPAL 2026".
+
+# --- Ouverture
+live/midi/nova/techno/bombe_dj.tidal
+live/midi/nova/dnb/wap.tidal
+
+# --- We call it TechnoJazz
+live/collab/raph/piment_bresilien.tidal
+live/midi/nova/remix/perfect.tidal
+live/midi/nova/acid/gimme_acid.tidal
+live/techno/vague_de_crime.tidal
+live/collab/raph/mafia_sans_serif.tidal
+live/midi/nova/dnb/liquid/you_my_sunshine.tidal
+
+# --- Finale
+live/collab/raph/desire.tidal
+
+# --- Encore
+live/midi/nova/jazz/the_revolution_will_be_sampled.tidal
+
+# Not yet resolved to files (in backlog but no .tidal located 2026-07-28):
+#   "Am i Doing it Right?", "Take five Drops", "Le shifteur marteau"
+# If these are real tracks under another name, add them here — a track missing from
+# this list is no longer fatal (lazy fallback is restored), but it does mean a
+# first-hit delay on its samples.

tools/setlist_samples.py

@@ -71,6 +71,46 @@ def last_n_tracks(n):
     return tracks[:n]
 
 
+# Audio extensions SuperDirt will actually register, compared case-insensitively.
+AUDIO_EXT = {'.wav', '.aiff', '.aif', '.flac', '.ogg'}
+
+
+def bank_file_count(folder):
+    """How many files SuperDirt will register for this folder.
+
+    Mirrors `pathMatch(folder +/+ "*")` (DirtSoundLibrary.sc:180): non-recursive,
+    and DOTFILES ARE EXCLUDED because a glob of `*` does not match them.
+
+    That last detail is not pedantry. `Dirt-Samples/jbk_kick` is a symlink into the
+    rhadamanthe pack, which carries an AppleDouble `._X.wav` twin beside every real
+    `X.wav`. A naive `glob('*.wav')` counts 508 where SuperDirt registers 254 — and
+    `ls` hides the twins, so the folder looks fine. Counting them would make the
+    boot-time integrity check cry wolf on a perfectly good bank.
+    """
+    n = 0
+    for p in folder.iterdir():
+        if p.name.startswith('.'):
+            continue
+        if p.suffix.lower() in AUDIO_EXT and p.is_file():
+            n += 1
+    return n
+
+
+def read_setlist(path):
+    """One track name/path per line; `#` comments and blank lines ignored.
+
+    A setlist FILE rather than a baked-in list, so reordering the show or swapping
+    a track never means editing Python — and so the preload whitelist is derived
+    from what will actually be PLAYED. See the warning in main() about --last.
+    """
+    tracks = []
+    for raw in Path(path).read_text().splitlines():
+        line = raw.split('#', 1)[0].strip()
+        if line:
+            tracks += resolve_track(line)
+    return tracks
+
+
 REPO = HOME / "Work/Sound/Tidal"
 
 
@@ -89,21 +129,96 @@ def resolve_track(arg):
     return matches[:1]
 
 
+def emit_sc(resolved, unresolved, tracks):
+    """Emit a SELF-CHECKING preload block for start_and_midi.scd to executeFile.
+
+    Two things in here are load-bearing, both learned the hard way on 2026-07-28.
+
+    1. `doNotReadYet` IS RESTORED TO TRUE AT THE END.
+       The previous version of this emitter set it to false and left it false. That
+       looks harmless and is catastrophic: `readFileIfNecessary` has exactly ONE
+       caller (DirtSoundLibrary.sc:259-261) and it is gated on `doNotReadYet`. With
+       the flag false, a bank that was registered header-only at boot is NEVER read
+       — not "read late", never. So every bank this whitelist MISSED became
+       permanently silent, emitting `Buffer UGen: no buffer data` on each event
+       (19741 of them in one 6h session) with no error a performer would ever see.
+       Restoring the flag makes a miss degrade to lazy-on-demand — a tiny first-hit
+       delay — instead of silence. It turns this whitelist from load-bearing into an
+       optimisation, which is the only safe thing for it to be on stage.
+
+    2. A per-bank COUNT ASSERTION.
+       Re-loading a bank REPLACES it (`addBuffer` defaults appendToExisting = false,
+       DirtSoundLibrary.sc:42-46), so this is index-stable and `like_sugar:21` keeps
+       meaning the same file. The assertion guards the one case that would break
+       that: a whitelist gone stale against the folders on disk. A silently
+       renumbered bank mid-set is worse than a silent one.
+    """
+    print('// preload — set-specific samples. GENERATED by tools/setlist_samples.py.')
+    print('// DO NOT HAND-EDIT: regenerate via gig-up.sh, or')
+    print('//   tools/setlist_samples.py --setlist setlist_opal2026.txt --emit-sc > preload.scd')
+    print('//')
+    print('// Warms the set\'s banks so they are read at BOOT instead of racing a')
+    print('// first-hit async read mid-set. `doNotReadYet` is restored to true at the')
+    print('// end so anything NOT listed here still lazy-loads on demand rather than')
+    print('// going permanently silent — see emit_sc() in the generator for why that')
+    print('// one line is the difference between a slow first hit and a dead orbit.')
+    print('//')
+    print(f'// {len(tracks)} track(s) scanned, {len(resolved)} banks, '
+          f'{len(unresolved)} non-sample tokens ignored.')
+    print()
+    print('var ok = 0, bad = [], t0 = Main.elapsedTime;')
+    print('~pvPreload = [')
+    for name in sorted(resolved):
+        folder = resolved[name]
+        print(f'\t[ \\{name}, {bank_file_count(folder)}, "{folder}" ],')
+    print('];')
+    print("""
+~dirt.doNotReadYet = false;              // read AUDIO now, not just WAV headers
+"\\n=== PRELOAD: % banks, eager ===".format(~pvPreload.size).postln;
+~pvPreload.do { |entry|
+\tvar name = entry[0], expected = entry[1], folder = entry[2], got;
+\t~dirt.loadSoundFiles(folder);
+\tgot = (~dirt.soundLibrary.buffers[name] ? []).size;
+\tif(got == expected) {
+\t\tok = ok + 1;
+\t} {
+\t\tbad = bad.add("%: expected % files, got %".format(name, expected, got));
+\t};
+};
+~dirt.doNotReadYet = true;               // CRITICAL: keep the lazy fallback alive
+"=== PRELOAD: %/% banks OK in % s ===".format(
+\tok, ~pvPreload.size, (Main.elapsedTime - t0).round(0.1)).postln;
+if(bad.notEmpty) {
+\t"=== PRELOAD COUNT MISMATCH — whitelist is STALE, regenerate it ===".postln;
+\tbad.do { |m| ("    " ++ m).postln };
+};""")
+
+
 def main():
     ap = argparse.ArgumentParser(description=__doc__,
                                  formatter_class=argparse.RawDescriptionHelpFormatter)
     ap.add_argument('tracks', nargs='*', type=Path)
     ap.add_argument('--last', type=int, metavar='N',
-                    help='use the N most-recently-edited live/ tracks')
+                    help='use the N most-recently-edited live/ tracks (NOT for gig '
+                         'preload — see --setlist)')
+    ap.add_argument('--setlist', metavar='FILE',
+                    help='track list to warm, one per line (# comments ok). PREFER '
+                         'THIS for a gig: --last guesses from mtime and silently '
+                         'omits set tracks you have not edited recently.')
     ap.add_argument('--emit-sc', action='store_true',
                     help='print a SuperCollider preload snippet for the resolved folders')
     args = ap.parse_args()
 
     tracks = [p for arg in args.tracks for p in resolve_track(arg)]
+    if args.setlist:
+        tracks += read_setlist(args.setlist)
     if args.last:
         tracks += last_n_tracks(args.last)
     if not tracks:
-        ap.error('give track names/paths or --last N')
+        ap.error('give track names/paths, --setlist FILE, or --last N')
+    # De-dup while preserving order: the same track can arrive from a setlist and
+    # from --last, and loading a bank twice is wasted boot time.
+    tracks = list(dict.fromkeys(tracks))
 
     idx = build_index()
     names, per_track = set(), {}
@@ -121,12 +236,7 @@ def main():
     folders = sorted({str(p) for p in resolved.values()})
 
     if args.emit_sc:
-        print('// preload — set-specific samples (setlist_samples.py)')
-        print('~dirt.doNotReadYet = false;')
-        for f in folders:
-            print(f'~dirt.loadSoundFiles("{f}");')
-        print(f'// {len(folders)} folders, {len(resolved)} names; '
-              f'{len(unresolved)} unresolved (lazy fallback)')
+        emit_sc(resolved, unresolved, tracks)
         return
 
     print(f'tracks scanned : {len(per_track)}')