Commit 0d203a02 by PLN (Algolia)

docs(tasks): archive #115/#120/#121 — the day every layer saw the failure and none acted

Three closed tasks from the 2026-08-01 session, written for a cold reader. The
through-line worth mining later: the rig SENSES everything and ACTUATES nothing.
scsynth died and sclang logged it, the Bridge warned 63 times, gig-up saw it and
printed GO — and separately, PLN's ears found a preload bug that four green tools
had missed. Plus the session's sharpest lesson: two of my own fixes (a watchdog, a
chmod) were themselves broken in exactly the way they were meant to catch.
parent 23f60e76
...@@ -1421,3 +1421,104 @@ by something other than a human with habits. ...@@ -1421,3 +1421,104 @@ by something other than a human with habits.
**Deps.** Superseded by the board it leaves behind — #112 carries the triage, #16 carries **Deps.** Superseded by the board it leaves behind — #112 carries the triage, #16 carries
the remaining 20%. the remaining 20%.
---
## #115 — Supervise scsynth, not sclang
**Description.** The rig went silent overnight (2026-08-01, OPAL J-7) and nothing
reported it. PLN: *"no sound atm and i cant say if its pulsar, sclang, or else."*
**Done.** `parvagues-sc.service` enabled (commits b978a2f, 0c553fd) — it had been
`disabled`, the only user unit producing audio not in the boot set, alive since Jul 28
purely as a manual leftover. Added `Restart=on-failure` bounded by
`StartLimitBurst=3/300s`, plus `tools/sc-watchdog.sh` +
`parvagues-sc-watchdog.service` (both enabled), and `tools/tests/test-sc-watchdog.sh`
(7/7 against a fake unit + fake child, no real audio). Verified on the real rig:
`kill -9 scsynth` → recovered in **14s unattended**.
**Learnings.**
* **A green unit is not sound.** scsynth is sclang's CHILD and systemd's MainPID is
sclang, so the unit reported `active (running)` for two days while the rig was mute.
Status must derive from the thing that makes noise, never from `systemctl is-active`.
* **Every layer detected it; no layer acted.** sclang logged the death; the Bridge
printed `scsynth process not found` **63 times** to a journal nobody reads; gig-up
saw `scsynth=no` and still printed GO. Detection without a surface or an actuator is
worth nothing. That reframed #25 from "build a cockpit" to "the cockpit has eyes and
no mouth or hands" — PLN: *"'i have no mouth and i must livecode' poor gear-state"*.
* **A mitigation can be one layer too shallow.** The d834667 `QT_QPA_PLATFORM=offscreen`
fix worked perfectly — sclang survived the suspend. Suspend killed scsynth instead.
* **A supervisor must OUTLIVE its supervisee.** v1 flapped on PLN's desk. Not a bad
timeout: `PartOf=` propagates RESTARTS, so issuing the restart killed the watchdog,
`Restart=always` revived it, and the rate-limit counter reset to zero — MAX_RESTARTS
unreachable *by construction*. The state enforcing the limit was destroyed by the
action it limited. **My recovery caused more downtime than the fault it fixed.**
* **Stickiness is the signal.** Every toast was `urgency=critical`, which Plasma never
expires; recovery buried the screen. normal+timeout = handled itself; critical+sticky
= needs your hands.
* **A test must never page a human about something that is not real** — the harness
fired sticky toasts naming its own fake unit, already torn down.
**Deps.** Spawned #116, #117, #118, #119, #122. Reframed #25. Corrected #18's claim
that "the launch layer is largely done".
---
## #120 — Two disagreeing setlists
**Description.** PLN, live: *"getting some crackles as i move from do_it_right to
take_5_drops"*.
**Done.** Measured rather than guessed: 0 SuperDirt lates, 0 pipewire xruns, **17 lazy
soundfile reads, all `take5:*`** — disk I/O, not CPU. Root cause: `preload.scd` was
generated from `setlist_opal2026.txt` (10 tracks) while gig-up/cheat-sheet trust
`set-coherence.setlist_tracks()` (13). The three missing were `do_it_right`,
`take_5_drops`, `electric_hammer` — exactly the transition PLN was playing.
Regenerated (47 → 51 banks); `tools/check-preload.sh` added as a HARD gate (6ab09ea).
**Resolved by PLN's ruling: backlog.md is the SSOT, for membership AND order.** → #123.
**Learnings.**
* **A generated artifact with no freshness check is a stale artifact waiting for the
worst moment.** `preload.scd` is gitignored (correctly — generated), so no diff ever
showed it drifting, and nothing compared it to the set. Every tool was green.
* **Long uptime hides cold-start bugs.** It only surfaced because SuperDirt had been up
since Jul 28 with those banks cached from a first play weeks earlier; a restart
cleared the cache. Its natural debut was a cold boot **at the venue**.
* Compare **bank sets, not bytes** — the generator stamps a header and orders banks, so
a byte diff cries wolf on a no-op rebuild.
* Ears found this before any tool did. The measurement only confirmed what PLN heard.
**Deps.** Blocked-by-in-spirit #12 (running order). Spawned #123.
---
## #121 — gig-up --live / --converge
**Description.** Teach the pre-gig gate that a *rig* must exist, not just a set.
**Done.** `--live` (7 links, HARD/SOFT chosen per link), `--converge` (starts services,
reports apps), `tools/check-preload.sh`, and a HARD **"tools executable"** self-check.
Commits 6ab09ea, 8d30740. 16/16 green on the live rig.
**Learnings.**
* **The gate must be mutation-tested against the failure it exists for.** Run under a
PATH shim whose `pgrep` reports scsynth absent → NO-GO, exit 1, rig untouched. Green
proves nothing until you have seen it go red for the right reason.
* **`chmod +x` is not a fix — git records the mode separately.** gig-up's day-one win
(check-boot/check-tracks at 644) was FAKE: the chmod only touched the working tree,
git still had 100644, every fresh clone stayed broken. gig-up.sh itself was 100644
from birth.
* **"No bad output" is satisfied by a program that never ran.** The permission-denied
failure was invisible because the caller filtered output through `sed`, so a run that
never executed looked like a clean no-op — and was reported as one. Assert something
positive, never merely the absence of complaint.
* The self-check found **four** real problems immediately, including
`parvagues-sc-watchdog.service` enabled at boot with an ExecStart it could not
execute — stuck `activating`, restart-looping in silence. *The supervisor added to
catch silent failure was itself failing silently.*
* **Converge safely = every action conditional on breakage.** A healthy rig converges
to a verified no-op. Start ≠ restart: the signature failure (unit active, server
dead) needs a restart, because `start` is a no-op on an active unit.
**Deps.** Closes the assert+converge half of the "big red button". Feeds #116, #119.
Left open: `--audio` integration, and #122 (does Tidal actually resume?).
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment