# RUNBOOK — when it breaks mid-set

**Task #15.** Symptom-first, because that's how you read it in a panic: you don't know the
cause, you know *the sound stopped*. Written 2026-07-25 (J-10 to OPAL).

> **Rule zero: the room forgives silence, not flailing.** Everything below is ordered
> cheapest-first. Do not reboot SuperCollider as a first move — it costs 30–120s of dead air
> and loses every pattern. Almost nothing here needs it.

---

## 0. The 10-second triage

Something's wrong. Ask in this order:

1. **Is it only ONE stream?** → §1 (a track problem)
2. **Is it ALL sound, but Pulsar still responds?** → §2 (a routing/eval problem)
3. **Is Pulsar frozen/unresponsive?** → §3 (an editor problem — audio may be fine!)
4. **Did the laptop just wake / did the screen just do something weird?** → §4 (the display
   stack takes the audio stack down with it)

**Silence-with-a-working-editor is the most recoverable state there is.** Keep a hand on the
LCXL and keep the last working track's code in front of you.

---

## 1. One stream died

### 1a. You pushed a fader and the sound vanished — THE MUTE-BOMB
**This is the #1 historical footgun.** Old `gF1/2/3` sent values into `djfbus`, a phantom
SuperDirt effect; faders rest at 0, so gF-wrapped streams muted with no recovery.

- **Fixed permanently 2026-07-25** (commit `98d073f`) — `BootTidal.hs` now uses safe-at-rest
  `hpf`/`lpf` where fader-rest = bypass. A fresh boot is immune.
- **But if you're on a STALE boot** (an old ghci that survived, a machine that didn't pick up
  the new file): symptom is *push fader 49 → sound MUTES instead of sweeping*.
- **Rescue, no reboot:** open `live/lib/prelude.tidal`, Ctrl-Enter inside each `:{ … :}`
  block. It shadows the boot defs in the live session. Patterns keep playing.
- **Instant mitigation while you find it:** pull the fader back to 0. Rest = bypass on the
  new defs; on the old broken defs, 0 is the *mute* position, so if pulling to 0 makes it
  worse you have confirmed you're on a stale boot.

### 1b. A stream is just gone / wrong
- `d5 $ silence` then re-eval the block. With default-xfade `dN` (#13) this blends over 4
  cycles rather than cutting.
- Need it back NOW with no blend: `jumpTo 5 $ …` (orbit-correct hard cut) or `p 5 $ …`.
- Muted by a global mask? Check `gM1/2/3` — buttons **73 / 74 / 75**. Press to un-toggle.
- Panic chord stuck on? Buttons **73+74+91+92** together toggles `^93`. If `gPanic` is wired
  into that stream, `^93=1` means gain 0. Hold the chord again to release (LEDs show state).

---

## 2. All sound gone, Pulsar still works

### 2a. Check SuperDirt is actually alive — VERIFIED diagnostic
```sh
pgrep -x scsynth && ss -uln | grep 57120
```
- **Both OK** → SuperDirt is up, so it's a Tidal-side problem → §2b.
- **Nothing** → scsynth died → §5 (the only real reboot case).

### 2b. Tidal booted but nothing reaches SuperCollider — the ZOMBIE PORT
**Observed live 2026-07-25.** A leftover `ghc` process holds Tidal's control port 6010; the
new Tidal boots, fails to bind, and prints *"Can't handshake with SuperCollider without
control port"*. Pulsar looks completely normal. You eval, and nothing happens — **no error**.

```sh
ss -lunp | grep 6010          # who holds it
```
Expected healthy output — exactly one `ghc` process, and it's *your current* Tidal:
```
UNCONN 0 0 127.0.0.1:6010 0.0.0.0:* users:(("ghc-9.4.7",pid=NNNN,fd=22))
```
If that pid is a stale one, kill it (`kill NNNN`) and restart Tidal **in Pulsar only**
(Ctrl-Alt-B / Tidal: Boot) — SuperCollider is untouched, samples stay warm.

> Worth doing as part of soundcheck, not just as a fix: this failure is *silent*.

### 2c. Ardour stole the LaunchControl XL
Whoever opens the LCXL first wins it. `gig-up.sh` exists precisely to force
SuperDirt-first ordering, but if something restarted out of order:
```sh
aconnect -l | grep -iE "client [0-9]+:"
```
You want a `client 128: 'SuperCollider' [type=user,...]` present. If SuperCollider isn't
listed as a MIDI client, it never claimed the controller — knobs will do nothing while
audio still plays.

### 2d. Master path
- Ardour master fader / limiter — is the record-bus limiter clamping everything?
- `# gain` global: `^77` is the global gain fader (`midiGGlobal = "^77" * 1.3`). **If fader
  77 is down, everything is silent and nothing looks broken.** Check it early — it's a
  one-move fix that's easy to overlook.

---

## 3. Pulsar frozen or crawling

**Audio usually keeps playing — do not panic-kill it.** Tidal patterns live in the ghci
process, not the editor; a wedged renderer doesn't stop the music.

- **`Window: Reload`** (Ctrl-Alt-Ctrl-L / command palette) — reloads the renderer, keeps the
  ghci session and therefore the playing patterns. This is the cheap fix and it resolves the
  known highlighter marker-leak renderer burn.
- If the reload loses the Tidal session, re-boot Tidal in Pulsar and re-eval the current
  track from the file — SuperDirt and your samples are still warm.
- Known standing issue: the highlighter marker pool leaks and pegs a core (#7). Reload is the
  mitigation until that's fixed.

---

## 4. Post-suspend / display-stack death

**Established root cause** (memory: `project_resume_audio_crash`): on resume, kwin can abort,
and because sclang was Qt-linked it got cascade-killed along with scsynth and Ardour —
"music death" after an overnight suspend.

- **Mitigated** by headless sclang (`QT_QPA_PLATFORM=offscreen`, commit `d834667`) — SuperDirt
  no longer shares fate with the display stack.
- **Don't suspend the gig laptop between soundcheck and set.** The mitigation reduces the
  trigger; it doesn't make resume free.
- dGPU off (`./gpu-mode.sh`) further reduces the trigger.

---

## 5. Full SuperCollider restart — the LAST resort

Only when scsynth is genuinely dead (§2a found nothing).

```sh
./gig-up.sh
```
Idempotent: it converges the rig rather than spawning duplicates, brings SuperDirt up
**first**, waits for a real readiness gate (scsynth alive + `:57120` listening + a
SuperCollider MIDI client), and only then touches Ardour/Pulsar. Preloads the last 20 edited
tracks' samples so the first plays don't crack.

**Expect up to ~120s** (`READY_TIMEOUT`). That is a long time on stage. Have something to say,
or a track already playing off another source.

---

## 6. Hardware

### LCXL lights are dark but knobs still work
**Diagnosed pattern** (memory: `reference_lcxl_led_stall`): the USB **OUT** endpoint stalls
(urb -32) while IN keeps working — so input is alive and LED feedback is dead.
**Fix: physically replug the LCXL.** It re-enumerates and LEDs come back. Input bindings
survive because the CC→OSC bridge re-claims it.

### Knobs do nothing at all
Not the same failure — that's §2c (SuperCollider never claimed the controller).

---

## 7. Always-on recording — check it's still going

The set is being captured continuously to rotating opus segments, independent of Ardour.

```sh
bash tools/gig_record.sh status
```
Healthy output is `gig_record: RUNNING (pid NNNN)` plus recent segment files in
`recordings/`. A brand-new segment reading `0` bytes is **normal** — it just rotated (they
rotate hourly).

If it died mid-set: `bash tools/gig_record.sh start`. It's a separate process from Ardour
on purpose — if Ardour dies you still have the set.

---

## Pre-set checklist (the 2 minutes that prevent §2b and §2d)

1. `./gig-up.sh` — wait for the readiness gate, don't race it.
2. `ss -lunp | grep 6010` → exactly one ghc, and it's the current one.
3. `aconnect -l | grep SuperCollider` → present.
4. **Fader 77 up.** Faders 49/50 down (= bypass).
5. Eval one track, hear it, then `hush`.
6. `bash tools/gig_record.sh status` → RUNNING.
7. Do **not** suspend the laptop after this point.

---

## Status of these procedures

- **§1a, §2a, §2b, §2c, §7** — commands verified by running them on 2026-07-25.
- **§4, §6** — from documented prior incidents with established root causes; the *fixes*
  (headless sclang, replug) are known-good, but not re-tested tonight.
- **§5** — `gig-up.sh` behaviour read from source, not exercised end-to-end tonight. Worth a
  real rehearsal run (#14), because a runbook you've never executed is a hypothesis.
