Commit d5baca32 by PLN (Algolia)

feat(gig-log): log WHICH TRACK is loaded — the key that decodes the whole MIDI stream

PLN recorded an afternoon of OPAL takes, then asked for "a master split per track
loaded time". The answer should have been trivial. It was not: nothing in the rig
recorded WHEN each track was loaded.

The gig log had 78222 samples of thermals, xruns, gear and 7429 coalesced MIDI CC
events for today alone — and no way to say what any of it was. A CC 41 is
meaningless on its own, because the LCXL map is PER TRACK: on one track index 41 is
a crush bus, on another it is a break gate. The performance data was all there and
none of it was resolvable.

Meanwhile the exact ground truth was sitting in a 33-byte file the whole time.
Pulsar has published `~/.cache/parvagues/current-track` since #84 (2576e77) for the
LED watcher to follow. Nobody was reading it into the log.

WHY THIS MATTERS MORE THAN THE SPLIT IT WAS ASKED FOR

Splitting a livecoded set by ear is genuinely hard, and we have the scars: orbit
roles stay stable across tracks so level/presence detection fails, and every dN is a
4-cycle crossfade that bleeds the incoming track into the outgoing one's tail — the
existing Bandcamp splits carry that bleed. A `k:"track"` line replaces all of that
inference with a timestamp. Boundaries stop being clever and become read.

And it retro-actively upgrades the CC stream from telemetry to an annotated
performance score: track + CC + time resolves, through the per-track LCXL map we
already generate, to "d4's DJ filter opened at 18:42:13". That is the input for
telling practice apart from performance (hands-off vs hands-on density), for finding
the moment worth clipping, and eventually for the emotion/timeline bundle.

IMPLEMENTATION — deliberately the boring version

- `_read_track()` never raises: absent file (publisher not up yet) or a short read
  both degrade to None. A missing track must degrade the log, never stop it.
- Written ONLY on change, so a 3-hour session costs a handful of lines instead of
  10800 repeats of one string. The header carries the initial value, so a session
  that never switches still has one authoritative statement of what was loaded.
- Polled, not watched — for the same reason the LED watcher polls. One stat+read of
  33 bytes per second is unmeasurable; an inotify watch adds a dependency and a
  whole class of "the watch silently died", which is the exact failure mode this
  data exists to prevent.
- `load()` folds track records into `marks` so they render in the existing timeline
  for free, keeping `k:"track"` so a splitter can tell an automatic boundary from a
  human annotation.

VALIDATED
- `selftest` PASS; sampler + both readers still 0.66% of one core over 6s, well
  under the 3% observer budget (the observer must not perturb — we already learned
  that lesson when probe-chain's own capture streams caused the xruns).
- END-TO-END, not just unit: ran the real recorder against a temp track file, moved
  it twice, and read the log back. Header carried the initial track; both changes
  appeared exactly once with their `from`, each within the 1s poll. Green tests on a
  pure function would not have proven the value reaches the file.
parent 466df85d
...@@ -38,6 +38,22 @@ was 67072. Both count history, including power excursions and device changes tha ...@@ -38,6 +38,22 @@ was 67072. Both count history, including power excursions and device changes tha
never touched audio. Only the delta across the session says anything, so the header never touched audio. Only the delta across the session says anything, so the header
records the baseline and every sample records the delta. records the baseline and every sample records the delta.
RECORD KINDS (the `k` field)
hdr once, at start: machine state + counter baselines + the loaded track
s one per tick: thermals, freq, cpu, gear, xrun deltas
cc coalesced MIDI control-change, per (port, channel, controller) per second
note MIDI note, never coalesced (a note is an event, a CC is a state)
track the loaded .tidal changed (from Pulsar, #84) — written ONLY on change
mark a human annotation from `gig-log.py mark`
end once, at stop
`track` is what makes the `cc` stream MEAN something. A CC 41 is unresolvable on its
own — which control it is depends entirely on which track is loaded, because the
LCXL map is per-track. With the track timeline alongside it, the log stops being
telemetry and becomes an annotated performance score: which effect, on which orbit,
at which second. That is the input for splitting a session into tracks, for telling
practice apart from performance, and for finding the moment worth clipping.
USAGE USAGE
tools/gig-log.py record # 1 Hz recorder, Ctrl-C to stop tools/gig-log.py record # 1 Hz recorder, Ctrl-C to stop
tools/gig-log.py mark "vague de crime" # annotate the running session tools/gig-log.py mark "vague de crime" # annotate the running session
...@@ -69,6 +85,36 @@ import perf # noqa: E402 (rootless Thermals/_read live here already — DRY wi ...@@ -69,6 +85,36 @@ import perf # noqa: E402 (rootless Thermals/_read live here already — DRY wi
LOG_DIR = Path(os.environ.get( LOG_DIR = Path(os.environ.get(
"GIG_LOG_DIR", os.path.expanduser("~/.local/share/parvagues/gig-log"))) "GIG_LOG_DIR", os.path.expanduser("~/.local/share/parvagues/gig-log")))
# Where Pulsar publishes the loaded track (#84, pulsar-parvagues-hud). One tiny
# read per tick, and only a CHANGE is written — so a 3-hour session costs a handful
# of lines instead of 10800 repeats of the same string.
#
# WHY THIS EXISTS (2026-07-29): PLN recorded an afternoon of OPAL takes and asked to
# "master split per track loaded time". Nothing had logged WHEN each track was
# loaded, so the split had to be inferred from the audio — and inference on a
# livecoded set is genuinely hard (stable orbit roles defeat level/presence
# detection, and every dN is a 4-cycle crossfade that bleeds the next track into the
# previous one's tail). Meanwhile the exact answer was sitting in a 33-byte file the
# whole time. A `k:"track"` line is the ground truth that makes the splitter trivial
# instead of clever.
TRACK_FILE = Path(os.environ.get(
"PARVAGUES_TRACK_FILE",
os.path.expanduser("~/.cache/parvagues/current-track")))
def _read_track() -> str | None:
"""The repo-relative .tidal path Pulsar says is loaded, or None.
Never raises: the publisher may not have run yet (the file is absent until the
HUD activates), and a half-written file just reads short. A missing track must
degrade the log, never stop it.
"""
try:
s = TRACK_FILE.read_text(errors="replace").strip()
except OSError:
return None
return s or None
# Gear we care about, by /proc/<pid>/comm. A value of `True` means "sum the whole # Gear we care about, by /proc/<pid>/comm. A value of `True` means "sum the whole
# process group" — pulsar is 8 processes (main, GPU, renderers) and the interesting # process group" — pulsar is 8 processes (main, GPU, renderers) and the interesting
# number is the total, with the worst single one called out separately. # number is the total, with the worst single one called out separately.
...@@ -474,6 +520,10 @@ class Recorder: ...@@ -474,6 +520,10 @@ class Recorder:
self.path = self._open() self.path = self._open()
self.samples = 0 self.samples = 0
self._stop = threading.Event() self._stop = threading.Event()
# Last track seen, so only CHANGES are written. Seeded from the header's
# value below, which means a session that never switches track still has
# exactly one authoritative statement of what was loaded.
self._track: str | None = None
def _open(self) -> Path: def _open(self) -> Path:
self.dir.mkdir(parents=True, exist_ok=True) self.dir.mkdir(parents=True, exist_ok=True)
...@@ -516,6 +566,7 @@ class Recorder: ...@@ -516,6 +566,7 @@ class Recorder:
"base_pkg_throttle": self._pkg_base, "base_pkg_throttle": self._pkg_base,
"period": round(self.period, 3), "period": round(self.period, 3),
"xruns": bool(self.xr), "midi": bool(self.mid), "xruns": bool(self.xr), "midi": bool(self.mid),
"track": _read_track(),
"gear": {c: bool(self.gear._pids.get(c)) for c in GEAR}, "gear": {c: bool(self.gear._pids.get(c)) for c in GEAR},
} }
...@@ -544,11 +595,27 @@ class Recorder: ...@@ -544,11 +595,27 @@ class Recorder:
rec["xrun_by"] = hot rec["xrun_by"] = hot
return rec return rec
def poll_track(self) -> dict | None:
"""A `k:"track"` record when the loaded track changed, else None.
Polled rather than watched, for the same reason the LED watcher polls: one
stat+read of a 33-byte file per second is unmeasurable, while an inotify
watch adds a dependency and a whole class of "the watch silently died"
failures — which is precisely the bug this data is meant to prevent.
"""
cur = _read_track()
if cur == self._track:
return None
prev, self._track = self._track, cur
return {"t": round(time.time(), 3), "k": "track",
"path": cur, "from": prev}
def run(self) -> int: def run(self) -> int:
for t in (self.xr, self.mid): for t in (self.xr, self.mid):
if t: if t:
t.start() t.start()
self._write(self.header()) self._write(self.header())
self._track = _read_track()
self.prune() self.prune()
print(f"gig-log: recording -> {self.path}", file=sys.stderr) print(f"gig-log: recording -> {self.path}", file=sys.stderr)
print(f"gig-log: xruns={'pw-top' if self.xr else 'off'} " print(f"gig-log: xruns={'pw-top' if self.xr else 'off'} "
...@@ -560,6 +627,9 @@ class Recorder: ...@@ -560,6 +627,9 @@ class Recorder:
next_t += self.period next_t += self.period
self._write(self.sample()) self._write(self.sample())
self.samples += 1 self.samples += 1
tr = self.poll_track()
if tr:
self._write(tr)
if self.mid: if self.mid:
cc, disc = self.mid.drain() cc, disc = self.mid.drain()
for r in disc: for r in disc:
...@@ -844,6 +914,15 @@ def load(path: Path) -> tuple[dict, list[dict], list[dict], list[dict], dict]: ...@@ -844,6 +914,15 @@ def load(path: Path) -> tuple[dict, list[dict], list[dict], list[dict], dict]:
ccs.append(r) ccs.append(r)
elif k == "mark": elif k == "mark":
marks.append(r) marks.append(r)
elif k == "track":
# A track change IS a mark — the most reliable one in the log,
# because it comes from the editor rather than from a human
# remembering to type `gig-log.py mark`. Folded into `marks` so it
# shows up in the existing timeline for free; `k` stays "track" so
# the splitter can tell an automatic boundary from an annotation.
r = dict(r)
r["label"] = f"▸ {r.get('path') or '(none)'}"
marks.append(r)
elif k == "end": elif k == "end":
end = r end = r
return hdr, samples, ccs, marks, end return hdr, samples, ccs, marks, end
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment