feat(surface): measure whether each orbit's knobs sit ABOVE its fader — the…

feat(surface): measure whether each orbit's knobs sit ABOVE its fader — the whole corpus is off by one

PLN, mid-remap, spotted the half of #46 I had not measured:

    "but we need the coverage of the effects moving in the tracks, e.g. bass
     from 81 to 80 means effects on 53 now move to 52"

He is right, and my earlier conflict scan answered the wrong question. That scan
looked for CC *collisions* — two things claiming one control — and found only two
(^78, ^14), which is how #46 came to be costed as "two track edits". But the LCXL
is a GRID of eight channel strips, and the property that makes a surface readable
is not absence-of-collision, it is COLUMN COHERENCE: the knobs directly above a
fader must shape the same orbit that fader levels. Otherwise the bass fader is in
column 4 while the bass filter is in column 5, and every reach is a lookup.

tools/surface-columns.py measures it: per track, per orbit, which grid columns
that orbit's "^NN" references actually land in, and what would have to move for
column == orbit. It separates two classes, because conflating them would have
produced a work list that is mostly noise:

  * TRACK controls — a raw ^NN in the .tidal. Free to move; a text edit.
  * HELPER controls — CCs baked into BootTidal (gF1/2/3 on 49/50/51, gMask 41,
    gMute1-3 on 73/74/75, gPanic 93, plus 18/34/77 — 11 in all). A track cannot
    move these by editing itself. They are global by construction and will always
    read as "misaligned" against a per-column model.

THE RESULT, over the 13 OPAL setlist tracks — 84 d1-d8 orbits:

    only 19/84 orbits are column-aligned; full alignment = 174 ^NN renumbers

and the pattern is startlingly uniform across all 13 files:

    d1 -> col 2    d4 -> col 5    d7 -> col 7   (aligned)
    d2 -> col 3    d5 -> col 6    d8 -> col 8   (aligned)
    d3 -> col 4    d6 -> col 7

So the corpus convention is "orbit N lives in column N+1" for d1-d6, and N+0 for
d7-d8. The +1 is not an accident: column 1's C-knob and both its buttons are
already spoken for by gF1 / gMask / gMute1, so per-orbit controls were pushed one
column right to dodge them. And the two rules meet badly — column 7 is double
booked by d6 and d7 (in desire.tidal both really do react to ^59).

The consequence for #46 is the useful part: because the corpus is +1 for six
orbits and +0 for two, THERE IS NO FADER MAPPING THAT MAKES TODAY'S TRACKS
COHERENT. Shifting the faders +1 to match would strand d8; leaving them arbitrary
is where we are. Either the tracks move, or the surface stays a lookup. PLN's
instinct — that the Ardour re-learn is only half the job — was exactly right.

Survey, not a gate: exits 0 always, because alignment is a design choice and this
tool's job is to price it, not to enforce it. --plan prints the exact ^NN -> ^NN
moves per track for when we do it (#92).

tools/surface-columns.py

+#!/usr/bin/env python3
+"""surface-columns — does each orbit's controls live in that orbit's COLUMN?
+
+The LCXL is a grid: eight columns, each a channel strip.
+
+      col      1    2    3    4    5    6    7    8
+  A knob     13   14   15   16   17   18   19   20
+  B knob     29   30   31   32   33   34   35   36
+  C knob     49   50   51   52   53   54   55   56
+  button1    41   42   43   44   57   58   59   60
+  button2    73   74   75   76   89   90   91   92
+  D fader    77   78   79   80   81   82   83   84
+
+#46 makes fader D_N control orbit N. That only pays off if the knobs ABOVE
+each fader shape the SAME orbit -- otherwise the bass fader is in column 4
+while the bass filter is in column 5, and the surface is a lie you have to
+remember your way around.
+
+PLN put it exactly right while mapping (2026-07-29):
+
+    "we need the coverage of the effects moving in the tracks, e.g. bass from
+     81 to 80 means effects on 53 now move to 52"
+
+So the Ardour re-learn is only HALF the remap. This tool measures the other
+half: per track, per orbit, which columns does that orbit actually reach into,
+and what would have to move for column == orbit.
+
+Two classes of control, and the distinction is the whole point:
+
+  TRACK controls -- a raw "^NN" written in the .tidal file. These are free to
+      move. Renumbering them is a text edit.
+
+  HELPER controls -- CCs baked into BootTidal.hs (gF1/gF2/gF3 on 49/50/51,
+      gMask on 41, gMute1-3 on 73/74/75, gPanic on 93...). A track cannot move
+      these by editing itself; they are global by construction, and they will
+      always look "misaligned" against a per-column model. Reporting them as
+      work to do would be noise, so they are counted separately.
+
+Usage:
+    tools/surface-columns.py                  # the OPAL setlist
+    tools/surface-columns.py --all            # every .tidal under live/
+    tools/surface-columns.py TRACK.tidal ...  # named tracks
+    tools/surface-columns.py --plan TRACK     # the exact ^NN -> ^NN moves
+
+Exit 0 always: this is a survey, not a gate. Alignment is a design choice.
+"""
+from __future__ import annotations
+
+import argparse
+import re
+import sys
+from collections import defaultdict
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+from pvlint.core import Track, load, strip_comment  # noqa: E402
+
+ROOT = Path(__file__).resolve().parent.parent
+BOOT = ROOT / "BootTidal.hs"
+SETLIST = ROOT / "armada" / "setlist_opal2026.txt"
+
+# cc -> (column, row-label). Built from the physical grid above.
+GRID: dict[int, tuple[int, str]] = {}
+for _n in range(1, 9):
+    GRID[12 + _n] = (_n, "A")
+    GRID[28 + _n] = (_n, "B")
+    GRID[48 + _n] = (_n, "C")
+    GRID[76 + _n] = (_n, "fader")
+for _n in range(1, 5):
+    GRID[40 + _n] = (_n, "btn1")
+    GRID[72 + _n] = (_n, "btn2")
+for _n in range(5, 9):
+    GRID[52 + _n] = (_n, "btn1")   # 57..60
+    GRID[84 + _n] = (_n, "btn2")   # 89..92
+
+CC_REF = re.compile(r'"\^(\d+)"')
+
+# Faders are Ardour's, never Tidal's -- seeding or reading them from here is the
+# CC77-goes-to-silence footgun. A track referencing one is a CONFLICT, not a
+# column question, so flag rather than plan a move.
+ARDOUR_CC = set(range(77, 85))
+
+
+def helper_ccs(boot: Path = BOOT) -> set[int]:
+    """CCs hard-wired into BootTidal's helpers -- not movable by a track edit."""
+    if not boot.exists():
+        return set()
+    text = boot.read_text()
+    # Only the helper let-block, not the #55 seed list (which mentions every CC).
+    i = text.find("let -- DPV specific parameters")
+    if i >= 0:
+        j = text.find("\n:}", i)
+        text = text[i:j if j > 0 else len(text)]
+    return {int(m) for m in CC_REF.findall(text)}
+
+
+def orbit_ccs(track: Track) -> dict[int, list[tuple[int, int]]]:
+    """orbit number -> [(cc, 1-indexed line)] for every ^NN it references."""
+    out: dict[int, list[tuple[int, int]]] = defaultdict(list)
+    for orb in track.orbits():
+        for off, raw in enumerate(orb.lines):
+            for m in CC_REF.finditer(strip_comment(raw)):
+                out[orb.number].append((int(m.group(1)), orb.start + off))
+    return out
+
+
+def analyse(track: Track, helpers: set[int]) -> dict:
+    """Per-orbit column alignment for one track."""
+    rows = []
+    for orbit, refs in sorted(orbit_ccs(track).items()):
+        own, held, ardour = [], [], []
+        for cc, line in refs:
+            if cc in ARDOUR_CC:
+                ardour.append((cc, line))
+            elif cc in helpers:
+                held.append((cc, line))
+            else:
+                own.append((cc, line))
+        cols = {GRID[cc][0] for cc, _ in own if cc in GRID}
+        offgrid = sorted({cc for cc, _ in own if cc not in GRID})
+        # An orbit only *has* a column if it lives in 1..8 -- d9..d12 are the
+        # knob-row orbits under #46 and have no fader column of their own.
+        target = orbit if 1 <= orbit <= 8 else None
+        aligned = target is not None and cols == {target}
+        rows.append(dict(orbit=orbit, target=target, cols=sorted(cols),
+                         aligned=aligned, own=own, held=held,
+                         ardour=ardour, offgrid=offgrid))
+    return dict(track=track, rows=rows)
+
+
+def moves(row: dict) -> list[tuple[int, int, int]]:
+    """(cc, new_cc, line) to bring one orbit's own controls into its column."""
+    t = row["target"]
+    if t is None:
+        return []
+    out = []
+    for cc, line in row["own"]:
+        if cc not in GRID:
+            continue
+        col, kind = GRID[cc]
+        if col == t:
+            continue
+        new = {"A": 12, "B": 28, "C": 48}.get(kind)
+        if new is None:            # a button or a fader -- different geometry
+            if kind == "btn1":
+                new_cc = (40 + t) if t <= 4 else (52 + t)
+            elif kind == "btn2":
+                new_cc = (72 + t) if t <= 4 else (84 + t)
+            else:
+                continue
+        else:
+            new_cc = new + t
+        out.append((cc, new_cc, line))
+    return out
+
+
+def resolve(names: list[str]) -> list[Path]:
+    if not names:
+        if not SETLIST.exists():
+            raise SystemExit(f"surface-columns: no setlist at {SETLIST}")
+        names = [l.strip() for l in SETLIST.read_text().splitlines()
+                 if l.strip() and not l.startswith("#")]
+    out = []
+    for n in names:
+        p = Path(n)
+        if p.exists():
+            out.append(p)
+            continue
+        hits = sorted(ROOT.glob(f"live/**/{Path(n).stem}.tidal"))
+        if hits:
+            out.append(hits[0])
+        else:
+            print(f"  ?? no .tidal found for {n}", file=sys.stderr)
+    return out
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__,
+                                 formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("tracks", nargs="*", help="tracks (default: OPAL setlist)")
+    ap.add_argument("--all", action="store_true", help="every .tidal under live/")
+    ap.add_argument("--plan", action="store_true",
+                    help="print the exact ^NN -> ^NN moves per track")
+    a = ap.parse_args()
+
+    paths = (sorted(ROOT.glob("live/**/*.tidal")) if a.all else resolve(a.tracks))
+    helpers = helper_ccs()
+    print(f"surface-columns: {len(paths)} track(s) | "
+          f"{len(helpers)} helper CCs held by BootTidal: "
+          f"{','.join(str(c) for c in sorted(helpers))}\n")
+
+    tot_orb = tot_ok = tot_moves = 0
+    per_track = []
+    for p in paths:
+        try:
+            tr = load(p)
+        except Exception as e:                      # noqa: BLE001
+            print(f"  !! {p.name}: {e}")
+            continue
+        res = analyse(tr, helpers)
+        n_moves = sum(len(moves(r)) for r in res["rows"])
+        d1_8 = [r for r in res["rows"] if r["target"] is not None]
+        ok = sum(1 for r in d1_8 if r["aligned"] or not r["own"])
+        tot_orb += len(d1_8); tot_ok += ok; tot_moves += n_moves
+        per_track.append((p, res, n_moves, ok, len(d1_8)))
+
+    for p, res, n_moves, ok, n in per_track:
+        flag = "OK " if n_moves == 0 else f"{n_moves:2d} moves"
+        print(f"== {p.name:34} d1-d8 aligned {ok}/{n:<2}  {flag}")
+        for r in res["rows"]:
+            if not r["own"] and not r["ardour"] and not r["offgrid"]:
+                continue
+            own = ",".join(f"^{cc}" for cc, _ in r["own"]) or "-"
+            cols = ",".join(str(c) for c in r["cols"]) or "-"
+            mark = "==" if r["aligned"] else ("  " if r["target"] is None else "!=")
+            print(f"     d{r['orbit']:<2} {mark} cols[{cols:9}] own {own}")
+            if r["ardour"]:
+                ccs = ",".join(f"^{cc}@L{l}" for cc, l in r["ardour"])
+                print(f"          !! ARDOUR-OWNED CC in a track: {ccs}")
+            if r["offgrid"]:
+                print(f"          ?? off-grid CC: "
+                      f"{','.join(f'^{c}' for c in r['offgrid'])}")
+            if a.plan:
+                for cc, new, line in moves(r):
+                    print(f"          plan  ^{cc} -> ^{new}   (line {line})")
+        print()
+
+    print("=" * 66)
+    print(f" d1-d8 orbits already column-aligned: {tot_ok}/{tot_orb}")
+    print(f" total ^NN renumbers for full alignment: {tot_moves}")
+    print("=" * 66)
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())