Commit 128de62d by PLN (Algolia)

perf(lcxl): the LEDs lagged because we deduped on VALUE, and colour is a step function

PLN, watching the board for the first time on real hardware: "its slow to track if
i move all fders quickly i see the animations lagging a sec or two behind".

Not a slow device. Three compounding faults, each of which alone would have been
survivable:

1. THE BUG. The read loop repainted whenever a CC *value* changed. But colour is a
   STEP function of value -- six steps in value_ramp, seven in filter_colour. Sweeping
   one knob 0 -> 127 emits ~128 events and can change the board at most 6 times. We
   were asking the wire for ~20x the work that could possibly be visible.
2. Every send re-resolved the port from scratch: `aconnect -o` AND `amidi -l` AND then
   `aseqsend`. Three forks per LED, ~1.7 ms each, measured.
3. All of it ran INSIDE `for line in proc.stdout`, so a write in flight stopped us
   reading the next MIDI event. aseqdump's pipe QUEUES rather than drops, so nothing
   was lost -- everything just arrived later, and later, without bound. That
   unboundedness is why it read as "a second or two" rather than a constant delay.

The fix, in payoff order. A new `Painter` thread owns the wire; the reader only touches
the model and hands it colours, never blocking. The painter dedupes on the COLOUR the
board will show, coalesces so only the last colour per index within a frame reaches the
wire, and batches every dirty index into ONE SysEx (the Launchpad dialect takes
(index, colour) pairs, so a whole-surface repaint is a single write). Port resolution
is cached for 2 s and thrown away the instant a write fails -- evidence, not a timer --
which keeps the replug-recovery property that made it uncached in the first place.
Rate limiting is a floor on the GAP between writes, not a fixed tick, so an isolated
button press still goes out immediately.

Measured, not claimed. `--bench` replays PLN's own complaint (8 knobs + 8 faders swept
together, 500 events/s) against a transport modelled at its real cost, and runs the
legacy path beside the new one:

    legacy     744 wire msgs   wall 11.23s   overrun +8.23s
    coalesced   48 wire msgs   wall  3.00s   overrun +0.00s
    15.5x fewer messages; latency p50 7.9 ms, p99 20.2 ms

Overrun IS the visible lag -- it is how far behind his hands the board finishes. On the
real device the per-write cost also fell 7.70 ms -> 2.72 ms with ports cached, so the
total wire work is down roughly 44x.

Twelve regression tests, no hardware needed. Speed regresses silently -- nothing goes
red, it just gets slow again -- so the assertions are numeric: wire rate bounded by the
frame rate and not the input rate, reader never falls behind, p99 under 50 ms, a lone
press not delayed by the frame boundary. The last one is the one that matters: fast and
wrong beats nothing, so we decode every SysEx the coalescer emitted through the mock
surface and assert the board that LANDS is exactly what a full `build_frame`
recomputation would have produced.

Also: verbose logging now fires only when the board actually changes. Printing 400
lines a second of "CC77 = 63" was itself I/O in the hot loop, and told nobody anything.
parent 89d9f127
"""Regression tests for the LED paint pipeline's SPEED — and for its honesty.
The bug (2026-07-28): "its slow to track if i move all fders quickly i see the
animations lagging a sec or two behind". Not a slow device, a structural one — the
reader loop did its own blocking I/O, so it could not read the next MIDI event while a
write was in flight, and aseqdump's pipe QUEUES rather than drops. Every event was
faithfully rendered, just later and later.
Speed is the easiest thing to regress silently, because nothing goes red — it just gets
slow again. So these tests are numeric and they run without hardware: the transport is
modelled at its measured cost (~1.7 ms of fork+exec per subprocess) and the assertions
are on messages, latency and overrun.
The last test is the one that matters most. A coalescer that is fast but paints the
wrong board is worse than a slow correct one, so we assert that whatever the coalescer
folds away, the frame that LANDS is byte-for-byte the frame a full recomputation would
have produced — decoded through the same mock surface PLN's colour convention was
designed against.
"""
from __future__ import annotations
import importlib.util
import sys
import time
from pathlib import Path
import pytest
TOOLS = Path(__file__).resolve().parents[2]
def _load(name: str, path: Path):
spec = importlib.util.spec_from_file_location(name, path)
mod = importlib.util.module_from_spec(spec)
sys.modules[name] = mod
spec.loader.exec_module(mod)
return mod
leds = _load("lcxl_leds", TOOLS / "lcxl-leds.py")
mock = _load("mock_lcxl", TOOLS / "mock-lcxl.py")
class RecordingSender(leds.Sender):
"""Costs what the wire costs, and remembers every byte it was asked to write."""
def __init__(self, send_ms: float = 5.0):
super().__init__(dry_run=False, verbose=False)
self.cost = send_ms / 1000.0
self.messages: list[str] = []
def send(self, hexmsg: str) -> bool:
self.messages.append(hexmsg)
time.sleep(self.cost)
self.sends += 1
return True
def drive(events, rate=500.0, fps=60.0, send_ms=5.0, bindings=None):
"""Feed events through the REAL pipeline at a real arrival rate."""
bindings = bindings if bindings is not None else leds.convention_bindings()
s = RecordingSender(send_ms)
p = leds.Painter(s, fps=fps).start()
period = 1.0 / rate
t0 = time.monotonic()
for k, (cc, val) in enumerate(events):
hit = leds.resolve_paint(cc, val, bindings)
if hit:
p.set(*hit)
slack = t0 + (k + 1) * period - time.monotonic()
if slack > 0:
time.sleep(slack)
read_wall = time.monotonic() - t0
p.flush(timeout=10.0)
p.stop()
return s, p, read_wall
# ------------------------------------------------------------------ dedupe
def test_a_full_fader_sweep_costs_at_most_one_update_per_colour_step():
"""THE bug. The old loop repainted on every VALUE change; colour is a step
function of value, so a 128-event sweep asked for 128 writes and needed six."""
s = RecordingSender(send_ms=0.0)
p = leds.Painter(s, fps=0.0)
idx = leds.cc_to_index(13)
accepted = sum(p.set(idx, leds.control_colour(13, "rhythm", v, True))
for v in range(128))
distinct = len({leds.control_colour(13, "rhythm", v, True) for v in range(128)})
assert accepted <= distinct
assert p.deduped >= 128 - distinct
def test_dedupe_is_against_what_the_wire_last_saw_not_what_is_pending():
"""Setting a colour back to a still-unsent pending value must not queue a write."""
p = leds.Painter(RecordingSender(0.0), fps=0.0)
assert p.set(3, 60) is True
assert p.set(3, 60) is False # already pending
assert p.set(3, 28) is True # genuinely different
assert p.set(3, 60) is True # back again, but 28 is what's pending
# ------------------------------------------------------- coalescing + rate
@pytest.mark.parametrize("rate", [200.0, 500.0])
def test_wire_rate_is_bounded_by_the_frame_rate_not_the_input_rate(rate):
"""The guarantee: however fast his hands are, the wire is bounded."""
fps, secs = 60.0, 1.0
events = list(leds.sweep_events(secs, rate))
s, p, _ = drive(events, rate=rate, fps=fps)
assert len(s.messages) <= fps * secs * 1.5, (
f"{len(s.messages)} messages for {len(events)} events at {rate}/s")
def test_the_reader_never_falls_behind_the_input():
"""Overrun IS the visible lag. The legacy path overran 3 s of input by +8.2 s."""
secs, rate = 1.0, 500.0
_, _, read_wall = drive(list(leds.sweep_events(secs, rate)), rate=rate)
assert read_wall < secs * 1.25, f"reader took {read_wall:.2f}s for {secs}s of input"
def test_p99_latency_stays_perceptually_instant_under_a_full_surface_sweep():
secs, rate = 1.0, 500.0
_, p, _ = drive(list(leds.sweep_events(secs, rate)), rate=rate)
st = p.stats()
assert st["p99_ms"] < 50.0, st
assert st["max_ms"] < 150.0, st
def test_an_isolated_press_is_not_delayed_by_the_frame_rate():
"""Rate limiting is a floor on the GAP between writes, not a fixed tick — a lone
button press must not wait for the next frame boundary."""
s = RecordingSender(send_ms=0.0)
p = leds.Painter(s, fps=30.0).start()
try:
t0 = time.monotonic()
p.set(leds.cc_to_index(41), leds.HUE["green"]["full"])
p.flush(timeout=1.0)
assert time.monotonic() - t0 < 0.020
finally:
p.stop()
def test_a_burst_of_forty_leds_leaves_as_one_sysex_not_forty():
"""Batching: the Launchpad dialect takes (index, colour) pairs, so a whole-surface
repaint is one write."""
s = RecordingSender(send_ms=0.0)
p = leds.Painter(s, fps=0.0)
p.set_frame([leds.HUE["green"]["full"]] * leds.LIT_INDICES)
pairs, _ = p._drain()
msg = leds.sysex_pairs(pairs)
assert len(mock.parse_frames(msg)) == leds.LIT_INDICES
# -------------------------------------------------------- the invariant
def test_the_board_that_lands_equals_a_full_recomputation():
"""Correctness under coalescing, checked through the mock surface.
Fast and wrong is worse than slow and right. Whatever the coalescer drops or folds,
decoding every SysEx it emitted must reproduce exactly `build_frame` over the final
model — the same pure function the colour convention is defined by.
"""
bindings = leds.convention_bindings()
s = RecordingSender(send_ms=1.0)
p = leds.Painter(s, fps=60.0).start()
p.set_frame(leds.build_frame(bindings, {}, set()))
values: dict[int, int] = {}
for cc, val in leds.sweep_events(0.4, 500.0):
values[cc] = val
hit = leds.resolve_paint(cc, val, bindings)
if hit:
p.set(*hit)
p.flush(timeout=5.0)
p.stop()
landed = mock.parse_frames("\n".join(s.messages))
expected = leds.build_frame(bindings, values, set(values))
assert landed == {i: c for i, c in enumerate(expected)}
def test_faders_never_receive_a_write():
"""CC 77-84 are Ardour's track gains and have no LEDs. They generate load and
nothing else — a write there would be both meaningless and dangerous."""
for cc in range(77, 85):
assert leds.cc_to_index(cc) is None
assert leds.resolve_paint(cc, 100, leds.convention_bindings()) is None
# --------------------------------------------------------- port caching
def test_port_lookup_is_cached_then_dropped_on_failure(monkeypatch):
"""The cache buys 3 forks -> 1 per write, but must not survive the one event that
makes it wrong. A binding resolved once and silently invalidated by a device event
is this rig's most expensive recurring bug class."""
calls = {"n": 0}
def fake():
calls["n"] += 1
return "24:0"
leds.invalidate_ports()
monkeypatch.setattr(leds, "_find_hw_port", fake)
assert leds.find_hw_port() == "24:0"
assert leds.find_hw_port() == "24:0"
assert calls["n"] == 1, "second lookup should have been served from cache"
leds.Sender(verbose=False)._fail("simulated write failure")
assert leds.find_hw_port() == "24:0"
assert calls["n"] == 2, "a failed write must force re-resolution"
leds.invalidate_ports()
def test_port_cache_ttl_is_short_enough_to_be_invisible():
assert leds.PORT_TTL <= 5.0
...@@ -71,10 +71,12 @@ Usage ...@@ -71,10 +71,12 @@ Usage
Port resolution Port resolution
--------------- ---------------
Resolved **at every call** by matching the name "Launch Control XL" in `amidi -l`. Resolved by matching the name "Launch Control XL" in `amidi -l` / `aconnect`, behind
Never cached across a failure. The port has already moved (20:0 -> 24:0) on a replug, a 2-second cache that is **dropped on any send failure**. The port has already moved
and a binding resolved once at boot then silently invalidated by a device event is (20:0 -> 24:0) on a replug, and a binding resolved once at boot then silently
this rig's single most expensive recurring bug class. invalidated by a device event is this rig's single most expensive recurring bug class
-- so the cache is deliberately shorter than a human notices and cannot survive the
event that would make it wrong. See `invalidate_ports`.
If sends start failing while input still arrives, that is a USB OUT endpoint stall If sends start failing while input still arrives, that is a USB OUT endpoint stall
(urb -32) and **only a replug fixes it** — this tool says so rather than looping. (urb -32) and **only a replug fixes it** — this tool says so rather than looping.
...@@ -378,9 +380,37 @@ def sysex_frame(frame: list[int]) -> str: ...@@ -378,9 +380,37 @@ def sysex_frame(frame: list[int]) -> str:
# --------------------------------------------------------------------------- # # --------------------------------------------------------------------------- #
NAME_RE = re.compile(r"launch\s*control\s*xl", re.I) NAME_RE = re.compile(r"launch\s*control\s*xl", re.I)
# Port lookups are `aconnect -o` / `amidi -l` subprocesses, ~1.7 ms each measured on
# this machine. The original code ran BOTH on every single LED write, so one lit knob
# cost three forks (~5 ms) -- and at 400 CC/s from eight moving faders that is 2000 ms
# of work per second of input. The queue could only grow; PLN saw it as "the animation
# lags a second or two behind". Cache the answer for a beat, and throw the cache away
# the instant a write fails, which is exactly when the answer might have changed.
PORT_TTL = 2.0
_PORT_CACHE: dict[str, tuple[float, str | None]] = {}
_PORT_LOCK = threading.Lock()
def find_hw_port() -> str | None:
"""The `amidi` raw port (hw:C,D,S), resolved BY NAME, every call. Never cached. def invalidate_ports() -> None:
"""Forget every resolved port. Called on send failure and before `--ports`."""
with _PORT_LOCK:
_PORT_CACHE.clear()
def _cached_port(key: str, fn) -> str | None:
now = time.monotonic()
with _PORT_LOCK:
hit = _PORT_CACHE.get(key)
if hit is not None and now - hit[0] < PORT_TTL:
return hit[1]
val = fn()
with _PORT_LOCK:
_PORT_CACHE[key] = (time.monotonic(), val)
return val
def _find_hw_port() -> str | None:
"""The `amidi` raw port (hw:C,D,S), resolved BY NAME (see `find_hw_port`).
Excludes the HUI port: the LCXL exposes two, and only the first speaks the Excludes the HUI port: the LCXL exposes two, and only the first speaks the
Launchpad SysEx dialect. Launchpad SysEx dialect.
...@@ -401,7 +431,11 @@ def find_hw_port() -> str | None: ...@@ -401,7 +431,11 @@ def find_hw_port() -> str | None:
return None return None
def find_seq_port(direction: str = "-i") -> str | None: def find_hw_port() -> str | None:
return _cached_port("hw", _find_hw_port)
def _find_seq_port(direction: str = "-i") -> str | None:
"""The ALSA-sequencer client:port for the LCXL. """The ALSA-sequencer client:port for the LCXL.
`-i` = readable (source) side, for `aseqdump` — a READ-ONLY subscription that `-i` = readable (source) side, for `aseqdump` — a READ-ONLY subscription that
...@@ -428,6 +462,10 @@ def find_seq_port(direction: str = "-i") -> str | None: ...@@ -428,6 +462,10 @@ def find_seq_port(direction: str = "-i") -> str | None:
return None return None
def find_seq_port(direction: str = "-i") -> str | None:
return _cached_port(f"seq{direction}", lambda: _find_seq_port(direction))
class Sender: class Sender:
"""SysEx-only writer with stall detection. """SysEx-only writer with stall detection.
...@@ -460,8 +498,10 @@ class Sender: ...@@ -460,8 +498,10 @@ class Sender:
self.sends += 1 self.sends += 1
return True return True
# Both transports re-resolve the port on EVERY write. A replug already moved # Port resolution is cached for PORT_TTL and dropped on failure (see
# this device once (20:0 -> 24:0); a cached binding would go silently dead. # `invalidate_ports`), so a replug -- which already moved this device once,
# 20:0 -> 24:0 -- is recovered from within one write, while the steady state
# costs one fork instead of three.
attempts: list[tuple[str, list[str]]] = [] attempts: list[tuple[str, list[str]]] = []
seq = find_seq_port("-o") seq = find_seq_port("-o")
if seq and shutil.which("aseqsend"): if seq and shutil.which("aseqsend"):
...@@ -492,6 +532,9 @@ class Sender: ...@@ -492,6 +532,9 @@ class Sender:
return False return False
def _fail(self, why: str) -> None: def _fail(self, why: str) -> None:
# A failed write is the one moment we KNOW the cached port may be stale, so
# this is where the cache dies -- not on a timer, on evidence.
invalidate_ports()
self.fails += 1 self.fails += 1
if self.verbose: if self.verbose:
print(f"lcxl-leds: send FAILED — {why}", file=sys.stderr) print(f"lcxl-leds: send FAILED — {why}", file=sys.stderr)
...@@ -504,6 +547,161 @@ class Sender: ...@@ -504,6 +547,161 @@ class Sender:
" reference_lcxl_led_stall memory.", file=sys.stderr) " reference_lcxl_led_stall memory.", file=sys.stderr)
class Painter:
"""A coalescing LED writer. The MIDI reader hands it colours and never blocks.
Why (2026-07-28): PLN moved several faders at once and watched the board catch up
a second or two later. The cause was structural, not a slow device -- the reader
loop did its own I/O, so `for line in proc.stdout` could not advance while a write
was in flight. aseqdump's pipe filled, and because a pipe QUEUES rather than drops,
every event was faithfully rendered... late. Latency grew without bound for as long
as the input rate exceeded the write rate.
Three compounding reductions, in payoff order:
1. DEDUPE ON COLOUR, not on value. This is the big one and it was the actual bug:
the old loop repainted whenever the CC *value* changed, but colour is a step
function of value (six steps in `value_ramp`, seven in `filter_colour`). Sweeping
one fader 0 -> 127 emits ~128 events and needs at most 6 wire messages. ~20x.
2. COALESCE. Only the LAST colour per index within a frame reaches the wire, so a
burst of a thousand events costs one message per index per frame, not a thousand.
3. BATCH. All dirty indices go out in ONE SysEx, because the Launchpad dialect takes
(index, colour) pairs -- so a whole-surface repaint is one write, not forty.
The rate limit is a floor on the interval BETWEEN writes, not a fixed tick: an
isolated button press goes out immediately (no added latency), and only sustained
input gets batched. Bounded wire rate, bounded latency, no polling when idle.
Correctness invariant: the model is still the truth and `control_colour` is still a
pure function of it, so whatever the coalescer folds away, the frame that lands is
the frame a full recomputation would have produced. `test_lcxl_latency` asserts
exactly that.
"""
def __init__(self, sender: Sender, fps: float = 60.0, track_latency: bool = True):
self.sender = sender
self.min_interval = (1.0 / fps) if fps > 0 else 0.0
self._dirty: dict[int, int] = {}
self._stamp: dict[int, float] = {}
self._sent: dict[int, int] = {}
self._cv = threading.Condition()
self._stop = False
self._thread: threading.Thread | None = None
self.track_latency = track_latency
self.latencies: list[float] = []
self.frames = 0 # wire messages actually sent
self.updates = 0 # index-updates that reached the wire
self.deduped = 0 # dropped: the colour was already correct
self.coalesced = 0 # folded into a pending frame
# -- producer side (called from the reader thread; must never do I/O) --
def set(self, idx: int, colour: int) -> bool:
"""Queue one LED. Returns True if this actually changes the board."""
with self._cv:
pending = self._dirty.get(idx, self._sent.get(idx))
if pending == colour:
self.deduped += 1
return False
if idx in self._dirty:
self.coalesced += 1
elif self.track_latency:
self._stamp[idx] = time.monotonic()
self._dirty[idx] = colour
self._cv.notify()
return True
def set_frame(self, frame: list[int]) -> None:
"""Queue a full re-assert. Unconditional: the point is to overrule the device."""
with self._cv:
now = time.monotonic()
for i, c in enumerate(frame):
if i not in self._dirty and self.track_latency:
self._stamp[i] = now
self._dirty[i] = c
self._cv.notify()
# -- consumer side --
def _drain(self) -> tuple[list[tuple[int, int]], list[float]]:
with self._cv:
pairs = sorted(self._dirty.items())
stamps = [self._stamp.pop(i, None) for i, _ in pairs]
self._dirty.clear()
self._sent.update(pairs)
now = time.monotonic()
return pairs, [now - s for s in stamps if s is not None]
def _run(self) -> None:
next_ok = 0.0
while True:
with self._cv:
while not self._dirty and not self._stop:
self._cv.wait(0.25)
if self._stop and not self._dirty:
return
gap = next_ok - time.monotonic()
if gap > 0:
time.sleep(gap) # the coalescing window: late arrivals join us
pairs, lat = self._drain()
if not pairs:
continue
self.frames += 1
self.updates += len(pairs)
if self.track_latency:
self.latencies.extend(lat)
self.sender.send(sysex_pairs(pairs))
next_ok = time.monotonic() + self.min_interval
def start(self) -> Painter:
self._thread = threading.Thread(target=self._run, daemon=True)
self._thread.start()
return self
def stop(self, timeout: float = 1.0) -> None:
with self._cv:
self._stop = True
self._cv.notify()
if self._thread:
self._thread.join(timeout)
def flush(self, timeout: float = 2.0) -> None:
"""Block until the wire has caught up with the model (tests, shutdown)."""
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
with self._cv:
if not self._dirty:
return
time.sleep(0.005)
def stats(self) -> dict[str, float]:
lat = sorted(self.latencies)
def pct(p: float) -> float:
return lat[min(len(lat) - 1, int(p * len(lat)))] * 1000 if lat else 0.0
return {"frames": self.frames, "updates": self.updates,
"deduped": self.deduped, "coalesced": self.coalesced,
"p50_ms": pct(0.50), "p99_ms": pct(0.99),
"max_ms": (lat[-1] * 1000) if lat else 0.0}
def resolve_paint(cc: int, val: int, bindings: dict[int, str],
ack_unbound: bool = False) -> tuple[int, int] | None:
"""(led index, colour) for one incoming control event, or None if not a lit index.
Pulled out of the read loop so the bench and the tests exercise the SAME function
the live daemon does -- a benchmark of a re-implementation measures nothing.
"""
idx = cc_to_index(cc)
if idx is None:
return None
role = bindings.get(cc)
if role is None:
# Unbound on this track. Default is to STAY DARK: "dark means this control does
# nothing here" is the strongest signal on the board, and lighting it on touch
# would undermine the one thing PLN reads at a glance. The touch is still
# recorded in the model, so --ack-unbound can surface it when debugging.
return idx, (HUE["red"]["dim"] if (ack_unbound and val) else OFF)
return idx, control_colour(cc, role, val, True)
# --------------------------------------------------------------------------- # # --------------------------------------------------------------------------- #
# commands # commands
# --------------------------------------------------------------------------- # # --------------------------------------------------------------------------- #
...@@ -612,7 +810,8 @@ ASEQ_NOTE = re.compile(r"Note (on|off)\s+(\d+),\s*note\s+(\d+),\s*velocity\s+(\d ...@@ -612,7 +810,8 @@ ASEQ_NOTE = re.compile(r"Note (on|off)\s+(\d+),\s*note\s+(\d+),\s*velocity\s+(\d
def cmd_watch(s: Sender, track: str | None, reassert: float = 30.0, def cmd_watch(s: Sender, track: str | None, reassert: float = 30.0,
verbose: bool = True, ack_unbound: bool = False) -> int: verbose: bool = True, ack_unbound: bool = False,
fps: float = 60.0) -> int:
"""Own the LED state; repaint the touched control from the model on every change. """Own the LED state; repaint the touched control from the model on every change.
Persistence is not a feature bolted on — it is the data model. `values` is the Persistence is not a feature bolted on — it is the data model. `values` is the
...@@ -632,8 +831,11 @@ def cmd_watch(s: Sender, track: str | None, reassert: float = 30.0, ...@@ -632,8 +831,11 @@ def cmd_watch(s: Sender, track: str | None, reassert: float = 30.0,
values: dict[int, int] = {} values: dict[int, int] = {}
touched: set[int] = set() touched: set[int] = set()
# Paint immediately, so the board is never dark while we wait for a first touch. # ONE thread owns the wire. The reader below only ever touches the model and hands
s.send(sysex_frame(build_frame(bindings, values, touched))) # colours to the painter, so a slow write can no longer stall the read of the next
# MIDI event -- which is what made the board lag behind PLN's hands (#71).
painter = Painter(s, fps=fps).start()
painter.set_frame(build_frame(bindings, values, touched))
if verbose: if verbose:
print("lcxl-leds --watch: initial frame painted; listening (Ctrl-C to stop)") print("lcxl-leds --watch: initial frame painted; listening (Ctrl-C to stop)")
...@@ -645,17 +847,12 @@ def cmd_watch(s: Sender, track: str | None, reassert: float = 30.0, ...@@ -645,17 +847,12 @@ def cmd_watch(s: Sender, track: str | None, reassert: float = 30.0,
# long silences that make up most of a set (exactly when a device hiccup would go # long silences that make up most of a set (exactly when a device hiccup would go
# unnoticed) it would never run at all. A timer thread makes persistence hold # unnoticed) it would never run at all. A timer thread makes persistence hold
# while nobody is touching anything, which is the whole point of "persistent post # while nobody is touching anything, which is the whole point of "persistent post
# touches". The lock keeps a full repaint from interleaving with a single-LED one. # touches". It queues through the painter like everything else, so a full repaint
send_lock = threading.Lock() # can never interleave with a single-LED one.
def locked_send(msg: str) -> None:
with send_lock:
s.send(msg)
def reassert_loop() -> None: def reassert_loop() -> None:
while reassert: while reassert:
time.sleep(reassert) time.sleep(reassert)
locked_send(sysex_frame(build_frame(bindings, dict(values), set(touched)))) painter.set_frame(build_frame(bindings, dict(values), set(touched)))
if reassert: if reassert:
threading.Thread(target=reassert_loop, daemon=True).start() threading.Thread(target=reassert_loop, daemon=True).start()
...@@ -697,33 +894,28 @@ def cmd_watch(s: Sender, track: str | None, reassert: float = 30.0, ...@@ -697,33 +894,28 @@ def cmd_watch(s: Sender, track: str | None, reassert: float = 30.0,
val = 0 if n.group(1) == "off" else int(n.group(4)) val = 0 if n.group(1) == "off" else int(n.group(4))
if cc is not None and val is not None: if cc is not None and val is not None:
events += 1 events += 1
idx = cc_to_index(cc)
changed = values.get(cc) != val
values[cc] = val values[cc] = val
touched.add(cc) touched.add(cc)
if idx is not None and changed: hit = resolve_paint(cc, val, bindings, ack_unbound)
role = bindings.get(cc) # Print only when the BOARD changes. Logging every event was itself
if role is None: # a per-event stdout write in the hot loop, and 400 CC/s of "CC77 =
# Unbound on this track. Default is to STAY DARK: "dark # 63" tells nobody anything.
# means this control does nothing here" is the strongest if hit and painter.set(*hit) and verbose:
# signal on the board, and lighting it on touch would
# undermine the one thing PLN reads at a glance. The
# touch itself is still recorded in the model, so
# --ack-unbound can surface it when debugging.
colour = (HUE["red"]["dim"] if (ack_unbound and val)
else OFF)
else:
colour = control_colour(cc, role, val, True)
locked_send(sysex_pairs([(idx, colour)]))
if verbose:
row, lane = decode_cc(cc) row, lane = decode_cc(cc)
print(f" CC{cc:<3d} {row}{lane} = {val:<3d} " print(f" CC{cc:<3d} {row}{lane} = {val:<3d} "
f"-> idx {idx:02X} colour {colour}" f"-> idx {hit[0]:02X} colour {hit[1]}"
f"{'' if role else ' [unbound]'}") f"{'' if cc in bindings else ' [unbound]'}")
except KeyboardInterrupt: except KeyboardInterrupt:
proc.terminate() proc.terminate()
painter.flush()
painter.stop()
st = painter.stats()
print(f"\nlcxl-leds --watch: stopped after {events} events, " print(f"\nlcxl-leds --watch: stopped after {events} events, "
f"{s.sends} sends, {len(touched)} controls touched") f"{s.sends} sends, {len(touched)} controls touched")
print(f" coalescer: {int(st['frames'])} wire frames, "
f"{int(st['deduped'])} no-op events dropped, "
f"{int(st['coalesced'])} folded; "
f"latency p50 {st['p50_ms']:.1f} ms p99 {st['p99_ms']:.1f} ms")
return 0 return 0
except Exception as e: except Exception as e:
print(f"lcxl-leds --watch: read loop error: {e}", file=sys.stderr) print(f"lcxl-leds --watch: read loop error: {e}", file=sys.stderr)
...@@ -744,6 +936,114 @@ def cmd_watch(s: Sender, track: str | None, reassert: float = 30.0, ...@@ -744,6 +936,114 @@ def cmd_watch(s: Sender, track: str | None, reassert: float = 30.0,
# --------------------------------------------------------------------------- # # --------------------------------------------------------------------------- #
# --bench : prove the speed, don't claim it
# --------------------------------------------------------------------------- #
class BenchSender(Sender):
"""A Sender that writes nothing but costs what a real write costs.
`aseqsend` measured ~1.7 ms of fork+exec on this machine and the old code paid it
three times per LED (aconnect -o, amidi -l, aseqsend). Modelling that cost is the
whole point: a bench against an instant sender would show a beautiful number and
predict nothing about the rig.
"""
def __init__(self, send_ms: float = 5.0):
super().__init__(dry_run=False, verbose=False)
self.cost = send_ms / 1000.0
self.messages = 0
self.pairs = 0
def send(self, hexmsg: str) -> bool:
self.messages += 1
self.pairs += max(0, (len(hexmsg.split()) - 8) // 2)
time.sleep(self.cost)
self.sends += 1
return True
def sweep_events(seconds: float, rate: float):
"""The worst realistic case: a hand dragging across the whole surface at once.
Eight row-A knobs (CC 13-20, which HAVE LEDs) and the eight faders (CC 77-84, which
do NOT) swept together, full travel. The faders matter even though they light
nothing: their events still have to be read and parsed, and half the load of PLN's
"i move all fders quickly" is exactly that. Nothing is ever WRITTEN to 77-84 --
those are Ardour's track gains and this tool only ever listens.
"""
ccs = [13 + i for i in range(8)] + [77 + i for i in range(8)]
for k in range(int(seconds * rate)):
lane = k % len(ccs)
phase = (k // len(ccs)) * 4 + lane * 16
yield ccs[lane], abs((phase % 254) - 127)
def cmd_bench(seconds: float = 3.0, rate: float = 500.0, fps: float = 60.0,
send_ms: float = 5.0) -> int:
bindings = convention_bindings()
events = list(sweep_events(seconds, rate))
period = 1.0 / rate
def run(coalesced: bool) -> tuple[BenchSender, dict, float]:
# Legacy paid the write cost THREE times per LED: `aconnect -o` and `amidi -l`
# ran inside every send() to re-resolve the port, then aseqsend actually wrote.
s = BenchSender(send_ms if coalesced else send_ms * 3)
p = Painter(s, fps=fps) if coalesced else None
if p:
p.start()
last_val: dict[int, int] = {}
t0 = time.monotonic()
for k, (cc, val) in enumerate(events):
hit = resolve_paint(cc, val, bindings)
if hit:
if p:
p.set(*hit)
elif last_val.get(cc) != val:
# The OLD behaviour, faithfully: dedupe on the CC *value*, then one
# blocking write per event, inline in the read loop. Colour is a
# step function of value, so this repainted a knob ~20x more often
# than the board could possibly change. That was the bug.
last_val[cc] = val
s.send(sysex_pairs([hit]))
# Real MIDI arrives on a wire at a fixed rate. If we fall behind, the
# events queue -- they are NOT dropped, which is exactly why the old
# version lagged instead of skipping.
due = t0 + (k + 1) * period
slack = due - time.monotonic()
if slack > 0:
time.sleep(slack)
drain = time.monotonic()
if p:
p.flush(timeout=10.0)
p.stop()
wall = time.monotonic() - t0
st = p.stats() if p else {"p50_ms": 0.0, "p99_ms": 0.0, "max_ms": 0.0}
st["drain_s"] = time.monotonic() - drain
return s, st, wall
print(f"lcxl-leds --bench: {len(events)} events over {seconds:.0f}s "
f"({rate:.0f}/s, 8 knobs + 8 faders swept together), "
f"write cost {send_ms:.1f} ms (legacy paid it 3x -- port re-resolution), "
f"coalescer {fps:.0f} fps\n")
rows = []
for label, coalesced in (("legacy (write per LED)", False), ("coalesced", True)):
s, st, wall = run(coalesced)
overrun = wall - seconds
rows.append((label, s.messages, s.pairs, wall, overrun, st))
print(f" {label:<24s} {s.messages:6d} wire msgs {s.pairs:6d} LED updates "
f"wall {wall:6.2f}s overrun {overrun:+6.2f}s "
f"drain {st['drain_s']:5.2f}s")
if coalesced:
print(f" {'':<24s} latency p50 {st['p50_ms']:.1f} ms "
f"p99 {st['p99_ms']:.1f} ms max {st['max_ms']:.1f} ms")
(_, m0, _, _, o0, _), (_, m1, _, _, o1, _) = (rows[0][:6], rows[1][:6])
print(f"\n wire traffic {m0} -> {m1} ({m0 / max(1, m1):.1f}x fewer messages)")
print(f" overrun {o0:+.2f}s -> {o1:+.2f}s "
f"(overrun IS the visible lag: the board finishes that far behind the hands)")
return 0
# --------------------------------------------------------------------------- #
def main() -> int: def main() -> int:
ap = argparse.ArgumentParser( ap = argparse.ArgumentParser(
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
...@@ -756,16 +1056,30 @@ def main() -> int: ...@@ -756,16 +1056,30 @@ def main() -> int:
g.add_argument("--all", metavar="COLOUR", help="paint everything one colour") g.add_argument("--all", metavar="COLOUR", help="paint everything one colour")
g.add_argument("--off", action="store_true", help="all LEDs off") g.add_argument("--off", action="store_true", help="all LEDs off")
g.add_argument("--ports", action="store_true", help="show resolved ports and exit") g.add_argument("--ports", action="store_true", help="show resolved ports and exit")
g.add_argument("--bench", action="store_true",
help="measure the paint pipeline against the legacy one (no hardware)")
ap.add_argument("--dry-run", action="store_true", help="print hex, send nothing") ap.add_argument("--dry-run", action="store_true", help="print hex, send nothing")
ap.add_argument("--ack-unbound", action="store_true", ap.add_argument("--ack-unbound", action="store_true",
help="--watch: faintly light controls this track does NOT bind " help="--watch: faintly light controls this track does NOT bind "
"when touched (off by default — dark is the signal)") "when touched (off by default — dark is the signal)")
ap.add_argument("--reassert", type=float, default=30.0, ap.add_argument("--reassert", type=float, default=30.0,
help="--watch: full re-assert interval in seconds (0 = off)") help="--watch: full re-assert interval in seconds (0 = off)")
ap.add_argument("--fps", type=float, default=60.0,
help="--watch: max LED frames per second (0 = no rate limit)")
ap.add_argument("--bench-rate", type=float, default=500.0,
help="--bench: incoming MIDI events per second")
ap.add_argument("--bench-seconds", type=float, default=3.0)
ap.add_argument("--bench-send-ms", type=float, default=5.0,
help="--bench: modelled cost of one wire write")
ap.add_argument("-q", "--quiet", action="store_true") ap.add_argument("-q", "--quiet", action="store_true")
args = ap.parse_args() args = ap.parse_args()
if args.bench:
return cmd_bench(seconds=args.bench_seconds, rate=args.bench_rate,
fps=args.fps, send_ms=args.bench_send_ms)
if args.ports: if args.ports:
invalidate_ports()
print(f"aseqsend (LED out, primary): {find_seq_port('-o') or 'NOT FOUND'}") print(f"aseqsend (LED out, primary): {find_seq_port('-o') or 'NOT FOUND'}")
print(f"amidi (LED out, fallback): {find_hw_port() or 'NOT FOUND'}") print(f"amidi (LED out, fallback): {find_hw_port() or 'NOT FOUND'}")
print(f"aseqdump (CC in, read-only) : {find_seq_port('-i') or 'NOT FOUND'}") print(f"aseqdump (CC in, read-only) : {find_seq_port('-i') or 'NOT FOUND'}")
...@@ -783,7 +1097,8 @@ def main() -> int: ...@@ -783,7 +1097,8 @@ def main() -> int:
return cmd_map(s, args.map or None, quiet=args.quiet) return cmd_map(s, args.map or None, quiet=args.quiet)
if args.watch is not None: if args.watch is not None:
return cmd_watch(s, args.watch or None, reassert=args.reassert, return cmd_watch(s, args.watch or None, reassert=args.reassert,
verbose=not args.quiet, ack_unbound=args.ack_unbound) verbose=not args.quiet, ack_unbound=args.ack_unbound,
fps=args.fps)
return 2 return 2
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment