#!/usr/bin/env python3
"""cut_lens3 — measure the crossfade as a SPAN, not a point.

Lineage. v1 (`cut_lens.py`) missed by a median 24 s. v2 (`cut_lens2.py`) got to
16.6 s and, more usefully, explained itself: every single error was NEGATIVE,
and the accurate ones were exactly the boundaries PLN described as short. v2's
docstring states the conclusion:

    THE LENS FINDS WHERE THE INCOMING TRACK FIRST APPEARS.
    THE EAR MARKS WHERE IT TAKES OVER.

Those two differ by the crossfade length, which is the thing that varies (2-41 s
on this set). So a point estimator is the wrong shape of answer. v3 reports the
span [onset, takeover] and lets the *fraction through the fade* be the tunable —
except it is not tuned, because of the normalisation below.

## Why the fraction is honest and not a fitted threshold

Per orbit we build the series

    d(t) = dist(win_t, ref_prev) - dist(win_t, ref_next)

which is negative while the window still sounds like the outgoing track and
positive once it sounds like the incoming one. v2 hunted a change-point in d.
The problem is that d's SCALE is different for every orbit and every boundary —
a kick that barely changes and a lead that changes completely produce the same
sign pattern with wildly different amplitudes, so no threshold on d can mean the
same thing twice.

The fix needs no data: evaluate d on the reference windows themselves.

    d(ref_prev) = 0 - sep = -sep        d(ref_next) = sep - 0 = +sep

where `sep` is the orbit's own prev-vs-next timbral distance. So

    u(t) = (d(t) + sep) / (2 * sep)     in [0, 1]

is calibrated by construction: u=0 is "identical to how this orbit sounded
before the switch", u=1 is "identical to after". u is a genuine fraction-through-
the-crossfade, per orbit, comparable across orbits and across boundaries. The
only real choice left is which fraction the ear calls the boundary — and that is
a question about PLN, not about the audio, so it is validated against his eight
ear-verified boundaries rather than assumed.

## Reading the output

`--truth` scores every candidate fraction against the ear boundaries. Selecting
a fraction on 8 points is weak evidence and is reported as such: what ships is
not one number but the SPAN — onset (u=0.15) and takeover (u=0.85) — as two
candidates PLN confirms by ear in the boundary webview. Seconds of listening per
cut instead of a full pass, which is the whole point.

    python3 cut_lens3.py --spec judge_specs/opal26.json \
        --truth judge_specs/opal26_boundaries_ear.json -v
    python3 cut_lens3.py --spec judge_specs/opal26.json --out cut_candidates.json

>>> RESULT, 2026-08-16: the normalisation works, the GATING broke coverage. <<<

    frac   n  median|e|  worst        (10m21 run, 12 orbits, 14 boundaries)
    0.15   3      48.2s  48.5   <- confirms u-low = onset, far before the ear
    0.30   3      16.4s  40.0
    0.50   2       4.7s   5.5
    0.70   2       3.7s   5.0   <- best
    0.85   2       6.9s  13.0

u=0.7 landing at 3.7 s median is a real improvement on v2's 16.6 s AND the
error finally changes sign (+1.3 mean, vs v2's every-error-negative) — which is
the span model behaving as predicted. **But n=2.** Six of fourteen boundaries
returned "no separating orbit", including #13 and #15, the two this run existed
to answer. Two points cannot choose between five fractions; do NOT quote 3.7 s
as an accuracy.

The gate is the bug, not the measure. `MIN_SEP` is applied to a single averaged
profile per reference window, and a 40 s average over a livecoded track washes
out exactly the orbits that do change — the longer and more varied the segment,
the more the mean of its timbre looks like everyone else's. Next attempt should
gate on per-window DISTRIBUTIONS (e.g. separation of the two reference windows'
frame-level profiles, or a t-like statistic) rather than distance between two
means, and shorten REF_MAX. Expect coverage, not accuracy, to be the win.

Usable output today: #11 clusters tight (3553.7-3557.2, all five fractions
within 3.5 s => a SHORT crossfade, so the nominal 3560.2 is ~4 s late) and #12
spreads wide (3662.8-3694.6 => a long fade, nominal 3681.6 sits mid-span).
Both are candidates for ear confirmation, not answers.

>>> RUN 2, same day — the distribution gate made coverage WORSE. STOP HERE. <<<

Framing the references, taking medoids instead of means, and gating on an
effect size (cross-side distance must beat within-side spread) was supposed to
buy coverage. It did the opposite:

    boundaries with any voter:  run 1  8/14   ->   run 2  4/14
    #13 Vague de CRIME, #15 REVOLUTION: no voters in EITHER run, nor in v2

The accuracy figures look spectacular and must not be believed: u=0.85 scores
median 0.4 s, worst 0.7 s — on n=2, and those two are #8 and #10, both of which
already had ear answers. Four surviving boundaries out of fourteen cannot
validate five candidate fractions. A metric computed on the cases that were
already easy is not evidence.

THE REAL RESULT IS THE THREE-VERSION PATTERN, NOT ANY ERROR NUMBER.
#13 and #15 have never produced a single voting orbit across v2, v3 run 1 and
v3 run 2 — three different gates, two different estimators. That is no longer
plausibly a tuning failure. It says those transitions have NO orbit whose
timbre differs materially across the switch: the outgoing and incoming tracks
share their sounds, so a timbral lens is structurally blind there, exactly as
activity-based detectors are blind to a crossfade. Loosening the gate a fourth
time can only admit noise, because there is no signal in this feature space.

Do not tune this again. Either change the FEATURE (harmonic/key change, or
per-sample onset identity from the score's sample banks, which do differ even
when the spectrum does not), or accept that #13/#15 are ear-only calls. The
cheap honest move is the latter: two timestamps, seconds of listening each.
"""
from __future__ import annotations

import argparse
import json
import sys
from pathlib import Path

import numpy as np

HERE = Path(__file__).parent
sys.path.insert(0, str(HERE))
from audio_lens import load_window, profile  # noqa: E402

SEARCH = 45.0     # +/- seconds around the nominal cut
WIN = 3.0         # profiling window
STEP = 0.5        # slide
REF_MAX = 24.0    # cap on a reference window (was 40 — too long to be one sound)
REF_STEP = 1.5    # frame hop inside a reference window
REF_CLEAR = 12.0  # keep references clear of the boundary being measured
MIN_SEP = 0.03    # medoid-to-medoid distance floor
MIN_EFFECT = 0.01  # ...and it must beat the orbit's own within-side spread
SUSTAIN = 3.0     # seconds a level must hold to count as reached
SR = 44100

FRACTIONS = [0.15, 0.30, 0.50, 0.70, 0.85]
ONSET_Q, TAKEOVER_Q = 0.15, 0.85


def mmss(t):
    return f"{int(t) // 60}:{int(t) % 60:02d}"


def timbral(a, b):
    """Distance in 'what kind of sound is this' — band shape + centroid octave."""
    if a is None or b is None:
        return None
    ba = np.array([a["bands"][k] for k in sorted(a["bands"])])
    bb = np.array([b["bands"][k] for k in sorted(b["bands"])])
    band = np.abs(ba - bb).sum() / 200.0
    ca, cb = max(a["centroid"], 1.0), max(b["centroid"], 1.0)
    cen = min(abs(np.log2(ca / cb)) / 4.0, 1.0)
    return float(0.65 * band + 0.35 * cen)


def make_m2s(keeps):
    """master time -> stem time, from the master's EXPLICIT edit list."""
    def f(t):
        acc = 0.0
        for a, b in keeps:
            if t < acc + (b - a):
                return a + (t - acc)
            acc += b - a
        return keeps[-1][1]
    return f


def ref_window(seg_start, seg_end, boundary, side):
    if side == "prev":
        hi = boundary - REF_CLEAR
        lo = max(seg_start + 2.0, hi - REF_MAX)
    else:
        lo = boundary + REF_CLEAR
        hi = min(seg_end - 2.0, lo + REF_MAX)
    return (lo, hi) if hi - lo >= 6.0 else None


def ref_frames(path, m2s, t0, t1):
    """Frame-level profiles across a reference window.

    v3's first run gated on ONE profile averaged over 40 s per side, and six of
    fourteen boundaries came back "no separating orbit" — a 40 s average over a
    livecoded track washes out precisely the orbits that change, so the two
    means converge and the orbit looks uninformative when it is the informative
    one. Framing keeps the distribution, which is what the gate below needs.
    """
    try:
        st = m2s(t0)
        sig = load_window(path, st, st + min(t1 - t0, REF_MAX))
    except Exception:
        return []
    n = int(WIN * SR)
    hop = int(REF_STEP * SR)
    out = []
    for a in range(0, max(0, sig.size - n + 1), hop):
        p = profile(sig[a:a + n])
        if p is not None and p["rms_db"] >= -55:
            out.append(p)
    return out


def medoid(profs):
    """The most typical frame — robust where a mean spectrum is not.

    A mean of band-percentages is a sound that never occurred; the medoid is an
    actual moment of the track, so `dist(medoid_prev, medoid_next)` compares two
    real timbres instead of two smeared ones.
    """
    if not profs:
        return None, None
    if len(profs) == 1:
        return profs[0], 0.0
    D = np.array([[timbral(a, b) for b in profs] for a in profs], dtype=float)
    i = int(np.argmin(D.sum(axis=1)))
    spread = float(np.median(np.delete(D[i], i)))   # how varied this side is
    return profs[i], spread


def reaches(u, grid, q):
    """First t where u >= q and STAYS there for SUSTAIN seconds.

    A crossfade is monotone-ish but noisy; a bare first-crossing fires on one
    stray window. Requiring the level to hold is what makes this a handover
    rather than a glitch, and it needs no threshold on the noise itself.
    """
    hold = int(round(SUSTAIN / STEP))
    ok = u >= q
    n = len(u)
    for i in range(n):
        if not ok[i]:
            continue
        j = min(n, i + hold)
        seg = ok[i:j]
        if seg.size and np.count_nonzero(seg) >= max(1, int(0.8 * seg.size)):
            return float(grid[i])
    return None


def orbit_series(path, m2s, grid, rp, rn, sep):
    """Normalised fraction-through-the-crossfade u(t) for one orbit, or None."""
    st0 = m2s(float(grid[0]))
    try:
        sig = load_window(path, st0, st0 + float(grid[-1] - grid[0]) + WIN)
    except Exception:
        return None, None
    n = int(WIN * SR)
    u = []
    for t in grid:
        a = int((t - grid[0]) * SR)
        seg = sig[a:a + n]
        p = profile(seg) if seg.size >= n // 2 else None
        if p is None or p["rms_db"] < -55:
            u.append(np.nan)              # silence carries no timbre
            continue
        d = timbral(p, rp) - timbral(p, rn)
        u.append((d + sep) / (2 * sep))   # calibrated by the references themselves
    u = np.array(u, dtype=float)
    if np.isnan(u).sum() > 0.6 * u.size:
        return None, None
    # fill gaps so a hold test is not broken by a silent bar
    idx = np.arange(u.size)
    good = ~np.isnan(u)
    if good.sum() < 8:
        return None, None
    u = np.interp(idx, idx[good], u[good])
    # light smoothing: 1.5 s, well under a 4-cycle fade
    k = max(1, int(round(1.5 / STEP)))
    u = np.convolve(u, np.ones(k) / k, mode="same")
    return u, sep


def estimate(stems, m2s, seg_prev, seg_next, verbose=False):
    boundary = seg_next["start"]
    lo = max(seg_prev["start"] + 3, boundary - SEARCH)
    hi = min(seg_next["end"] - 3, boundary + SEARCH)
    grid = np.arange(lo, hi - WIN, STEP)
    if grid.size < 10:
        return None

    wp = ref_window(seg_prev["start"], seg_prev["end"], boundary, "prev")
    wn = ref_window(seg_next["start"], seg_next["end"], boundary, "next")
    if not wp or not wn:
        return None

    per_frac = {q: [] for q in FRACTIONS}
    voters = 0
    for path in stems:
        fp, fn = ref_frames(path, m2s, *wp), ref_frames(path, m2s, *wn)
        rp, sp = medoid(fp)
        rn, sn = medoid(fn)
        if rp is None or rn is None:
            continue
        sep = timbral(rp, rn)
        # An orbit votes only if it sounds MORE different across the switch than
        # it does within either side — a livecoded track varies constantly, so a
        # bare distance floor admits noise and (v3 run 1) rejects real changes.
        effect = sep - 0.5 * (sp + sn)
        if sep < MIN_SEP or effect < MIN_EFFECT:
            continue
        u, sep = orbit_series(path, m2s, grid, rp, rn, sep)
        if u is None:
            continue
        voters += 1
        hits = {}
        for q in FRACTIONS:
            t = reaches(u, grid, q)
            if t is not None:
                per_frac[q].append(t)
                hits[q] = t
        if verbose:
            got = " ".join(f"{q:.2f}:{mmss(hits[q])}" for q in FRACTIONS if q in hits)
            print(f"      {path.name[-6:]:>6} sep={sep:.2f}  {got}")

    if voters < 3:
        return None
    out = {"voters": voters}
    for q in FRACTIONS:
        v = per_frac[q]
        out[q] = float(np.median(v)) if len(v) >= 3 else None
    return out


def main() -> int:
    ap = argparse.ArgumentParser(description=__doc__,
                                 formatter_class=argparse.RawDescriptionHelpFormatter)
    ap.add_argument("--spec", required=True)
    ap.add_argument("--truth")
    ap.add_argument("--out")
    ap.add_argument("--only", help="comma-separated track numbers")
    ap.add_argument("-v", "--verbose", action="store_true")
    a = ap.parse_args()

    spec = json.loads(Path(a.spec).read_text())
    segs = json.loads(Path(spec["segments"]).read_text())
    m2s = make_m2s(spec["keeps"])
    stems = sorted(Path(spec["stemsDir"]).glob(spec.get("stemsGlob", "*.wav")))
    only = {int(x) for x in a.only.split(",")} if a.only else None

    truth = {}
    if a.truth:
        td = json.loads(Path(a.truth).read_text())
        for k, v in td.get("verified", {}).items():
            if v.get("start") is not None:
                truth[int(k)] = float(v["start"])

    print(f"cut-lens3 — {len(stems)} orbits, {len(truth)} ear-verified boundaries")
    print(f"span = u{ONSET_Q:g} (onset) .. u{TAKEOVER_Q:g} (takeover)\n")
    hdr = f"{'#':>2} {'track':<26} {'nominal':>8} " + \
          " ".join(f"{'u'+format(q,'g'):>7}" for q in FRACTIONS) + \
          f" {'n':>3} {'truth':>8}"
    print(hdr)
    errs = {q: [] for q in FRACTIONS}
    rows = []
    for i in range(1, len(segs)):
        prev, nxt = segs[i - 1], segs[i]
        if only and nxt["track"] not in only:
            continue
        if a.verbose:
            print(f"\n  -- into #{nxt['track']} {nxt['title']}")
        est = estimate(stems, m2s, prev, nxt, a.verbose)
        line = f"{nxt['track']:>2} {nxt['title'][:26]:<26} {nxt['start']:>8.1f}"
        if est is None:
            print(line + "   (no separating orbit)")
            continue
        for q in FRACTIONS:
            line += f" {est[q]:>7.1f}" if est[q] is not None else f" {'—':>7}"
        line += f" {est['voters']:>3}"
        if nxt["track"] in truth:
            line += f" {truth[nxt['track']]:>8.1f}"
            for q in FRACTIONS:
                if est[q] is not None:
                    errs[q].append(est[q] - truth[nxt["track"]])
        print(line)
        rows.append({
            "track": nxt["track"], "title": nxt["title"],
            "nominal": nxt["start"],
            "onset": round(est[ONSET_Q], 2) if est[ONSET_Q] is not None else None,
            "takeover": round(est[TAKEOVER_Q], 2) if est[TAKEOVER_Q] is not None else None,
            "mid": round(est[0.50], 2) if est[0.50] is not None else None,
            "voters": est["voters"],
            "truth": truth.get(nxt["track"]),
        })

    if any(errs.values()):
        print(f"\nVALIDATION against the ear ({len(truth)} boundaries):")
        print(f"  {'frac':>6} {'n':>3} {'median|e|':>10} {'mean e':>8} {'worst':>7} {'<=2s':>6} {'<=5s':>6}")
        for q in FRACTIONS:
            e = np.array(errs[q])
            if not e.size:
                continue
            print(f"  {q:>6.2f} {e.size:>3} {np.median(np.abs(e)):>10.1f} "
                  f"{e.mean():>+8.1f} {np.abs(e).max():>7.1f} "
                  f"{(np.abs(e) <= 2).sum():>6} {(np.abs(e) <= 5).sum():>6}")
        best = min((q for q in FRACTIONS if errs[q]),
                   key=lambda q: np.median(np.abs(errs[q])))
        be = np.abs(np.array(errs[best]))
        print(f"\n  best fraction u={best:g}: median {np.median(be):.1f}s, worst {be.max():.1f}s")
        print("  NOTE: chosen over 5 candidates on 8 points — treat as a CANDIDATE")
        print("        generator for ear confirmation, not as ground truth.")

    if a.out:
        Path(a.out).write_text(json.dumps(rows, indent=2))
        print(f"\n✓ {a.out}")
    return 0


if __name__ == "__main__":
    sys.exit(main())
