- 18 Jul, 2026 1 commit
-
-
Records of the editor-lag deep-dive: local root-cause analysis (renderer main-thread saturation, console DOM growth, three marker leaks, .line transition churn, Electron 12 ceiling, thermal state) and the companion web-research report on the 2026 TidalCycles editor ecosystem (uzu/Codeberg migration, upstream issue #229, vim/VSCode/browser alternatives, no prior art for Pulsar forks). Fixes landed in the pulsar-tidalcycles fork, branch perf/event-highlighter, commit 6e9f374.
PLN (Algolia) authored
-
- 11 Jul, 2026 8 commits
-
-
Two refinements from measuring the first rotated re-cuts of the real kits: 1. Junction in the pre-attack pocket. The rotation point was zc-snapped ±symmetrically around the downbeat, which could land the loop wrap mid-transient (rotated hammer drums read seam 0.50 / click_db −0.03: physically continuous, but the junction sat in busy material and a re-trigger clipped into the groove). _pocket_zc() searches [−25 ms, +2 ms] around the target and picks the zero crossing with the smallest local 2 ms RMS — the natural cut point in the quiet dip before the hit. Re-trigger catches the full attack; hammer drums grade recovered A 0.808 → S 0.988, bass 0.861 → S 0.916. 2. Exports ship DC-free (B2), mirroring vox._finish_clip (4a74213c): the demucs stems' ~1e-4 DC bias was flowing into every exported loop (all three OLD superfreak kit files flag dc-offset on re-grade today). Mean-subtract after mono-sum, before write; shifts the zc-snapped edges by ≤1e-4 — inaudible, still effectively zero. 86 tests green.
PLN (Algolia) authored -
Refinement caught before re-cutting the real kits: superfreak drums and bass ship from the SAME window (107.21s, 4 bars). If each stem rotated to its own strongest attack, their relative groove could shift by whole beats — bass's phrase-start is not necessarily on the drums' 1, and the kit is played together (loopAt on a common cycle), so relative phase between kit loops is musical content. export_take now derives the downbeat slot ONCE per shared time window from its most rhythmic stem (drums > bass > other > vocals priority) and applies that same slot to every stem of the window (_rotate_to_downbeat force_slot; per-stem roll still zc-snaps within ±10 ms on the stem's own mono ref). Stems at different windows keep independent rotation. Stem reads are cached (each stem was loaded twice otherwise). Test: two-stem synthetic window, drums downbeat at slot 3, other's own loudest attack at slot 6 — drums' marker must land at 0 and other's at 3 (= 6−3, following drums), not 0. 86 passed.
PLN (Algolia) authored -
PLN's kit audition (logged 6cfc7af0): loops felt 'timed ok but cut BCDA/DABC — not at a good start'. Measured and confirmed — the PLP beat grid has no downbeat anchor, so candidate windows start on an arbitrary beat: superfreak other_skank's strongest onset sat on slot 2/8 (its first 30 ms −23 dB below its own peak 30 ms), bass_deep's on slot 4/16, drums_dub's on slot 1/16. Bar-exact and tempo-locked, but musically rotated. The key insight that makes the fix free: for a verified clean-seam, bar-exact loop, ROTATION IS SAFE BY CONSTRUCTION. Looped playback of a rotated loop is the same audio cycle — the old wrap junction plays interiorly, and the new wrap (the rotation point) joins samples that were contiguous in the source, i.e. perfectly continuous. Rotation only changes where the loop STARTS, never how it wraps. _rotate_to_downbeat(): score each beat slot by its circular low-band (<200 Hz, kick-weighted) attack — RMS just after minus just before — plus a small full-band term; roll the winning slot to position 0, zc-snapping the roll offset (B1) so a re-trigger starts at a crossing. Length untouched (np.roll) ⇒ the <1 ms bar-multiple guarantee (0759def5) holds through rotation. Wired into export_take (on by default, rotate=False opts out; chops bars=0 never rotate). Verify-rerank stays export-faithful: _true_seam now returns min(old-wrap seam, rotated-wrap seam) — both junctions of the shipped cycle are measured, so rotation can never hide a bad wrap by moving it inside the file. Tests: +3 (downbeat at slot 5 → rotated to 0; slot-0 no-op; rotating a continuous loop never breaks the seam). 85 passed.
PLN (Algolia) authored -
docs(ear-feedback): superfreak/hammer kit audition — sub-only bass, BCDA loop rotation, demucs vocal high-pass PLN's on-disk verify of the forged kits, with measured confirmation of all three reactions: bass_deep is a pure-sub layer (99% <150 Hz at −12.8 dBFS — equal-loudness invisibility, not a level bug); loops are bar-exact but musically ROTATED (strongest onset lands on beat-slot 1/2/4 instead of 0; other_skank opens −23 dB below its own peak 30 ms — the 'cut DABC' feel); vox thinness is htdemucs routing vocal low end into bass/other (vocal stems carry ~0 energy <150 Hz). Fix directions logged: rotate-to-downbeat export post-pass, BS-Roformer vocals re-cut, harmonic-saturation bass variant.
PLN (Algolia) authored -
Two defects surfaced while shipping the hammer (U Can't Touch This) vox kit extension — both fixed in the shared cut tail, now factored as _finish_clip(): 1. DC offset. Demucs vocal stems carry a small DC bias (measured −9.6e-5 stem-wide on the hammer catch); every cut inherits it and longer phrases tripped the grader's B2 dc-offset flag (|mean|>1e-4, 'break it down' cut measured −1.1e-4). Subtlety: a naive mean-subtract BEFORE fading is undone by the fades — the edge ramps remove signal asymmetrically and re-introduce up to ~1e-3 DC (caught by the new regression test, not by the real-data smoke run). Fix: subtract the FADE-WEIGHTED mean c = Σ(w·x)/Σw (w = fade envelope), then fade — post-fade mean is exactly 0 per channel AND edges are exactly 0. On-disk verify of the shipped hammer vox files reads dc = −0.0e+00, edge = 0.0e+00 across all six. 2. Loop-length drift. cut_phrase_loop placed the tail at exactly loop_beats × beat_s … then zc-snapped it (±12 ms), re-introducing the very grid deviation the length-preserving export rule (0759def5) forbids. The end snap is unnecessary — the fades already guarantee clickless edges — so the tail is now length-rigid: shipped 08_vox_stophammertime_loop is 4.00 beats to 0.02 ms at BPM 132.51. Tests: tightened test_cut_phrase_loop_is_exact_beat_multiple from 0.06-beat slack to <1 ms, added test_cut_removes_source_dc_offset (biased source ⇒ DC-free, unflagged cut). 82 passed.
PLN (Algolia) authored -
The B1 zc-snap moved each loop edge independently (±10ms tolerance), so exported loop DURATIONS drifted off the bar grid: measured −4.2ms on a 4-bar (superfreak bass) and −11.4ms on 2-bar loops (hammer kit, both edges snapped inward). loopAt absorbs the drift, but the kit contract we want is stronger: every exported loop's duration == bars × bar_len. Fix — _snap_length_preserving(): hold the exact bar-multiple sample count RIGID (bar_len_samples from the take's grid BPM) and slide the WHOLE window by a single common offset δ ∈ ±ZC_TOL_MS, picking the δ that minimizes combined edge amplitude |y[start]|+|y[end]| (tie-break: smallest |δ|). Both edges move together ⇒ duration is preserved by construction; guarantee |exported_dur − bars·bar_len| < 1ms (measured <0.02ms on real stems). Chops (bars=0) keep the per-edge snap. _snapped_window/_true_seam (verify-rerank, 791a0d78) now mirror the same scheme so the finder still measures the exact window export writes — export-faithfulness is preserved through the change. kit_multiples_check() makes the guarantee explicit per forge: after export, each file's duration-in-bars at the shared grid BPM is compared to the nearest integer multiple; >0.5% deviation warns with numbers. export_take_report() bundles export + check; the server /api/export now returns the multiples block. Spot-check on real stems (old per-edge vs new length-preserving snap, top-5 finder candidates re-graded on the exact export window): superfreak drums: dev +0.4…+3.6ms → ≤0.01ms; grades C→S, A→A, S→A, S→S superfreak bass: dev −1.1…+3.7ms → ≤0.01ms; grades A→S, B→C, S→B, A→A hammer drums: dev −1.8…+1.9ms → ≤0.01ms; all S stay S hammer bass: dev −2.9…+6.9ms → ≤0.01ms; C→B, A→S, D→C, B→C Net: duration guarantee achieved; grade moves are the rigid-length window exposing true seams (some old windows only looked seamless because per-edge snap trimmed the tail). No systematic regression; existing superfreak/hammer kit exports untouched (validated, in use). Tests: +4 (snap length invariance, bar_len_samples, end-to-end exported- duration-is-exact-bar-multiple on a synthetic stem, multiples-check flags off-grid). 81 passed.
PLN (Algolia) authored -
feat(foundry): lyric-aware vocal sampler (engine/vox.py) — phrase-level, hook-ranked, loop-capable vox one-shots PROBLEM. The Foundry's vocal path (loops.analyze_chops) cuts vocals blind to what is sung — onset→onset slices scored on seam/zc mechanics. But a vocal one-shot's value to a livecoder IS the lyric. The freshly-shipped superfreak dub kit had four tiny 0.35-0.65s onset chops (03-06) with no idea they sat on top of some of the most iconic lyrics in funk ("she's a super freak", "kinky girl", "the kind you read about"). APPROACH. engine/vox.py: transcribe → phrase-segment → iconicity-rank → cut → grade → lyric-name, DRY-reusing grade.py sub-scorers and naming.py convention. - transcribe: whisper word-level timestamps, shelled out like separate.py→demucs, cached at workspace/vox_transcript.json so re-runs are instant. Model escalates small→medium empirically when the known hooks don't surface (small garbled the patois delivery; medium recovers the real lyrics). - segment: group words on inter-word gaps (>=0.45s), cap 6s, split over-long phrases at their widest interior gap; keep per-phrase avg ASR confidence. - iconicity: feature-engineered rank = repetition (normalized phrase text + content n-grams recurring across the track — hooks repeat) + hook keywords (title-derived or --hooks) + clarity (ASR prob) + energy (RMS vs stem median) + duration sweet-spot. - cut: pre-pad + post-pad (clamped to next phrase), zero-crossing snap both edges, 3-10ms fades so edge samples are ~0 regardless of where the snap landed; as-cut level. - kit-level loopability: report each phrase's duration in BEATS at the kit BPM; flag loop-capable phrases (within ±3% of a 1/2/4-bar multiple) and emit a bar-quantized _loop variant (tail pulled to the exact beat grid, still zc-snapped) for the best hooks, so a vocal can loopAt alongside instrument loops. - name: NN_vox_<lyricslug> (naming.lint-clean), dedup identical texts (best-graded instance, up to 2 takes of THE hook). VALIDATION. 17 new mocked-transcription tests (whisper never runs in tests): gap segmentation, over-long split, iconicity repetition-beats-oneoff + hook-keyword boost, beat/loop annotation, cut edges ~0 after zc+fade, next-start clamp, exact-beat loop variant, lyric-slug + lint contract. Full suite 60->77 green. Applied to the superfreak stem: medium transcript contains every iconic line; shipped 6 curated vox files (S/B tier) replacing the 4 blind chops, incl. one 2-bar (8.00-beat) loop variant. All re-graded on disk: no clip (peaks <0.8), no DC (<2e-4), edges exactly 0.0.PLN (Algolia) authored -
Problem — the finder's rank didn't reflect true seam quality. On a real end-to-end run over the Super freak dub kit (Soul Sugar meets Dub Shepherds), several top-scored 2-bar windows CLICKED at the wrap after export while cleaner windows ranked below them. Root cause: analyze_stem scores the seam on the RAW candidate window, but export_take zero-crossing-snaps both boundaries (B1) AND slices at the 3-decimal-rounded start_s/end_s — and both transforms move the wrap. A beat-exact window measuring seam 0.79 read 0.07 once rounded, and that rounded, snapped slice is what actually ships. So the finder's seam proxy diverged from the grader (grade.py), which measures the window that exists on disk — the ground truth. Approach — a verify-rerank pass (loops._verify_rerank) after candidate generation, before dedup/top_n. It re-measures the ~3×top_n survivors' seam on the TRUE post-snap window — the exact zc-snapped, rounded-bounds slice export writes — reusing grade.seam_score (DRY, no duplicated DSP). The proxy seam term is swapped for the true seam in the composite (same weights), and any candidate whose true seam falls below VERIFY_SEAM_FLOOR is hard-demoted so a click can never top the list. Gated behind a new analyze_stem(..., verify=True) kwarg (default on; False reproduces the pre-#19 raw-proxy ranking for autotune baselines). Public signatures (analyze_stem, find_takes, weights dict) unchanged — autotune.py/server.py intact. Also fixed a sparse-material seam false-positive in grade.seam_score: the wrap curvature was normalized by the MEAN |2nd-diff|, which collapses to ~0 on sparse dub percussion (mostly silence + a few hits), blowing the click ratio up (129× on a genuinely clean loop). Now normalized by the 90th-percentile |2nd-diff|, amplitude-floored — 2–3× on the same clean loop. Validation (drums+other stems, bars 1/2/4, shared 132.5 BPM grid, export-faithful grades): drums top-3 BEFORE C 0.65 / C 0.63 / A 0.80 (clicks @93.08s, @24.06s) AFTER S 0.98 / S 0.95 / A 0.81 other top-3 BEFORE B 0.72 / B 0.72 / B 0.70 (bad favorite @56.74s clicks) AFTER S 1.00 / S 0.98 / S 0.99 The known-good drums 4-bar @107.21 (S) and other regions surface; the clicking other @56.74 is demoted out of the top entirely. Tests: +2 verify-rerank regression tests (a proxy-clean but discontinuous wrap that survives the snap is demoted below a seamless one; verify=False leaves the proxy untouched) and +1 sparse-percussion test (a clean sparse loop is NOT flagged clicking). Suite 57 → 60 green.PLN (Algolia) authored
-
- 29 Jun, 2026 10 commits
-
-
Refreshed the onboarding template (rendered PDF is gitignored, carries live tokens): -
✨ NEW §3 "The YouTube → emotion pipeline": /sources fetch → /jobs poll → /artifacts download → /analyze/emotion, the showcase end-to-end flow (curl + the @nech/api TS shape), idempotent-on-repeat noted. - Credentials block now carries BOTH tokens: the api:* Bearer AND the npm read token for installing @nech/api from npm.nech.pl. - Endpoint table gains /sources, /jobs/{id} (+DELETE), /artifacts/{cid}/{name}. - §1 surfaces the live /docs + /openapi.json links and corrects "access": only /healthz is public now; docs/spec need api:docs (covered by api:*). - Sections renumbered (pipeline=3, client=4, good-to-know=5, support=6).PLN (Algolia) authored -
So the session task board survives a push to git.nech.pl (it is local harness state, not in git). Two cold-readable TODOs next to their code: - armada/api/TODO.md — Fourier audio-API remaining: heavy chain #30 /separate + #31 /loops/grade/correlate (need a GPU runner), optional #47 X-Accel + #46 Grafana, follow-ups (nech_api python client, hexa npm read token); EPIC #21 closes with the chain. - tools/foundry/TODO.md — #19 auto-tune loop (in-progress, harness validated +10.5%, exact resume steps) + #20 batch-explore corpus. Plus archived #44 (Verdaccio + @nech/api live) to completed-archive.md. Root TODO.md left untouched (it is the paused Pulsar livecoding-perf session).
PLN (Algolia) authored -
Verdaccio is live on npm.nech.pl and @nech/api@0.1.0 is published + install-verified (AudioApi/Configuration import clean as a consumer). So the hand-written zero-dep nech.ts drop-in is retired (git rm); @nech/api is the one true client. README + onboarding.html now show the @nech scope install with the read token, and the docs row reflects that /docs + /openapi.json are bearer-gated (api:docs) not public. VERDACCIO.md marked DEPLOYED with the three gotchas folded in (listen 0.0.0.0, chown 10001, TLSv1.2-only) + the auth-gated-reads note.
PLN (Algolia) authored -
Rich entry for the documentary: the anonymous-exposure finding, the cross-cutting api:docs design (read-specs decoupled from call-API), the lockstep drift-guard dance, and the shared-repo rebase + diff-before-tee discipline that kept SRE work intact.
PLN (Algolia) authored -
Mirrors the merged platform canonical: required_for() maps docs|openapi.json|redoc → {realm}:docs. Byte-identical to nechapi/_platform/scopes.py (drift guard green). Part of #48; deploy = nechapi-platform redeploy + nginx reload.PLN (Algolia) authored -
Rich entries for the documentary trail: the freebox-deferred simplification of #27, the open-url-scope-with-apikey-trust decision on #29, the SSRF block set, and how #25/#26 pre-built the seams (result_ref/gc, alias table, born-done path) that made both mostly wiring.
PLN (Algolia) authored -
Two links of the heavy chain, built on the #25 job backbone. Both ship the CODE now; they go live the moment erable ssh is back (yt-dlp install) — no GPU needed. #27 — artifact store (artifacts.py). Big binaries (fetched sources, later stems/ loops) live content-addressed on local disk under FOURIER_ARTIFACTS, tracked as ordinary cache rows (kind→result_ref), so the existing LRU cache.gc() already evicts the coldest under a cap. Freebox was the original SSOT plan — deferred per PLN; erable-local for now. Serving is zero-copy via nginx X-Accel-Redirect (FOURIER_X_ACCEL), FileResponse fallback in dev. resolve() refuses path traversal / anything outside the root. #29 — /sources (engines/sources.py + POST /sources). The worker shells out to yt-dlp → bestaudio → 44.1k WAV (analysis-ready for /separate, /loops, features), content-addresses it, stores it, and aliases url→content_id so a repeat URL is an idempotent born-done job (no re-download). Submit returns 202 {job_id}; poll /jobs/{id}. URL scope is OPEN (any http(s)) per PLN — gated by the platform apikey + nechapi monitoring — but we still hard-block the SSRF footguns (non-http(s), localhost, RFC1918/link-local/reserved IPs; incl. the 169.254.169.254 metadata classic). The fetch is a thin seam so tests mock the network with a synthetic WAV. Also: GET /artifacts/{cid}/{name} serving endpoint; healthz reports yt-dlp presence; _meter now counts by ROUTE TEMPLATE not concrete path (else /jobs/{id} & /artifacts/{id} would mint unbounded Prometheus series). OpenAPI refreshed 13→17 ops, @nech/api regenerated (submitSource/getArtifact/getJob/cancelJob). yt-dlp added to requirements-deploy; DEPLOY.md §5b documents the yt-dlp+ffmpeg install and the nginx internal location. 78→95 tests green.PLN (Algolia) authored -
The audio API was internally codenamed "Douanier" (Le Douanier Rousseau — a customs-officer pun on edge auth). Renamed to "Fourier": the FFT is literally the transform behind /spectrum and most of the feature stack, and central auth is the platform’s job now, not ours — so the name should point at the signal work, not the gate. Internal-codename-only: the public contract (/audio/v1, gateway headers) is untouched. Mechanical case-aware sweep across armada/api + the completed-archive, plus hand-rewritten prose (the Rousseau attribution → Joseph Fourier; dropped the customs-gate /
🛂 metaphors). Renames: douanier.py → fourier.py, deploy/douanier-worker.service → deploy/fourier-worker.service. Env prefix DOUANIER_* → FOURIER_* (config contract; safe — nech.pl is pre-users), dev venv ~/.virtualenvs/douanier → fourier (shebangs fixed). 78 tests green after.PLN (Algolia) authored -
PLN (Algolia) authored
-
The async spine for the heavy chain (sources/separate/loops): the API enqueues and returns a job_id; a worker runs it out-of-band so the API never blocks. Built to the four decisions taken with PLN: - HYBRID sync/async: the 11 cheap analyses stay synchronous (cache-backed); only heavy work becomes a job. The line is wall-clock cost, not endpoint kind. - PER-ENGINE submit + SHARED poll: engines return 202 {job_id} (wired in #29/#30/#31); GET /jobs/{id} polls, DELETE /jobs/{id} cancels. Typed JobStatus response_model so the generated client gets audio.getJob()/cancelJob(). - SINGLE serialized worker: one job at a time (erable is 2 GB / no GPU; two demucs runs would OOM). The atomic claim is still race-safe for N workers. - Identity = gateway tenant; a job is visible only to its owner (cross-tenant poll/cancel → 404, not 403, so existence doesn't leak). Pieces: - db.py: jobs table (status/priority/progress/result/attempts/webhook) + a claim-ordered index. - jobs.py: enqueue (incl. born-done idempotent fast path when the cache already has the result), atomic claim (candidate → guarded UPDATE WHERE status= 'pending'; rowcount-0 retry; WAL serializes writers so no double-claim), progress/finish/fail-with-requeue-under-cap, crash recovery (running→pending on boot), cooperative cancel + is_cancelled checkpoints. - worker.py: serialized loop — recover → claim → dispatch by type → finish/fail → optional webhook; SIGTERM-graceful; imports engine handler modules (none yet, idles politely); engines register via @jobs.handler. - deploy/douanier-worker.service: systemd --user unit (linger) — the durable run path (harness/nohup jobs die on teardown). Prod container-vs-host wiring is the paved-road call (SRE/#34). VALIDATION: tests/test_jobs.py — submit→claim→done; born-done fast path; finish/requeue-to-cap; crash recovery; NO double-claim across 6 threads × 25 jobs (each claimed exactly once); worker dispatch to done; missing-handler error; exception→requeue; cancel mid-flight; HTTP poll + ownership 404 + idempotent cancel. 11 new tests, full suite 67→78 green. OpenAPI snapshot refreshed (15 ops) + @nech/api regenerated (getJob/cancelJob); noImplicitAny relaxed for the 100%-generated client (typescript-fetch's camel/snake guard trips TS7053).PLN (Algolia) authored
-
- 28 Jun, 2026 10 commits
-
-
The hand-written nech.ts was always interim; this replaces it with a client GENERATED from the OpenAPI spec, so the SDK can never drift from the API. Source-of-truth fixes (the spec drives the ergonomics): - Clean operationIds on all 13 routes (operation_id="analyzeEmotion" etc.) so the generator emits `audio.analyzeEmotion({file})`, not the default `analyzeEmotionAnalyzeEmotionPost`. Also cleans /docs. - One router tag ["audio"] so the generated class is AudioApi, not DefaultApi. - refresh_openapi.py: the canonical snapshot dump. app.openapi() omits the public `servers` block (only injected when served behind root_path), and the old README recipe silently dropped it — test_openapi_snapshot guards it, so the refresh now injects https://api.nech.pl/audio/v1 itself. The package (@nech/api, clients/nech-api/): - typescript-fetch generator → src/audio/ (checked in; regenerate via codegen.sh), bundled with tsup to a single ESM file + .d.ts. - Umbrella package, per-domain SUBPATH exports: `import {AudioApi} from '@nech/api/audio'` — one install/version, tree-shakeable, geo/iris slot in as siblings later (codegen.sh + exports map have the stubs). - Build via tsup not bare tsc: the generated code uses extensionless relative imports (./runtime) that Node ESM can't resolve from plain tsc output; bundling sidesteps it entirely. Verified end-to-end: `@nech/api/audio` resolves through the exports map, 13/13 methods present. - publishConfig + .npmrc point the @nech scope at https://npm.nech.pl. VERDACCIO.md: copy-paste runbook to stand the private registry up on erable (container :4873, nginx TLS vhost for npm.nech.pl, seed publisher + lock signups, publish). NOT yet run — `ssh erable` failed with publickey from the build host; needs the key loaded. DNS for npm.nech.pl is already set. README reframed: @nech/api is the official client; nech.ts stays documented as the working drop-in until the registry is live, then it's retired. Scrubbed a $DOUANIER_TOKEN codename leak in the curl example. 67 API tests still green.PLN (Algolia) authored -
PLN (Algolia) authored
-
The platform moved under us between sessions. SRE.md now records the landscape the audio service actually deploys into: - nechapi got its own repo (git@git.nech.pl:pln/nechapi.git) with a golden base image (nechapi/py) + a `nechapi ship` CLI — the paved road. Onboarding is FROM nechapi/py:1 + NECHAPI_ROOT_PATH + `nechapi ship`, not the old hand-rolled docker save/load. - A sibling tenant (geo/v1, Verniquet) is already live through it. - Correction to the 2026-06-25 "CLAP is the wrong engine" line: CLAP is right, used the canonical way — embeddings precomputed OFFLINE, a compact ANN index shipped to erable, request-time = tiny text-embed + cosine. What's wrong is running CLAP at request time on a GPU-less 2GB host. This closes the ops task (#34): observability shipped on the platform side (per-call capture, /admin analytics, public uptime status page — nechapi f89d7f9, now rebased onto the paved road and pushed), and the systemd-restart / deploy-runbook half is subsumed by `nechapi ship` + the golden base, owned in the SRE repo. No bespoke systemd units to write here.
PLN (Algolia) authored -
The 67-test suite existed and was green, but only under a --system-site-packages venv — which made it quietly host-dependent. A system python bump to 3.14 dropped fastapi/httpx from that venv and the whole suite went to "no tests collected" (red, but for an env reason, not a code reason). CI that inherits the host's site-packages would hide exactly this class of breakage. Fix = make the suite hermetic and prove it: - requirements-test.txt pins the real test surface — fastapi/uvicorn/ multipart (via requirements.txt) + numpy + soundfile + librosa + pyloudnorm + httpx + pytest. NOT --system-site-packages. - The heavy ML stack (torch, laion_clap, demucs) is deliberately absent: those engines aren't exercised by the tests, so CI installs in seconds- to-a-minute instead of pulling multi-GB wheels. - Rehearsed in a clean throwaway venv: first run surfaced two masked deps the system venv had been silently supplying — librosa (lazy-imported by signal/feats/ears/grade) and pyloudnorm (the LUFS loudness engine). Pinned both; clean-venv run is now 67 passed. - .gitea/workflows/api-ci.yml runs it on git.plnech.fr for any push/PR touching armada/api/**. Dormant until an act_runner is registered for the repo; the workflow is correct and locally rehearsed. Closes the test-suite + CI task (#35): the suite is reproducible and the pipeline is declared.
PLN (Algolia) authored -
Rich entries for the api.nech.pl platform build-out: first erable deploy + clone3/seccomp gotcha, the realm:domain:path scope convention + central enforcement, Shipow onboarding, and the observability stack (capture + admin analytics + public uptime). Source material for the documentary.
PLN (Algolia) authored -
The /docs + /openapi.json title read "Douanier — the audio sub-API…" and /healthz returned service:"douanier" — internal codename leaking to consumers. - FastAPI title → "Nech.PL Audio Intelligence API"; description rewritten to describe the engines (no "customs gate" framing). - /healthz service → "nech-audio". - Regenerated clients/openapi.json snapshot (info.title now clean; 13 paths, servers=api.nech.pl/audio/v1) — feeds the generated-client pipeline (#44). Built + redeployed; verified at the edge: openapi.json info.title and healthz both clean. "Douanier" now survives only as the internal repo/metric name. 67 tests green.
PLN (Algolia) authored -
"Douanier" is the internal codename for the audio sub-API; it shouldn't be what a consumer imports. Renamed the client-facing surface to the platform brand (NechAPI), keeping "Douanier" only as the internal service/repo name. - clients/nech.ts (was douanier.ts) — ONE umbrella `NechAPI` client, namespaced per sub-API: `new NechAPI({token}).audio.emotion(clip)`. Future sub-APIs add `nech.geo.…` with no import change. Also exports the standalone `NechAudio` sub-client for the smaller-bundle path. DouanierError→NechError, DouanierOptions→NechOptions. Typechecks clean under tsc --strict. - response header X-Douanier-Cache → X-Nech-Cache (app.py + all tests + client + docs). Verified live end-to-end: miss→hit, old header gone. - clients/README + onboarding.html (the Shipow PDF source) updated to NechAPI / nech.audio. PDF re-rendered. Built + redeployed douanier:latest to erable; 67 tests green. NOTE (next iteration): OpenAPI info.title and /healthz `service` still say "douanier" — a cosmetic /docs leak, scrub on the next redeploy.PLN (Algolia) authored -
A shareable getting-started for the hydra-live-hexa Studio: what the Audio Intelligence API does, base URL + bearer auth, the full endpoint list, a curl quickstart and the zero-dep TS/Vercel snippet, plus caching/limits/errors and support. Branded to the Nech.PL APIs / Ship's Bridge look; A4, print-clean. onboarding.html is the committed template (token placeholder __NECHPL_TOKEN__); render a per-tenant PDF with chromium --headless --print-to-pdf after sed-filling the key. The rendered PDF carries a live token, so clients/*.pdf is gitignored — never commit it; deliver it to the tenant over a private channel.
PLN (Algolia) authored -
The audio API now speaks the platform scope convention (nechapi scopes.py): access is hierarchical realm:domain:path and the required scope is DERIVED FROM THE ROUTE, so it's maintenance-free — add an endpoint and its scope exists. - scopes.py — vendored byte-for-byte from nechapi/_platform/scopes.py; a drift-guard test (test_scopes.py) fails if the two ever diverge, so the gateway and this service can never disagree on who's allowed in. - app.py — replaced the per-route require_scope("emotion"|"features"|…) strings with ONE path-derived dependency: `require` computes api:audio:<path> from the request and checks it; `require_auth` covers /me (any identity). Also closed a footgun: a gateway-injected request with a MISSING X-Scopes header now defaults to NO scopes (was "*"). - auth.py — Principal.has_scope is now the hierarchical matcher (api:audio:* authorizes api:audio:analyze:emotion, etc.). - tests — gateway-header tests grant api:audio:*; the scope-enforcement tests now prove real path-derivation (a sibling grant like api:audio:features → 403 on /grade and /onsets). +test_scopes.py for the matcher + drift guard. 67 passing. - clients/README — scope table rewritten to the convention (api:audio:<path>, grant api:audio:* or api:* for breadth). Validated end-to-end through https://api.nech.pl/audio/v1 with a freshly minted api:* token: /me → scopes [api:*]; /features 200 (cache miss→hit); a sibling scope 403s; no-identity 401s. Built + redeployed douanier:latest to erable (seccomp=unconfined per DEPLOY.md).PLN (Algolia) authored -
First real deploy of douanier:latest to erable went green, but only after diagnosing two host-specific traps that DEPLOY.md now records so the next deploy is one shot: 1. clone3 vs old seccomp — the container booted uvicorn then segfaulted (exit 139) / aborted with "OpenBLAS blas_thread_init: pthread_create failed … Operation not permitted". Root cause: Docker 19.03 on kernel 4.9's default seccomp profile rejects the clone3 syscall that python:3.12-slim's glibc 2.36 uses for pthread_create. Fix: run with --security-opt seccomp=unconfined (safe — the container is loopback-only behind the gateway). 2. BLAS thread pool on a small shared box — pinned OPENBLAS/OMP/NUMEXPR/MKL _NUM_THREADS=1 in the env file: belt-and-braces with the seccomp fix on the old kernel and right-sized for CPU-light work on 4 vCPU / ~2 GB. Also: data volume is /home/pln/srv/douanier/data (no sudo for /srv; it's pure transient cache so the path is immaterial). Verified end-to-end through the gateway: healthz/openapi/docs all 200, authed routes 401 without a token.
PLN (Algolia) authored
-
- 25 Jun, 2026 11 commits
-
-
PLN (Algolia) authored
-
hexa can now call the audio API with types, not guesswork. - clients/openapi.json — checked-in OpenAPI 3.1 snapshot (servers pinned to the public https://api.nech.pl/audio/v1), covering all 13 routes. A snapshot-drift guard test asserts it stays in sync with the live app (add a route → refresh or CI fails). - clients/douanier.ts — a typed, ZERO-dependency client (global fetch/FormData/Blob; works in Node 18+, Vercel Functions, Edge). One method per engine (emotion, features, samples, grade, onsets, waveform, analyze, loudness, spectrum, naming), typed results, X-Douanier-Cache surfaced as result._cache, DouanierError on non-2xx. Typechecks clean under tsc --strict. - clients/README.md — base URL + bearer, a Vercel Function example, the endpoint/ scope table, curl, and the openapi-generator one-liner for full codegen. 60/60 tests (added the snapshot guard). The API is now self-describing (/audio/v1/docs + openapi.json) and has a drop-in client.
PLN (Algolia) authored -
Three more torch-free, cached building blocks → 11 engines total. - POST /loudness → BS.1770 integrated LUFS (pyloudnorm) + sample/true-peak (4× oversample) + crest + the gain to hit each delivery target (-14 streaming, -9 club, reference_postprod_master). The mastering numbers PLN + hexa gate on. engines/loudness.py; pyloudnorm added to requirements-deploy (pure-python, tiny). - POST /spectrum → FFT-AS-A-SERVICE (PLN's ask): a downsampled, render-ready spectrogram bands×frames, 0..1 normalized; mel (perceptual, for visuals) or log-linear; frames=1 collapses to a single averaged FFT spectrum. Bounded payload so it caches cheaply. signal.spectrum(). - POST /naming → convention-compliant sample name (NN_role_character) from the MEASURED role + character, never the file name. naming.py vendored from the Foundry (drift-guarded); character_of() derives the adjective from features. Validated in the torch-free container (LUFS -19.1 w/ gain +5.1/+10.1; mel band centers; "07_melodic_warm" lint-clean). 58/58 tests (test_extra.py: engines + endpoints + naming drift guard + character_of). Image now serves emotion/features/ samples/grade/onsets/waveform/analyze/loudness/spectrum/naming + separate-503.
PLN (Algolia) authored -
Three cheap, torch-free, cached building blocks — the no-model tier, plus a convenience composite. The image now serves 8 engines (emotion/features/samples/ grade/onsets/waveform/analyze + separate-503). - POST /onsets → onset hit times (s) + tempo + onset rate. Rhythmic hits for visual sync / slicing. scope `onsets`. - POST /waveform → render-ready waveform: per-bin [min,max] in [-1,1] + a 0..1 RMS energy envelope (?bins= ≤4000). For hexa's audio-reactive visuals. scope `waveform`. - POST /analyze → emotion + features + sample role in ONE cached call (fewer round-trips for hexa); each is the same engine the dedicated routes use. scope `analyze`. engines/signal.py is self-contained librosa (onsets uses feature.tempo, the 0.11-correct path). Validated in the torch-free container (/onsets tempo 107.7 n=41; /waveform bins honored, peaks in range; /analyze returns all three blocks). 50/50 tests (added test_signal.py: engine + endpoint + cache + scope-gate + torch-free).
PLN (Algolia) authored -
Exposes the Foundry's mechanical loop-quality grader (the "katana") on /audio/v1/grade: upload a loop/one-shot → composite 0..1 + S/A/B/C/D tier, per-rule sub-scores (seam click, zero-crossing cleanliness, DC, bar self-consistency, level, bass mono-compat) + human-readable flags. scope `grade`, cached, torch-free. Directly serves "iterate on sampling quality". engines/grade.py is a BYTE-FOR-BYTE vendored copy of tools/foundry/engine/grade.py (self-contained — numpy/soundfile/pydantic/librosa, no Foundry/torch — because the container holds only armada/api/). It's kept identical on purpose, and tests/test_grade_endpoint.py is a DRIFT GUARD: it imports the Foundry canonical standalone and asserts the vendored copy grades identically (grade/tier/sub + WEIGHTS/THRESH) on a synth signal — a future Foundry tweak that isn't re-vendored fails CI here, not silently in prod (parsers-over-copy, applied to a vendored copy). Validated in the torch-free container (healthz engines now emotion/features/ samples/grade/separate; a sine tone grades D with a correct seam-click flag) + 43/43 tests. Note: this is the CPU-doable slice of #31's /loops /grade /correlate — the finder (/loops) and corpus correlation (/correlate) remain.
PLN (Algolia) authored -
Two new CPU-native, torch-free, cached endpoints on the audio sub-API — the "interesting value points" for hexa beyond emotion, each the same shape/pattern as /analyze/emotion (upload → content-address → cache → compute). - POST /features → the ~35-dim audio feature stack (spectral moments, MFCCs, chroma/key, envelope/attack-decay, + rhythm/tempo). scope `features`. - POST /analyze/samples → per-sample EDA + role (percs|bass|melodic|tops|atmos) decided by the MEASURED spectrum (centroid + band energy), never the name; an optional ?name= only disambiguates breaks/drums (feedback_mastering_eda). scope `samples`. Engine: engines/feats.py is SELF-CONTAINED (vendored DSP), like ears_light — the deployed container holds only armada/api/, not armada/tide-table/, so it can NOT import sample_features/audio_lens at runtime. It mirrors their algorithms and fixes the librosa-0.11 tempo bug (feature.tempo, not the removed feature.rhythm.tempo that silently dropped tempo in the tide-table original). Wiring: a shared _cached() helper now backs all three analyze routes (DRY); healthz advertises the engines map; both routes are scope-gated via the gateway X-Scopes. Validated in the torch-free container: /features 200 (41 features, tempo 107.7, key=Amaj), /analyze/samples role-by-measurement, 403 scope gate, X-Douanier-Cache hit on repeat. 40/40 tests (added test_feats.py + endpoint cases).
PLN (Algolia) authored -
The SRE edge is live (api.nech.pl returns 503 warming_up); the only thing blocking a green healthz was a CPU-deployable image. This ships it, and folds in the SRE's platform reframe that landed in the same letter. Baseline image - Dockerfile: python:3.12-slim + ffmpeg/libsndfile, the LIGHT torch-free stack only (requirements-deploy.txt: fastapi/uvicorn/librosa/numpy/soundfile). Builds to ~1 GB, runs well under the 2 GB-RAM erable budget. Default engine = light. - Validated in-container end to end: healthz green, emotion miss->hit cache, 401/403/200 auth gating, /metrics, /separate 503. Platform contract (SRE update: api.nech.pl is a multi-API gateway; we're the `audio` sub-API) - Public path is /audio/v1/...; the gateway strips the prefix and proxies to us at root. Routes moved off the /v1 router to root; root_path=/audio/v1 so OpenAPI/docs advertise the real public paths (verified servers=[{/audio/v1}]). - Central auth: dropped our own bearer verification in the request path. We now trust the gateway-injected X-Tenant / X-Scopes (loopback-only bind = only the gateway can reach us). Local-dev keeps a DOUANIER_DEV_TOKEN bearer fallback. Supersedes #23/#24 (the SQLite token store + CLI remain for dev only). - /metrics: Prometheus text (douanier_up, requests_total{path,status}, cache_rows/hits{kind}) for the erable scraper. - /separate: deliberate 503 compute_unavailable (retriable) — no GPU path on erable; route exists so hexa can code against it now. GPU backend is env-selected later. Docs + tests - DEPLOY.md rewritten as the erable container contract (build, /data volume, env-file, loopback publish, cache cap, central-auth onboarding). README reframed to the platform shape. 32/32 tests pass (smoke retargeted to root paths + header auth; added /separate, /metrics, openapi-root-path coverage).PLN (Algolia) authored -
Rich archive entries (blog/video source material) for the day's shipped work: the Douanier Audio-Intelligence API's first endpoints (#22/#36 scaffold+emotion walking skeleton, #23 SQLite auth+CLI, #26 content-addressed cache with the measured 5663x repeat speedup) and the Foundry sampling-classics demo sources. Each entry stands alone for a cold reader — goal, what shipped (commit hashes), non-obvious learnings + numbers.
PLN (Algolia) authored -
PLN's closed feedback loop: generate candidate loops with a settings profile → grade them on the full quality rubric → score the profile → search for the settings that maximise quality. Auto-tune the composite weights instead of hand-guessing them. WHY: the finder/grader/correlator exist; the composite weights W were "provisional, calibrate with #7". This builds the machine that calibrates them against measured grade — and measures the lift honestly rather than asserting a tuning is better. WHAT: - engine/autotune.py — evaluate(stems, settings) runs the finder with a profile, grades every produced candidate on the FULL rubric (grade.py adds dc / level / bass-mono / bar-consistency — features the finder's own score does NOT use, so tuning pulls in signal beyond seam/zc), aggregates objective = mean_grade × coverage. search() sweeps random|grid and always includes the current defaults (loops.W) as a baseline row so the report shows lift, not just a number. - autotune.py — CLI (--stems glob --n --method --max-stems); PII-safe aggregated leaderboard (counts, never filenames). - engine/loops.py — analyze_stem now accepts an optional `weights` override (defaults unchanged); the grid is preloaded once per stem (weight-independent) so the sweep is cheap — a scarcity-minded speedup. VALIDATION: - 4/4 mocked harness unit tests (aggregation, empty-set, ranked+baseline-included, weights threaded through to the finder). - Real end-to-end run, 2 drums stems × 8 configs: baseline objective 0.690 → best 0.763 (+10.5%); the defaults ranked LAST of 8 — the search found real lift. NOT DONE ON PURPOSE: the finder defaults (loops.W) are UNCHANGED. The winning profile (seam 0.4, zc 0.05, bars=(4,8)) is from a 2-stem sample AND seam/zc are in both the finder score and the grade (a confound) — so it's suggestive, not a mandate. Remaining for #19: recall-vs-provenance-GT as an anti-gaming 2nd objective, per-stem-role tuning (vocals=chops want different weights than drums), a full-corpus + demo-corpus campaign, an extract-once/score-many speedup, THEN adopt a validated profile as the default.
PLN (Algolia) authored -
The SRE reply (armada/api/SRE.md) revealed the public host is erable — Debian 9, no GPU, ~2 GB RAM, ~3 GB disk free — where the ~4 GB CLAP/torch image can't fit. This unblocks deployment in principle: a CPU-native emotion path that needs no torch, behind a selector so the rich CLAP engine stays the dev/GPU default. WHY: the SRE is "blocking on a CPU-deployable image". The emotion read must work within ~2 GB RAM and sub-second, without torch — but the API response shape must not change, so hexa's integration and the cache are unaffected by the swap. WHAT: - engines/ears_light.py — heuristic valence/arousal from librosa features only (no torch): arousal from RMS energy + tempo + spectral brightness; valence from major/minor mode (Krumhansl key-profile correlation) + brightness. Mapped onto the SAME 12 emotion-ontology anchors by V/A distance → a CLAP-shaped {valence, arousal, top, dist, confidence} dict, tagged engine="light". Honest baseline; an Essentia/CLAP-grade precise tier lands later behind the same shape. - config.DOUANIER_EMOTION_ENGINE (clap|light); ears.emotion_read() dispatches; cache keys on the engine so clap/light reads don't collide; healthz reports the selected engine + availability. VALIDATION: - 27/27 tests green (+5): anchors drift-guarded == emotion_ontology.EMOTIONS; mode-valence major>minor; light read shape + V/A in range + dist sums to 1; arousal orders loud/bright/noisy above quiet/low; and the endpoint runs end-to-end via TestClient with engine=light — the torch-free erable path proven. REMAINING in #37 (queued, not rapid): Dockerfile (light image, no torch, fits ~3 GB / 2 GB RAM, bind 127.0.0.1:9780, --env-file); Essentia MusiCNN upgrade for the light tier; separation /v1/separate → 503 + env-selected GPU-runner dispatch seam (with #29/#30); /v1/metrics; 2 GB cache hard-cap. See SRE.md + memory project_douanier_api 'HOSTING REALITY'.PLN (Algolia) authored -
Add a curated list of 10 sampling classics as built-in demo sources — fuel for demoing the Foundry, precomputing a fixed validation corpus, and iterating on loop/chop quality (#19 autotune input). WHY: we need a stable, pedagogically-diverse set to validate the finder across material types instead of ad-hoc URLs. The set spans the axes that stress the finder differently: canonical drum BREAKS (Amen, Funky Drummer, Apache — the gold standards; if the rubric can't nail these it's wrong), a BASS-defining groove (Chic – Good Times), VOCAL/no-drums (Loituma), ORCHESTRAL/no-drums edge case (Mozart 40 — exercises the no-drums grid fallback), a full clean POP mix (Rickroll), hip-hop sample-collage (Grandmaster Flash Wheels of Steel, Humpty Dance), and a drum+vocal-stab combo (Lyn Collins – Think). WHAT: - demos.json — authored data (slug/title/year/source/why/expect/tags). Each entry flags drums-presence because the finder's grid-from-drums path depends on it (no-drums → bass/first-stem fallback, per the recall notes). - engine/demos.py — loader (load / by_slug / source_of). - foundry.py "demos" command — list, or precompute via --catch <slug> / --all (+ --sep to separate). This batch path is exactly what #19 autotune consumes. - server.py — GET /api/demos; /api/fetch now accepts "ytsearch1:" queries too. - ui/index.html — a 'sampling classics' quick-pick strip (click → fills the URL, tooltip shows why/expect). HONESTY ON LINKS: only Loituma (from the repo's own test fixture) and Rickroll (universally known id) ship as verified watch URLs (✓). The rest use "ytsearch1:" queries (≈) that resolve to the top hit at fetch time — so we never ship a guessed video id that silently breaks. yt-dlp accepts both forms. VALIDATION: foundry demos lists 10 (✓/≈ marked); /api/demos serves the JSON; UI strip renders; loader resolves source_of('amen-break').PLN (Algolia) authored
-