# Strand C — Loop-Detection Algorithms & Prior Art

> Research date: 2026-06-22.  
> Covers: beat/tempo tracking, loop-point / segment detection, seamless-cut tech, existing auto-loop tools.  
> Context: demucs-separated stems, livecoded TidalCycles music with BPM drift, target use in `engine/loops.py`.

---

## 1. Beat / Tempo / Downbeat Tracking

### 1.1 librosa — `beat_track`, `plp`, `onset_strength`, `tempo`, `fourier_tempogram`

**How it works.**  
`beat_track` (Ellis 2007 dynamic-programming) estimates a single global BPM via the autocorrelation of the onset-strength envelope, then picks beats consistent with that tempo using DP. It does support a *time-varying* `bpm` array (one value per onset frame), but the documentation is explicit: the algorithm "does not follow particularly well" when the tempo changes because it defaults to assuming a single average tempo across the signal. ([librosa docs — Dynamic Beat](http://librosa.org/doc/0.11.0/auto_examples/plot_dynamic_beat.html))

`plp` (Predominant Local Pulse, Grosche & Müller) analyses the onset-strength envelope in the *frequency domain* (Fourier tempogram) to find a locally-stable tempo **per frame**, then synthesises local half-waves whose peaks are the beat candidates. The librosa docs explicitly recommend `plp` over `beat_track` "when the tempo is expected to vary significantly over time" and for streaming use because it does not require the full signal. ([librosa docs — plp](https://librosa.org/doc/main/generated/librosa.beat.plp.html))

**Measured latency (system python3, librosa 0.11.0, 120 s synthetic signal, M²-era CPU):**

| Function | Wall time |
|---|---|
| `onset_strength` | 2.06 s |
| `plp` + peak-pick | 1.19 s |
| `beat_track` | 1.85 s |
| `MFCC` (13 coeffs) | 0.87 s |
| `chroma_cqt` | 0.76 s |
| Beat-sync `recurrence_matrix` (224×224) | 0.45 s |
| Foote novelty on beat-sync R | 0.013 s |
| ZCR | 0.008 s |
| **Full pipeline total** | **~5.4 s** |

**Suitability for BPM-drifting livecoded material.** `beat_track` is unsuitable for tracks where `setcps` changes mid-performance (e.g. 60→120→180→60 BPM). Its single-tempo model will snap to an average, producing misaligned beat frames. **`plp` is the preferred tracker** — frame-local tempo adapts across a `setcps` ramp. The extracted PLP maxima (via `scipy.signal.find_peaks`) form a variable-tempo beat grid without requiring the whole track up front. Caveats: PLP's time-frequency analysis still assumes locally stationary short windows (~384 frames at hop=512 → ~8 s context); abrupt single-step `setcps` jumps may cause a 1–2 beat slip at the transition boundary. Post-snap to the nearest onset-strength peak mitigates this.

**Dependency weight:** zero — already in the tide-table venv.

### 1.2 madmom — `DBNDownBeatTrackingProcessor`

**How it works.** Combines a bi-directional RNN (trained on annotated downbeat datasets) with a Dynamic Bayesian Network / HMM decoder that enforces metre consistency. Ranked #1 at MIREX 2016 downbeat tracking. ([madmom docs — downbeats](https://madmom.readthedocs.io/en/v0.16/modules/features/downbeats.html))

**Accuracy vs librosa.** Multiple comparisons and the BIFF.ai survey report madmom as the strongest *offline* downbeat tracker on classical and complex-rhythm recordings. It reliably emits (beat, downbeat) pairs. ([BIFF.ai rundown](https://biff.ai/a-rundown-of-open-source-beat-detection-models/))

**Suitability for livecoded material.** The RNN was trained on datasets with stable metre. BPM drift and `setcps` jumps are not in its training distribution. It assumes 4/4 or 3/4 and a constant metre within a track segment; a `slow 2` applied mid-pattern (halving effective BPM) may confuse the DBN state machine. The BIFF rundown notes madmom is "better suited to controlled environments rather than dynamic live scenarios."

**Latency.** Offline only; no streaming mode. Processing adds the RNN forward pass on top of signal processing: adds several seconds on long stems. BeatNet (below) wraps madmom's signal front-end with particle filtering and offers online/streaming modes but itself requires madmom as a dependency. ([BeatNet](https://github.com/mjhydri/BeatNet))

**Dependency weight: HIGH — NOT installed.** madmom requires Python ≤ 3.9 for its Cython extensions in the 0.16 line; installation on Python 3.12+ (likely the system python3 version) requires building from source and has known compatibility issues with NumPy 2.x. **Flag: do not add madmom to the tide-table venv without careful version-pinning and build testing.**

### 1.3 aubio

**How it works.** C library with Python bindings; beat tracker uses a phase-vocoder-based tempo estimator plus onset detector. Designed for real-time use (large buffer = higher accuracy at ~2048 samples, low latency at 512).

**Accuracy.** Within ±1% BPM on steady tracks; cannot correctly detect tempo changes smaller than 2.75%. ([aubio tempo issue #227](https://github.com/aubio/aubio/issues/227); cross-library comparison quoted in madmom paper). Prefers measurements around 107 BPM; may return half/double the real BPM on edge cases. In the BIFF survey madmom outperforms aubio on complex rhythms and classical material.

**Suitability for livecoded material.** Less suitable than librosa PLP: no per-frame tempo adaptation, assumes locally constant BPM window. Good for real-time onset-triggered applications (e.g. live VJ sync) but not the offline post-processing use case here.

**Dependency weight: NOT installed, C build required.** Not worth adding when librosa already covers the use case more robustly in our offline pipeline.

### 1.4 BeatNet (ISMIR 2021)

CRNN + particle filter for joint beat/downbeat/tempo/metre tracking. Handles streaming and real-time modes (latency < 50 ms). Best-of-all-worlds candidate for accuracy AND variable tempo — particle filtering can track abrupt BPM jumps that break the madmom DBN. ([BeatNet paper](https://arxiv.org/pdf/2108.03576))

**However:** requires madmom as a dependency (see 1.2 weight warning above). Not installed. Overkill for an offline batch loop-finder — the latency advantage is irrelevant for our use case.

### 1.5 "Beat This!" (2024)

A 2024 ISMIR paper proposes eliminating DBN post-processing entirely with a Transformer-based model that directly predicts beat activation — reportedly matching or beating madmom accuracy without the HMM overhead. ([arXiv 2407.21658](https://arxiv.org/pdf/2407.21658)). Promising for future adoption but currently requires a custom model checkpoint; not available as a pip drop-in and not installed.

---

## 2. Loop-Point / Segment Detection

### 2.1 Self-Similarity Matrix (SSM) + Foote Novelty

**Principle.** Build a feature matrix F (e.g. MFCC, chroma, or a concatenation) over time. Compute the SSM as pairwise cosine or Euclidean similarity between frames. Homogeneous sections form bright blocks along the diagonal; transitions create "checkerboard" patterns where two dissimilar blocks meet. Convolve the SSM diagonal with a checkerboard kernel to get a "novelty" function — peaks mark section boundaries (Foote 2000). ([Foote 2000 — FMP reference](https://www.audiolabs-erlangen.de/resources/MIR/FMP/C4/C4S4_NoveltySegmentation.html))

**Key implementation choice: operate on the beat-synchronous SSM, not the frame-level SSM.**  
Our benchmark confirms that:
- Frame-level SSM (5168×5168 for 120 s) → Foote convolution: **26.96 s** — too slow for interactive use.
- Beat-synchronous SSM (224×224) → Foote convolution: **0.45 s + 0.013 s = <0.5 s total** — perfectly acceptable.

Beat-synchronous features (`librosa.util.sync`) are the standard mitigation for the quadratic SSM cost.

**For loops (not just boundaries).** The SSM's off-diagonal stripes directly encode repeating sections: a stripe at lag=L means the passage at time t looks like the passage at time t+L. Loop candidates = positions where the diagonal-lagged SSM is high AND the loop duration is musically useful (1/2/4/8 bars). The `librosa.segment.recurrence_matrix(mode='affinity')` already computes this in affinity form.

**Kernel size trade-off.** A kernel of k beats corresponds to a sensitivity window of ~k bars (at 4/4, 120 BPM). k=8 detects 2-bar repeats; k=16 detects 4-bar sections. Recommendation: run two passes (k=4 for 1-bar loops, k=16 for 4-bar loops).

**Variable-tempo robustness.** Beat-synchronous SSM absorbs tempo drift automatically — the beat axis is in *beats*, not seconds. A `slow 2` ramp shows up as longer beat intervals in the time axis but the SSM similarity at the feature level remains correct. **This is the central advantage of beat-sync over frame-sync for livecoded material.**

**Features to use.** From our L0/L1 stack (`reference_audio_feature_stack`):
- **MFCC (13)** — timbral identity; best for detecting when the *instrumentation* repeats.
- **Chroma / CENS** — harmonic identity; loop detection in tonal material.
- **Stacked MFCC+chroma (25 dims)** — covers both; used in the 2025 MSA study ([PLoS One 2025](https://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0312608)).
- For stems (demucs): drums stem → MFCC+RMS works; bass stem → MFCC+chroma; other/vocals → full stack. Do NOT assume MFCC alone on bass covers timbre (cf. `feedback_mastering_eda`).

**Accuracy.** The MSAF survey and the 2025 beat-MSA study report HR3F (boundary at 3 s tolerance) of ~69 % for the best beat-based methods on the SALAMI dataset. For loop *finding* (not boundary labelling) what matters is whether the *same* repeating region is flagged: SSM similarity > 0.7 on beat-sync MFCC/chroma is a reliable proxy. Perceptual validation by ear remains necessary (cf. project design philosophy).

### 2.2 Autocorrelation / Beat-Spectrum for Repeating Sections

**Principle.** The beat spectrum (Foote & Cooper 2001) = the mean of the SSM diagonals as a function of lag. Peaks at lag L mean the music repeats every L beats — directly encoding loop period candidates. `librosa.segment.recurrence_matrix` + `np.diag` sum is the quick implementation.

For *loop point* finding (not period estimation), autocorrelation of the waveform or of a feature envelope around a candidate region can score how well the tail of a loop matches its head — i.e. will it click-free? Cross-correlation of the waveform windows at candidate start/end gives a seam-coherence score that complements the SSM structural score.

Our micro-benchmark shows scoring 74 4-beat candidates in a 30 s stem takes < 1 ms — trivially fast even with hundreds of candidates.

### 2.3 Laplacian Segmentation (`librosa.segment.agglomerative`)

**Principle.** McFee & Ellis (2014) — embed beat-sync features in a graph Laplacian, cluster with agglomerative hierarchical clustering. Returns a flat label sequence (k clusters = k musical sections). Good for *labelling* sections (verse/chorus/bridge) but the k must be supplied; does not output boundary confidences, so less useful for ranking loop candidates.

**Suitability.** Useful downstream for *grouping* found loop candidates by musical section (e.g. "these 3 candidates all come from the same chorus"). Not the primary loop-finding engine. Timing: < 5 ms for beat-sync features on a 120 s stem.

### 2.4 Graph-Based Audio Looping (Roma, Tremblay, Green — DAFx 2021)

**Algorithm.** Builds a directed similarity graph from spectral frames where edge weight = inverse feature distance; traversal allows non-linear looping and granulation. Beat spectrum from SSM diagonals drives onset detection; cluster analysis finds stable homogeneous zones. ([DAFx 2021 paper](https://dafx2020.mdw.ac.at/proceedings/papers/DAFx20in21_paper_41.pdf))

**Suitability.** Designed for creative real-time granulation/looping (FluCoMa context), not for batch extraction of pre-cut WAVs. Its loop transitions are probabilistic (graph traversal), not deterministic start/end pairs. Intellectually interesting but not the right abstraction for our "candidate → human picks → export WAV" flow. **Defer.**

---

## 3. Seamless-Cut Technology

### 3.1 Zero-Crossing Snap

Standard: nudge the candidate start and end samples to the nearest zero-crossing in the waveform, preventing a DC-offset click at the loop seam. `librosa.zero_crossings` computes the ZCR boolean array; argwhere over a ±N sample window around the beat-frame sample gives the nearest ZC. Takes < 1 ms. ([DSP Related discussion](https://www.dsprelated.com/thread/15880/calculating-optimal-loop-point-in-audio-sample))

**Limitations.** ZC snap fails for sustained bass/drone material where the waveform rarely crosses zero (e.g. a low-frequency sine oscillating slowly). In that case, a short equal-power crossfade (2–10 ms) replaces ZC snap. Strand B owns the crossfade policy.

### 3.2 Beat-Synchronous Slicing

Start/end samples derived from `librosa.frames_to_samples(beat_frames)`. This guarantees bar-aligned boundaries independently of ZC snap; ZC snap is then a local ±N-sample refinement within the beat-aligned window. The two are complementary, not alternatives.

### 3.3 PyMusicLooper (arkrow)

An active open-source project specifically for finding seamless loop points. v3.0 (2023) added "10× faster core loop search" and explicit zero-crossing alignment. Uses beat detection internally (skips beat analysis in brute-force mode). The scoring mechanism is not documented publicly; the author warns "the algorithmically chosen best loop point may not be perceptually good" — they recommend interactive review. ([PyMusicLooper README](https://github.com/arkrow/PyMusicLooper))

**Assessment.** Designed for game-audio style looping (music that loops forever at a single point), not for extracting multiple diverse loop candidates from a stem for use as Tidal samples. The interactive single-loop paradigm mismatches our "surface 20 candidates, human picks" design. **Useful only as reference; don't adopt as dependency.**

---

## 4. Existing Auto-Loop Extractor Tools / Repos Summary

| Tool | Core algorithm | Tempo drift | Stem-aware | Multi-candidate | Verdict |
|---|---|---|---|---|---|
| PyMusicLooper | Beat detect + ZC + waveform corr | No | No | Semi (pruning) | Reference only |
| librosa_loopfinder (Kexanone) | librosa-based, details private | Unknown | No | Yes (win_length) | Light candidate |
| loop_extractor (CurtisVonRubenhoff) | TF/Spleeter + ML | Unknown | Yes (via Spleeter) | Yes | Python 3.10 lock-in; avoid |
| MSAF (urinieto) | Foote, Serrà sf, OLDA, scluster | No (fixed tempo) | No | Boundary list | Useful for boundary comparison |
| BeatNet | CRNN + particle filter | Yes | No | Beats only | Heavy dep (madmom) |
| FluCoMa graph looping | SSM graph traversal | Yes | Yes | Probabilistic | Creative tool, not batch |

---

## 5. Comparison Table — Techniques

| Technique | Accuracy (approx.) | Latency on 120 s stem | Handles BPM drift | New heavy deps | Recommended use |
|---|---|---|---|---|---|
| `librosa.beat.beat_track` | Moderate (single-BPM) | 1.85 s | No — assumes constant BPM | None | **Avoid as primary**; fallback only |
| `librosa.beat.plp` + peak-pick | Good for local-tempo | 1.19 s | **Yes — per-frame** | None | **Primary beat grid** |
| `librosa.onset.onset_strength` | N/A (input feature) | 2.06 s | N/A | None | Required input for plp |
| madmom DBNDownBeat | High (MIREX #1 2016) | Offline + RNN overhead | No — DBN assumes stable metre | **madmom (NOT installed)** | Defer — NumPy 2.x incompatible |
| aubio beat tracker | Moderate | Real-time capable | No — window-local only | aubio (NOT installed) | Skip — covered by plp |
| BeatNet | High + streaming | <50 ms per frame | **Yes — particle filter** | madmom + torch | Defer — heavy; overkill for offline |
| SSM + Foote novelty (frame-level) | Good for boundaries | 26.96 s conv | Beat-sync absorbs drift | None | Avoid frame-level; use beat-sync |
| SSM + Foote novelty (beat-sync) | Good for boundaries | 0.46 s total | **Yes — beat-sync normalises** | None | **Primary section/loop detector** |
| `librosa.segment.agglomerative` | Good (section labels) | <5 ms | Yes (beat-sync) | None | Supplementary (section grouping) |
| Autocorr seam score (waveform) | Perceptual proxy | <1 ms/candidate | N/A | None | **Primary loopability score** |
| ZC snap | N/A (click removal) | <1 ms | N/A | None | **Required post-processing** |
| PyMusicLooper | Game-audio loops | Seconds | Unknown | librosa + soundfile | Reference only |

---

## 6. Recommended v1 Stack

### Rationale

The constraints drive the stack:

1. **BPM drift is real** — `beat_track` (single-BPM) is unsafe; `plp` (per-frame) is the only installed tracker that handles drift.
2. **Stay in existing venv** — librosa 0.11.0 + numpy 2.4 + scipy 1.17.1 + sklearn 1.9.0 are installed. madmom is incompatible with NumPy 2.x and NOT installed; do not add it.
3. **Reuse L0/L1 features** — MFCC + chroma are already computed by `sample_features.py`; the loop finder should read from the same pipeline, not fork it.
4. **Beat-sync SSM is the sweet spot** — fast enough (<0.5 s on a 120 s stem), absorbs tempo drift, directly encodes repeating structure.
5. **Human picks** — the machine enumerates candidates and scores them; no auto-apply. The score must be *explanatory* (seam quality, SSM similarity, bar alignment) not opaque.

### The v1 Pipeline

```
stem.wav (demucs output, any orbit)
  │
  ├─ 1. ONSET STRENGTH  librosa.onset.onset_strength(y, sr)
  │
  ├─ 2. VARIABLE-TEMPO BEAT GRID  librosa.beat.plp(onset_envelope, sr)
  │      → peak-pick via scipy.signal.find_peaks(pulse, distance=min_bpm_distance)
  │      → beat_frames[]  (variable spacing, handles setcps drift)
  │
  ├─ 3. FEATURES  librosa.feature.mfcc(y, sr, n_mfcc=13)
  │              + librosa.feature.chroma_cqt(y, sr)
  │              → stacked (25, n_frames)
  │
  ├─ 4. BEAT-SYNC  librosa.util.sync(feat, beat_frames)
  │      → beat_feat (25, n_beats)  ← tempo-drift-invariant representation
  │
  ├─ 5. RECURRENCE MATRIX  librosa.segment.recurrence_matrix(beat_feat,
  │      mode='affinity', sym=True)   → R (n_beats × n_beats)
  │
  ├─ 6. FOOTE NOVELTY (two passes)
  │      k=4 kernel  → novelty_1bar   (1-bar loop candidates)
  │      k=16 kernel → novelty_4bar   (4-bar / phrase loop candidates)
  │      peak-pick on novelty → boundary_frames[]
  │
  ├─ 7. CANDIDATE ENUMERATION
  │      For each boundary pair (start_beat, end_beat) at bar lengths
  │      {1, 2, 4, 8} bars:
  │        - convert beat index → sample index
  │        - duration filter (exclude < 0.25 s, > 16 s for 120 BPM baseline)
  │
  ├─ 8. LOOPABILITY SCORE (per candidate)
  │      a. SSM diagonal score = R[start_beat, end_beat]  (0–1; how similar
  │         are start and end contexts in feature space)
  │      b. Seam coherence = cross-corr of waveform tail [end-N:end] vs
  │         head [start:start+N] (N = 2048 samples)
  │      c. ZCR at seam endpoints (proximity to zero crossing)
  │      final_score = 0.5*SSM_score + 0.3*seam_coherence + 0.2*zcr_snap_proximity
  │
  ├─ 9. ZC SNAP  nudge start/end to nearest zero-crossing within ±512 samples
  │
  └─10. OUTPUT  List[LoopCandidate] sorted by final_score descending
                  (top ~20, deduplicated by ≥50% overlap)
```

### Data model (sketch for strand D)

```python
@dataclass
class LoopCandidate:
    start_sample: int          # after ZC snap
    end_sample: int            # after ZC snap
    start_beat: int            # beat index in PLP grid
    end_beat: int
    bars: int                  # 1 / 2 / 4 / 8
    bpm_local: float           # median inter-beat interval in region → BPM
    ssm_score: float           # 0–1, recurrence similarity
    seam_score: float          # 0–1, waveform cross-corr at seam
    zcr_snap_samples: int      # how far we nudged start from raw beat edge
    loopability: float         # weighted composite
    stem: str                  # "drums" / "bass" / "other" / "vocals" / "mix"
```

### What to defer (not v1)

| Technique | Why defer |
|---|---|
| madmom DBN downbeat | NumPy 2.x incompatible; madmom NOT installed; accuracy gain on livecoded music unclear |
| BeatNet | Requires madmom + torch model download; overkill for offline batch |
| Laplacian segmentation (as primary finder) | Needs k supplied; not self-contained |
| Graph-based looping (FluCoMa) | Wrong abstraction for our export flow |
| "Beat This!" Transformer | No pip-installable checkpoint yet |
| ML-based loopability (neural) | Insufficient labelled ParVagues loop training data; build ground truth first |
| Autocorr-based loop period detection | Subsumes into beat-spectrum diagonal of SSM — add only if SSM misses periodic structure in low-onset stems |

### Biggest risk

**BPM-drift at the `setcps` jump boundary** is the single hardest failure mode. PLP's local tempo adapts but needs ~8 s of context; at the exact frame of a `setcps` change, beat frames may be off by 1–2 beats, producing a slightly off-bar loop candidate. Mitigation: widen the ZC snap window at candidates near detected novelty peaks (which correlate with `setcps` transitions), and let the human reject/adjust in the UI. The SSM seam score and seam-coherence score will be *low* at bad-seam candidates, naturally demoting them in the ranking without needing to detect the tempo-change moment explicitly.

---

## 7. Sources

- Ellis, D.P.W. (2007). "Beat tracking by dynamic programming." *JNMR* 36(1). Via [librosa docs](https://librosa.org/doc/main/generated/librosa.beat.beat_track.html)
- Grosche, P. & Müller, M. (2011). "Extracting predominant local pulse information." *IEEE TASLP* 19(6). Via [librosa.beat.plp docs](https://librosa.org/doc/main/generated/librosa.beat.plp.html)
- Foote, J. (2000). "Automatic audio segmentation using a measure of audio novelty." *IEEE ICME*. [FMP reference](https://www.audiolabs-erlangen.de/resources/MIR/FMP/C4/C4S4_NoveltySegmentation.html)
- Böck, S. et al. (2016). "madmom: a new Python Audio and Music Signal Processing Library." *ACM MM*. [paper PDF](https://www.cp.jku.at/research/papers/Boeck_etal_ACMMM_2016.pdf)
- Böck, S. & Schedl, M. (2011). "Enhanced beat tracking with context-aware neural networks." *DAFx*. [madmom downbeats](https://madmom.readthedocs.io/en/v0.16/modules/features/downbeats.html)
- Hydri, M.J. et al. (2021). "BeatNet: CRNN and Particle Filtering for Online Joint Beat Downbeat and Meter Tracking." *ISMIR*. [arXiv 2108.03576](https://arxiv.org/pdf/2108.03576)
- BIFF.ai (2024). "A rundown of open source beat detection models." [biff.ai](https://biff.ai/a-rundown-of-open-source-beat-detection-models/)
- McFee, B. & Battenberg, E. (2024). "librosa 0.11.0: time-varying beat tracking." [librosa example](http://librosa.org/doc/0.11.0/auto_examples/plot_dynamic_beat.html)
- Roma, G., Tremblay, P.A. & Green, O. (2021). "Graph-Based Audio Looping and Granulation." *DAFx*. [proceedings](https://dafx2020.mdw.ac.at/proceedings/papers/DAFx20in21_paper_41.pdf)
- Nieto, O. & Bello, J.P. (2015). "MSAF: Music Structure Analysis Framework." *ISMIR LBD*. [paper](https://ismir2015.uma.es/LBD/LBD30.pdf)
- Tan, X. et al. (2025). "A music structure analysis method based on beat feature and improved residual networks." *PLoS One*. [DOI 10.1371/journal.pone.0312608](https://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0312608)
- Arkrow (2023). "PyMusicLooper." [GitHub](https://github.com/arkrow/PyMusicLooper)
- Kexanone. "librosa_loopfinder." [GitHub](https://github.com/Kexanone/librosa_loopfinder)
- internal: `reference_drops_detection_prior_art.md` — Foote novelty on stem-activity SSM, BPM-drift footguns, v1 threshold values.
- internal: `reference_audio_feature_stack.md` — L0/L1/L2 hierarchy, MFCC/chroma discriminators, installed deps.
