Test pipeline

Fifteen thousand comparisons, every week, against something that is not crête.

The harness is a separate pytest repository driven by Jenkins across two Linux agents of different architecture. Corpora live in object storage and are staged per suite. A run does not report pass or fail alone — it reports how many values were compared, how far each was from its reference, and whether any metric worsened against the stored baseline.

233

tests, weekly #106

217

passed · 0 failed · 16 skipped

15,483

metric comparisons, 65 albums

flagged (0.41 %), none catastrophic

catastrophic breaches, ever

01 — The pipeline

Corpus to verdict

01 Corpus

Real albums, never synthesised: PCM at six rates, DSD64→512, MQA, cue monoliths, mono stems, 5.1, 7.1, quad, DTS:X, Blu-ray disc audio.

02 Object store

One zip per suite with a pinned sha256, staged and denested on the agent. Corpus shape is asserted, not assumed — depth distribution is checked before a run.

03 Agents

Two Linux machines, aarch64 and x86-64, each building its own target natively. Per-suite capability flags decide which extra binaries get built.

04 Suites

Each suite reads crête’s JSON and compares every field against its reference rows. A suite that produces zero comparisons fails.

05 Report

Allure archive per run: comparison counts, per-metric max and mean |Δ|, flag lists, a top-20 worst-delta table.

06 Baselines

A fully green run snapshots per-album per-metric max |Δ|. Later runs fail only on a worsening beyond baseline + margin, with an absolute backstop for gross breakage.

Two report-reading rules were bought with real incidents and are now enforced in code. Read the comparison count before the deltas — one suite once passed while gating a single track out of twelve, because a filename-extension strip cut classical movement titles at the wrong dot. And read the average delta column, not the flag column — a metric with a wide tolerance can be the worst in the run while flagging almost nothing.

02 — Oracles

Four kinds of truth, ranked

No single reference is trusted. Each tier answers a question the others cannot, and the interesting findings have all come from disagreements between tiers.

Tier 1 · primary

MAAT DROffline MkII

The commercial reference meter, and the sole external oracle for PCM and DSD. Reference rows are keyed to a crête version and decode configuration and stored alongside the values. DSD rows are produced by measuring crête’s own 32-bit-float exports, because MAAT cannot open a .dsf — which is why the exporter itself is gated.

Tier 2 · cross-check

foobar2000 DR Meter

Retired as a routine oracle, kept as an independent second implementation. That distinction earned its keep: on 264 per-channel values foobar and MAAT agreed with each other to a mean of 0.006 dB while crête sat 0.095 from both — proof the residual was crête’s, not reference noise. It is back for one rule MAAT cannot adjudicate: MAAT reads at most two channels, so no MAAT row holds an LFE, and foobar is the only measured implementation of the all-channel rule --dr-lfe include reproduces. Validated before any row was written, pairing by track number: album DR matches exactly on all three 5.1 albums, per-channel DR to a mean of 0.036 dB over 264 values, and on 4.0 — where the two rules are the same rule — every track integer matches and the flag is byte-inert. Added after weekly #85; first through a weekly in #86, and passing in every weekly Jenkins holds since, to #106, but one: #99, a manual run on the experimental foobar block grid, where a quad track sat 0.007 dB either side of a .5 and the integers split. The harness now judges such a flip on raw DR.

Tier 3 · analytic

Synthetic self-checks

Signals whose answer is known on paper, needing no corpus and no harness. check_fix12.sh asserts that equal-peak / half-RMS channels differ by exactly 6.02 dB, and that raw track values 7.6/7.6/7.1 give album DR7. It passes on the fixed build and fails on the broken one. check_lfe_peak.sh, added in 0.19.0, holds a 5.1 with every channel at −20 dBFS and the LFE at −3 to the true-peak bound, and asserts its own premise — the joint sample peak must move ~17 dB between LFE settings, or the fixture has stopped testing anything. check_cue_multifile.sh, added in 0.19.1, cuts two synthetic WAV sides, each two tones at distinct levels, so a track’s sample peak names the exact file and half it was cut from; it passes, and fails on 0.19.0. This tier is what catches the input no corpus holds: that synthetic exposed the joint true-peak defect that every real album hides, and the DST decoder’s frame-consumption assertion caught a decoder that satisfied every other plausibility check available.

Tier 4 · controls

crête versus crête

Suites where both sides are crête, so the answer must be zero: the same audio in five containers, a cue monolith against per-track files, FFmpeg-decoded ALAC against the native FLAC decode, DSF against DFF. These are what isolate a measurement bug from a decode bug.

03 — Suites

What each suite gates

Suite	Gates	Oracle	Corpus	Latest
QUALIFICATION	The full 21-metric set on PCM	MAAT	11 albums — Thriller ×5 rates, East Meets West ×6 rates	green
SMOKE	The same metric set on a second, wider PCM corpus	MAAT	11 albums, 153 tracks, 44.1 to 96 kHz — run whenever the Monday cron fires	3366 comparisons, 33 flags
DSD64 / 128 / 256 / 512	DSD decode + measurement at 44.1 and 352.8 kHz out, the band limit pinned	MAAT, via `dsd2wav` exports	8 albums across four rates	green
MQA	Detection and measurement on MQA-encoded releases	MAAT	MQA-CD 16-bit and 24-bit hi-res	green
MONO	The 1-channel path and the dual-mono stereo path	MAAT	Turtles 96/24 dual-mono + its `[M]` stems	341 comparisons, zero flags
SURROUND	5.1 channel weighting, LFE exclusion, per-channel DR	MAAT on 2-channel stems	3 × 5.1 albums, 45 tracks × 6 channels	1620 comparisons, 3 flags
QUADIO	The 4-channel layout case	MAAT	1972 quad Blu-ray Audio	240 comparisons, zero flags
DTSX	DTS:X bed decode at 7.1	MAAT on four stem groups	Classical Blu-ray, 12 tracks	350 comparisons, 1 flag
DISCAUDIO	TrueHD / DTS-HD MA / AC-3 decode and stream selection	Self-carrier parity + snapshot	Six audio streams off one UHD remux	8/8
FORMAT	Container-independence: identical metrics across WAV, FLAC, AIFF, AIFF-C `sowt`, RF64, Wave64	crête vs crête	6 albums transcoded every way	704 comparisons, Δ 0.000
CUE	Cue slicing against the equivalent per-track album	crête vs crête	Monolithic albums with sheets	286 comparisons, Δ 0.000
FFMPEG_PARITY	ALAC through FFmpeg against the native FLAC decode	crête vs crête	5 ALAC albums against their FLAC equivalents	1660 comparisons, Δ 0.000 — 3 tracks under 10 s carry no DR since 0.20.0
DSD_PARITY	The DSF reader against the DFF deinterleaver	crête vs crête	One album, deliberately — it proves a property of the code	264 comparisons, max Δ 0.06 (Min PSR); DR 0.000
DSD_ROUNDTRIP	The `dsd2wav` exporter that produces every DSD reference row	crête vs crête, two-tier	Thriller DSD64 at 44.1 kHz; since 0.22.0 also its shortest track at 352.8 kHz, band-limited and `full`	3/3, Δ 0.000; order statistics to 4.5e-07 dB
SACD	The Scarlet Book walk, both areas, and the DST decoder	Recorded values; `.dsf` extraction, not yet wired	A reduced DSOTM image, 229 MB, plus an ISO9660 negative control	6 passed / 2 skipped
DVD	Title sets read as one stream, the group-change stop, stream selection	crête vs crête — each disc’s lossless stream against the same audio as FLAC	Pack-aligned byte ranges of one DVD-Video and one DVD-Audio	12/12 — 10/10 on its first weekly, #85; the foobar tier added the other two
INTEGRITY	The damaged-input contract: warnings, exit 2, and no crash	None needed — synthetic, seeded, redistributable	Built from a fixed seed into a deterministic zip	19/19
CROSSARCH	Bit-identity between the two architectures	Agent vs agent	Six disc-audio carriers	273 fields, 0.000e+00

Ordering matters and is enforced: DSD_ROUNDTRIP shares DSD64's corpus so it must run after it, and the dsd2wav build must come last in the manual build stage because branches above it may clean the tree.

Weekly #77 is the first run on 0.17.0 and the first to carry INTEGRITY and SACD. The result worth recording is not that it passed. It is that all fifteen of the per-suite detailed reports are identical to the ones the last archived weekly produced — same 11,931 comparisons, same 44 flags, same per-metric maxima and averages, same two album mismatches — with a single added header line as the only textual difference anywhere. #73 recorded 0.13.1 and #77 records 0.17.0, so that identity spans five releases: the RF64 and Wave64 containers, the joint true-peak LFE guard, the DST decoder, the shared input collector with its natural-order fix, and the integrity checks. Each was argued to be a no-op on this corpus when it shipped. This is the corpus agreeing, on every number it holds.

Weekly #85 is the first run on 0.18.0 and the first to carry DVD. The same fifteen reports are identical to #77's below the header — the same 11,931 comparisons, 44 flags and two album mismatches — so the identity now spans the DVD title-set reader and the change that makes every FFmpeg decode take its channel layout and rate from the first frame. The new suite went 10/10. Two runs sit between them and neither is counted: #81, the first on the new x86-64 default agent and the first on a cold cache in a long time, and #82 both went unstable on the same three tests. Staging, not measurement — crête was pointed at a directory holding only another directory, does not recurse, and said so. The check meant to catch it counted files recursively, and a recursive count cannot see a level of nesting.

Weekly #96 qualified the 0.20.0 tag. It carries sixteen reports: the fifteen above plus SMOKE, a second MAAT corpus of 11 PCM albums that the Monday cron runs. On the fifteen #85 also had, it compares 12,073 values over 52 albums and flags 44, as #85 did, with the same two album mismatches. The difference is a second cue album, 154 comparisons at Δ 0.000, and 12 fewer in FFMPEG_PARITY, where three tracks under 10 s no longer carry a DR. That rule moved two album values between #94 and #95: Kill Bill Vol. 1 in SMOKE read DR4 against MAAT's DR10 and now matches, and in CUE the per-track side of the multi-file album went from DR9 to DR8, agreeing with its cue sheet, which read DR8 both times. All sixteen reports are byte-identical below the header to #95's, the run on the short-track commit before the version bump. Of the weeklies Jenkins holds between #85 and #96, two went unstable, #87 and #92, both on the DVD bad_alloc that 0.19.5 fixed.

Weekly #106 qualified the 0.22.0 tag, and is the census on this site. Between it and #96 the totals did not move until 0.22.0: #104, which qualified 0.21.0, found all 15,439 values bit-identical to 0.20.0's, as the channel-mask change predicted, since the corpus holds no file whose mask disagrees with its count. In #106 eleven of the sixteen reports are still byte-identical below the header to #96's. The five that changed are the four DSD suites and the round trip. Album DR now matches on 59 of 60: Thriller_DSD64@352800, 16 against the reference's 15 in #96, matches; DSD64 flags fall from 19 to 5 and its four track-level DR mismatches, all on Thriller at 352.8 kHz, to none. One flag is new, a DSD128 track at 352.8 kHz whose DR LEFT is 0.53 off against a 0.50 tolerance. The only album mismatch left is BoneyM_10k_DSD128@44100, the .5 boundary at the default rate. The round trip grew from one case to three, which is the 44 extra comparisons.

One count in #106 is a harness fault, not a measurement. Against 0.21.0 the qualification reports 772 of 15,439 shared values moved, largest 4.831. 750 are the 352.8 kHz values in the four DSD suites, which the band limit was meant to move; no 44.1 kHz value moved. The other 22 are the round trip: all three of its cases recorded under one key in the measurement file, so the 352.8 kHz full values overwrote the 44.1 kHz ones. crete-pytest now keys the round trip by rate and bandwidth; the first weekly after it will show those 22 moving back. Of the weeklies Jenkins holds between #96 and #106, one went unstable: #99, a manual run with --dr-blocks foobar, failed two tests whose premises held only on the reference grid: one extra decoder frame on the DTS-HD MA carrier fell into a loud trailing block instead of forming its own, and a quad track sat 0.007 dB either side of a .5 against foobar. The harness now measures that carrier pair with the legacy block set and judges a quad integer flip on raw DR; #101, also on the foobar grid, passed. #97 to #101 kept no artifacts, only their test results.

04 — Case study

The block set: how a 33 % accuracy gain was decided

Up to 0.13.1 crête's 3-second block set differed from the PMF reference implementation in two places: it discarded the trailing partial block, and it gated blocks at a joint-RMS threshold. The reference keeps the tail, normalised by its own length, and has no gate. Both deviations shrink the block count, and since the top-20 % count is a floor division, a smaller count keeps fewer and therefore louder top blocks — raising the reference RMS and lowering DR. A structural low bias, worst on short tracks and sparse channels.

It was not shipped on the strength of that argument. Two full weekly runs, same binary, same corpus, back to back — #68 under the legacy block set, #69 under the reference one.

Block set	vs foobar	vs MAAT
legacy (≤ 0.13.1)	mean \|Δ\| 0.0598, bias −0.011	0.0628, bias −0.014
reference (default)	0.0355, bias −0.004	0.0339, bias −0.001

Weekly #68 → #69	Result
Weighted mean \|Δ\|, 2241 DR comparisons	0.0392 → 0.0262 (−33 %)
DR flags	29 → 19
Integer DR (PMF) flags	10 → 5
DR rows that regressed	not one
crête-vs-crête control suites	0.000, unmoved
Non-DR values that moved	none, anywhere

The control suites staying at exactly zero is what confirms the change is confined to block accounting rather than leaking into decode. And the reference block set reproduces a literal transcription of the reference implementation to 3.9e-05.

It also retired an earlier conclusion. Weekly #55 had recorded the surround residual as a short-track effect — the two shortest tracks in a 45-track corpus owned nine of the top twenty deltas. Under the reference block set the correlation between inverse duration and error falls from +0.465 to −0.044. It was never about track length; it was about what fraction of a track's blocks the old accounting threw away.

--dr-blocks legacy reproduces ≤ 0.13.1 bit-for-bit, for A/B work and for re-reading reference rows recorded under it.

05 — Forensics

Six more defects, and what caught each one

Each of these passed every gate at the time it was found. Most were caught by reading the deltas underneath a green run — and one was caught by a synthetic, because the corpus does not contain the case at all.

FIX 12 · 0.11.3

Per-channel DR shared one joint RMS

Left and right could differ only through the peak term, because both subtracted the same across-channel RMS. The error cancelled in the mean — which is exactly why it survived years of validation — but split wrongly per channel by up to 2.37 dB, often with the left-minus-right spread opposite in sign to the reference. Worse, crête emitted identical left and right values bit-for-bit whenever both channels happened to share a second-highest block peak. A per-channel metric cannot do that; a shared RMS term can.

Each channel now uses its own top-20 % RMS. The accumulation order of the old joint value was preserved verbatim so the gate stayed bit-identical.

Weekly #53 → #54	Before	After
DR LEFT flags	83	8
DR RIGHT flags	77	5
L−R spread error, median	up to 3.4 dB, sign inverted	0.000 dB
Max Δ, dsd512	2.09	0.15
Album mismatches	3	2

Where crête still reported left equal to right, so did MAAT — those tracks are genuinely dual-mono.

0.13.0

The top-20 % RMS averaged amplitude, not power

The PMF procedure averages power — a quadratic mean. crête averaged amplitudes. Since the arithmetic mean never exceeds the quadratic mean, the reference RMS was always biased low and DR was therefore always biased high, by an amount that grows with block-to-block RMS variance: about 0.03 dB on dense stereo, up to 1.47 dB on sparse channels.

That variance dependency is why it hid for the project's entire life. On left and right it sat an order of magnitude below every tolerance in the harness. A 5.1 corpus exposed it, because centre, surround and LFE channels are sparse and high-variance — but it was never a surround bug. It was a core defect that surround made visible.

Found by cross-checking a 5.1 corpus against two references at once, which is the only reason it could be attributed: the two oracles agreed with each other to 0.006 dB while crête sat 0.095 from both.

264 per-channel values	mean \|Δ\|	bias
arithmetic (shipped ≤ 0.12.1)	0.095	+0.056
quadratic (PMF)	0.061	−0.011

Weekly #56, mean \|Δ\| vs MAAT	#55	#56	Change
DR (PMF)	0.0265	0.0244	−8 %
DR LEFT	0.0588	0.0427	−27.5 %
DR RIGHT	0.0588	0.0420	−28.5 %
DR (ch)	0.0951	0.0584	−38.5 %

Net signed bias over integer DR mismatches went +9 → 0. The "always reads high" signature, open since an earlier fix, is gone. Zero non-DR metrics moved anywhere in the run.

0.15.0

The corpus could not have found this one

The joint sample peak has skipped unscored channels since 0.12.0. The loop behind the joint true peak had no such guard, so the two were computed over different channel sets on every multichannel file, and --dr-lfe reached one of them and not the other. PLR, which divides one by the other, mixed an LFE-inclusive numerator with an LFE-excluded denominator.

True peak is ≥ sample peak by definition and the excess is inter-sample overshoot — a few tenths of a dB. A 17 dB gap is not a reading anyone can interpret, and that is what a synthetic 5.1 produced the moment one was built for it.

No corpus file could have shown it. The LFE has to be the loudest channel, and in real material it never is: on the 5.1 master used to check the fix, the LFE sits at −11.14 against front left at −1.67. Both invariants were then measured whole-JSON on all six carriers of that disc — --dr-lfe include reproduces 0.14.0 exactly, 6/6, because the old behaviour was unconditional inclusion; and at the default setting nothing moved either, 6/6. That is not "the bug did not matter". It means the corpus never contained the case.

Synthetic 5.1 · all channels −20 dBFS, LFE −3	0.14.0	0.15.0
Joint sample peak	−20.000	−20.000
Joint true peak	−2.986	−19.992
Implied overshoot	17.014 dB	0.008 dB

No reference row moves with it either, and the reason is structural rather than lucky: MAAT reads at most two channels, so every referenced true-peak row in the corpus is stereo, mono or per-channel — all cases where the guard is a no-op or does not apply.

The lesson the harness took is the one its Tier 3 oracle exists for. A corpus of real albums proves what happens on real albums; it cannot prove what happens on the input nobody has.

From 0.15.0 to 0.18.1 the fix itself had no regression coverage — the measurement above was taken once, by hand. Since 0.19.0 check_lfe_peak.sh gates it on every build, and it was checked against a 0.14.0 build before it was trusted: the default setting fails there with a 17.01 dB gap, exactly the figure recorded when the fix landed, while --dr-lfe include passes — because include is the old unconditional behaviour, which re-confirms that every pre-0.15.0 number is still reachable through it.

0.17.0

A broken file was scored as if nothing had happened

The worst thing a meter can do is not refuse a file. It is to measure a broken one and print a number. crête did exactly that: a FLAC with one bad sector was scored, averaged into the album DR and returned exit 0 with no error field anywhere in the JSON — while the reference flac decoder refused the same file outright. A single flipped byte was worse, because it produced numbers identical to the clean file. No amount of reading the measurement could have found it.

Four checks now run, into one reporting channel. FLAC frame CRC-8 and CRC-16 are verified rather than read and discarded — the only detector that catches an altered sample value. Declared length is compared with decoded length on FLAC STREAMINFO, the WAV/RF64/Wave64 data chunk and AIFF COMM/SSND. A sample peak above 0 dBFS on integer PCM is impossible arithmetic and so is proof of a bad decode whatever the checksums said. And JSON carries a warnings array that is always present, so an empty array is the positive statement "nothing was wrong" rather than the absence of evidence a missing key would be.

The file is still measured. crête is a meter, not a repair tool: it reports what the bytes decode to and marks every affected number untrustworthy. That separation is what keeps the intact-file path bit-identical by construction — verified on 62 files across nine codecs and five channel layouts, whole-JSON against a build of 0.16.1, 0 differing.

Building the corpus immediately found a reachable segfault: a truncated AIFF walked off the end of the buffer, because the chunk walker used file-supplied lengths and an SSND offset without bounds-checking either — the same defect the RIFF walker had fixed in 0.14.0, in the decoder that never got the same treatment.

4-track album, one bad sector	0.16.1	0.17.0
Album DR, clean copy	DR13	DR13
Album DR, damaged copy	DR9	DR9
Sample peak, damaged	+48.16 dBFS	+48.16 dBFS
Reported as damaged	no	yes
Exit status	0	2

Corpus and suite	Result
`integrity` suite on 0.17.0	19/19
Same suite on 0.16.1	13 fail
Fuzz cases, five native formats	240, none crash
Truncation offsets swept per format	21
Reference files tripping a check	none, in 51 albums

WAV and AIFF carry no checksum, so altered sample values in them are undetectable — by crête or by any other meter — because a run of zeros inside a data chunk cannot be told apart from digital silence the artist put there. That blind spot is asserted rather than glossed: the suite requires a WAV with 4 KB of zeros written into its audio to come back clean, so the limitation stays visible in a green run.

Rejected

Band-limiting before measurement: tested, then refused

DSD read +2 against the reference at 352.8 kHz output on the worst album in the corpus. The obvious fix — filter 20 Hz to 20 kHz before measuring, since only audible content matters — was implemented as an experiment and measured. It works: band-limiting removes about 78 % of the inflation and returns both offending tracks to their 44.1 kHz integer.

It still was not built. It lands on no oracle — bandpassed reads DR12–13 where the reference at that rate reads 14, converging on neither. Going from 12 to 48 dB/oct barely moved it, so the residual is not filter slope but the sheer size of DSD noise-shaping above 30 kHz, which needs brickwall-grade attenuation. Which is precisely what decimating to 44.1 kHz already does, with a 641-tap filter, by default, at full parity. A flag would have been a worse version of the default path.

Thriller — Beat It, DSD64	raw DR	int	sample peak
@44100 (default)	11.665	12	−2.654
@352800 unfiltered	15.581	16	+1.706
@352800 + 12 dB/oct	12.533	13	−0.985
@352800 + 48 dB/oct	12.462	12	−1.002
MAAT @352800	—	14	—
MAAT @44100	—	12	—

The unfiltered 352.8 kHz sample peak is above full scale — retained noise-shaping alone drives the meter over. Ruled out as an explanation: inter-sample peak resolution. At 44.1 kHz the 4×-oversampled true peak is −2.630 against a sample peak of −2.654, so there is essentially no overshoot to find.

Overtaken in 0.22.0, on different terms. The band limit now lives in the decoder — --dsd-bandwidth, a 50 kHz linear-phase low-pass by default at 176.4 and 352.8 kHz — so the exports the reference measures carry it too, and the 352.8 kHz references were re-recorded with it. That answers the objection above: the filtered number now has a reference. Weekly #106 matches album DR at 352.8 kHz on all eight DSD albums, and --dsd-bandwidth full reproduces the unfiltered row. The reference's DR14 is explained as well: synthetic probes in October showed that MAAT pins a block peak above 0 dBFS to 0, so on over-full-scale material its DR drops by the overshoot of the second-highest block peak, about 1.6 dB here. 44.1 kHz remains the only parity rate.

Gated weekly

Cross-architecture identity is exactly zero

crête pins -ffp-contract=off. Without it the compiler fuses multiply-add into a single FMA instruction on architectures that have one and not on those that do not, so the ARM binary diverged from the x86 one at ULP level inside the K-weighting biquads — breaking the bit-reproducibility claim. For a long time that was an untested assertion. Now the two agents are compared directly, every week.

A genuine platform finding fell out of the same work: Linux and macOS differ by one ULP on 20·log10(x), identically on both architectures. That is libm rounding, not contraction — so a macOS-produced JSON cannot be equality-compared against a Linux one.

aarch64 vs x86_64 — 273 lossless fields

s01_truehd_atmos_7.1_48k.mkv  TRUEHD  EXACT  0.000e+00
s02_ac3_5.1_48k.mkv           AC3     tol    8.690e-07
s03_lpcm_5.1_96k.flac         FLAC    EXACT  0.000e+00
s04_dtshd_ma_5.1_96k.mkv      DCA     EXACT  0.000e+00
s05_lpcm_2.0_192k.flac        FLAC    EXACT  0.000e+00
s06_dtshd_ma_2.0_96k.mkv      DCA     EXACT  0.000e+00

Lossless carriers are asserted exact; the lossy AC-3 decoder is allowed 1e-2 for per-architecture SIMD. A lossless-tier failure on another agent is the identity question itself, not noise — it must not be loosened to pass.

06 — Where the reference is wrong

Three disagreements that turned out not to be crête's

LRA: the reference disagrees with itself

Nine LRA flags on one album, the largest non-DSD residual on record, closed as external. crête passes all six official EBU Tech 3342 compliance vectors within tolerance and its LRA is rate-invariant to ≤0.06 LU. MAAT's own LRA moves by up to 2.77 LU across sample rates of the same album — more than the entire crête-versus-MAAT disagreement — while the short-term loudness it is derived from moves ≤0.01 LU. crête agrees with MAAT better than MAAT agrees with itself.

Album DR cannot be matched by rule choice

MAAT prints per-track integers and an album value, so its aggregation rule is derivable with crête out of the loop. Rounding the mean of its own integers reproduces its own album DR on only 37 of 41 albums — and three exact .5 means round two ways. No rounding rule on integers can produce that, so MAAT averages internal raw values it never exports. crête averages raw and rounds once, matching the reference implementation's source.

A reference decode bug on a valid file

One track's FLAC verifies against its own embedded checksum and crête decodes it bit-perfectly, matching a five-rate consensus — yet MAAT deterministically mismeasures RMS, LRA and DR on that encoding, and measures a re-encode of identical audio correctly. The corpus was re-recorded from the new render; the original is kept as a minimal reproduction.

A fourth is a measurement artefact rather than a bug: on an all-zero channel the two tools print their own floors, which manufactures a 20 dB "failure" out of digital silence. The suite now asserts DR equal to zero on both sides rather than skipping the case.

The reference, modelled. In October 2026 MAAT DROffline MkII v2.2.3 measured about 630 synthetic probe files at 44.1 to 384 kHz, and a model of its TT DR now reproduces all 1256 per-channel values within its 0.01 display step, 81 % exactly. Its blocks are 3 × 44160/44100 s, 3.004082 s at every rate, so the grid drifts 4.08 ms per block against a 3.000 s one — 245 ms after 60 blocks, which is why long tracks disagree most. It pins block peak and RMS to −80…0 dBFS, truncates DR to two decimals and rounds half to even. Its true peak oversamples 4× at 44.1 and 48 kHz, 2× at 88.2 and 96 kHz and not at all from 176.4 kHz, and it computes loudness only at a fixed list of rates that leaves out 352.8 kHz. Its K-weighting follows the analogue curve rather than BS.1770's digital filter, which reads 0.04 to 0.05 LU low at 44.1 and 48 kHz; crête's LUFSi residual against MAAT in weekly #106 is an average |Δ| of 0.040 dB over 430 comparisons. None of it is adopted: crête stays on the published procedures, and what the harness should do with the model is undecided.

07 — Known limitations

Everything currently open, with the number attached

This is the complete list as of 0.23.0. Items are here because they are measured and unresolved, not because they are unexamined.

Limitation	Measurement	Status
352.8 kHz DSD carries no parity claim	Up to 0.21.0 DR read +1…+3 higher at 352.8 kHz than at 44.1 kHz, because the decimation cascade kept the modulator’s shaped ultrasonic noise; in weekly #96 `Thriller_DSD64@352800` mismatched at album level and on 4 of 9 tracks, maximum integer delta 2. 0.22.0 band-limits the decode at 50 kHz by default, and in weekly #106 album DR at 352.8 kHz matches the reference on all eight DSD albums; the only DR flag at that rate is one DSD128 track’s DR LEFT, 0.53 against a 0.50 tolerance. It is still not parity. A 50 kHz band keeps 20 to 50 kHz content the 44.1 kHz decode removes, the corner is the same for every DSD rate so DSD256 and DSD512 lose clean content above it, and MAAT computes no loudness at 352.8 kHz at all. At 44.1 kHz the continuous per-track DR comparison flags nothing on any of the 8 DSD albums — in weekly #106 the only `DR (PMF) raw` flag in the whole run is a PCM track in SMOKE — and album DR matches on 7 of 8. The exception is not rate-related: `BoneyM_10k_DSD128@44100` misses a `.5` boundary by 0.020 dB, which is an accuracy question and no aggregation rule can fix it.	Documented, deliberate
Min PSR diverges from the reference	Average \|Δ\| 0.574 dB across 573 comparisons in weekly #106, max 3.27 — about ten times the next-worst metric, LRA at 0.058. MAAT’s figure was reverse-engineered as a 0.5 dB/s decaying peak-hold; crête implements AES eBrief 373. A decode-versus-formula experiment proved it is a formula difference, not a decode artefact.	Closed as won’t-fix
Disc audio and SACD have no external oracle	TrueHD, DTS-HD MA and AC-3 are gated against crête’s own snapshot and against the LPCM carrier of the same master — where every metric matches exactly, with only RMS moving by 1.2e-04 dB from one frame of decoder tail. DST sits in the same position. For SACD the right comparison is written and is the strongest gate in that suite — crête on the ISO must equal crête on the `.dsf` extracted from the same disc, two independent container readers over one decode chain — but it is not running in CI: it needs the DSD64 corpus staged beside the SACD one, and wiring that through the corpus-reuse mechanism would make the whole suite skip whenever the sibling was not ready. Those are the 2 skips in its 6-passed / 2-skipped result. DVD is the partial exception: foobar cannot open a `.vob` or `.aob`, but the suite already requires crête through the container to equal crête on the FLAC of the same audio exactly, so foobar measuring that FLAC reaches the container decode transitively — DR9 against DR9 on the DVD-Video LPCM, DR13 against DR13 on the DVD-Audio MLP. That tier was added after weekly #85 and has passed in every weekly since, #86 to #106.	Open, not a blocker
The SACD suite gates drift, not correctness	Closed since 0.16.1: the reader and the DST decoder had no automated test of any kind until 0.17.0, which is uncomfortable for code whose failure mode is silence — each of the three format traps found while writing it produced audible output and a plausible DR. The suite now measures a reduced image, 229 MB instead of ~4 GB, every audio sector the disc’s own bytes and only the extent fields of the TOC rewritten; the builder’s `--verify` measures every kept track in both the reduced and the full image and requires identical results, so its own address arithmetic cannot certify its own mistake. 6 passed / 2 skipped on its first weekly, #77, and in every weekly since, through #106. Until the `.dsf` tier runs, what it gates is drift against recorded values rather than correctness against an oracle.	Gated, narrowly
A DVD-Audio title set measures as its first group	A title set can hold several mixes — Rumours carries a 96 kHz 5.1, a 96 kHz stereo and a 48 kHz 5.1 version in one, 39 tracks — and the group boundaries do not align with the file fragments. crête measures the first group as one stream, warns, and exits 2 rather than metering through the change. Since 0.18.1 `dvda-info` reads the titles and track lengths exactly, but durations do not give byte offsets in variable-rate MLP, and the table that would is a hypothesis until it meets the same arithmetic proof. So there is no per-track or per-title DVD-Audio measurement yet.	Open, tables parsed
A `.vob` lists one phantom audio stream	The compact FFmpeg cannot type the MPEG-2 video, content-probes it and matches it as MP3: three audio streams list as four, the extra decoding to 0.34 s at 16 kHz with a +23 dBFS peak. Adding the video parser and then the decoder changed nothing, and it cannot be filtered crête-side because MPEG audio is a legal DVD format. Automatic selection and every genuine `--stream` index are unaffected; the suite asserts those instead of an exact stream count.	Known, left alone
One LFE two oracles disagree with	DSOTM 5.1 `08 - Any Colour You Like`, LFE channel: foobar2000 reads 10.47 against crête’s 9.24, a 1.226 dB gap on a channel whose next-worst peer is 0.892 and everything else under 0.38. It is the same track and channel the MAAT stem comparison already flags as the one surround residual that never resolved, 1.38 → 1.23 across the quadratic-mean and block-set corrections. Two independent references, different paths, landing on one LFE makes it a lead rather than reference noise.	Open, a lead
DST decoding is slow	Roughly 2× realtime for six channels — a 43-minute 5.1 disc takes about 20 minutes. DST interleaves every channel through one arithmetic decoder, so no parallelism exists inside a frame; the prediction history is reinitialised per frame, so frames are independent and could be decoded in parallel. Not done.	Open, known fix
Altered samples in WAV and AIFF are undetectable	Neither container carries a checksum, so a changed sample value cannot be caught — by crête or by any other meter — because a run of zeros inside a `data` chunk is indistinguishable from digital silence the artist put there. Only declared length and the 0 dBFS range check apply. FLAC is the one native format that can prove itself, through frame CRC-8 and CRC-16. The `integrity` suite asserts the blind spot: a WAV with 4 KB of zeros written into its audio must come back clean, so a green run keeps the limitation visible rather than assuming it away.	Format-limited, asserted
Object audio is measured as its bed	No Atmos or DTS:X renderer exists in the chain. The 7.1 TrueHD bed is what a non-Atmos playback chain delivers, and since 0.15.0 crête names the profile and marks the layout as a bed rather than leaving it to be inferred. Do not read a bed measurement as an Atmos measurement.	Out of scope, labelled
Auro-3D cannot be identified	FFmpeg reports an Auro carrier as plain DTS-HD MA, so the profile field cannot see it. On the one disc measured the height channels were in the carrier’s low bits — real enough to find, at 803,853 non-zero LFE samples within ±24 against the DTS:X carrier’s exact zero — but reading that is inference, not a bitstream field. If it is ever built it belongs in the confidence-scored forensics namespace, never in the metrics. A discrete Auro-3D file — one PCM channel per speaker — is a different case: it declares its layout in its channel mask, and since 0.21.0 crête labels and weights it from that.	Not attempted
Masked speaker layouts are gated synthetically	Since 0.21.0 a file’s declared channel mask decides labels, LFE and weights when it describes a layout the count does not — Auro-3D 8.0 was read as 7.1 before, with a surround taken for an LFE. The weekly corpus holds no masked file that disagrees with its count, so the weekly does not exercise the new path: `scripts/check_channel_mask.sh` gates it with synthetic Auro 8.0 and 9.1 files whose loudness is arithmetic. The weights are a stated choice — every height channel at unity — and FFmpeg’s `ebur128` weights the top-back pair +1.5 dB, so on height layouts the two read differently. The DSF channel-type field and the DFF channel chunk are not read yet, so a 4-channel DSF declaring L R C LFE is still measured as quad.	Open, synthetic only
The DSD round trip shared one measurement key	0.22.0 gave `DSD_ROUNDTRIP` two 352.8 kHz cases beside the 44.1 kHz one, and all three recorded under the bare album key, so in weekly #106 the 352.8 kHz `full` values overwrote the 44.1 kHz ones in the measurement file and 22 values were counted as moved that had not — `The Girl Is Mine` reading its unfiltered 352.8 kHz DR. The suite itself passed: each case compares within itself. crete-pytest `098ede6` keys the 352.8 kHz cases by rate and bandwidth; the next weekly should show those 22 moving back, and has not run yet.	Fixed, awaiting a weekly
The 0.19.5 duration guard, confirmed on Linux	Weekly #87, the first on 0.19.0, and #92, on 0.19.4, both failed the DVD suite’s group-change test with `std::bad_alloc` on the Linux agent: a wrapped MPEG-PS timestamp declared 95 268 s for 12 MB of MLP, and the 0.19.0 decode reserve asked for about 40 GB per channel — which Linux refuses and macOS maps lazily, so the suite had passed on the development Mac. 0.19.5 rejects a duration the file is too small to hold. It was verified on macOS at release — DVD suite 12/12, seven FFmpeg carriers byte-identical with unchanged peak memory — and weekly #93, the first on 0.19.5, is the Linux confirmation: DVD 12/12 on the x86-64 agent, group-change test included, and again in every weekly since, through #106. The DVD suite runs on the x86-64 agent only.	Closed, weekly #93
Memory detection has not run under a limit	0.19.0 fits the worker pool to free memory, honouring a container limit. The Linux paths have now run: in weekly #96 `crete --version` reports `6 cpu (sched_getaffinity), 15 GB ram, 14 GB usable (/proc/meminfo MemAvailable)` on the x86-64 agent and `4 cpu`, `16 GB`, `15 GB usable` on the aarch64 one. Neither agent reports a memory or CPU limit, so the cgroup v1 and v2 reads — the case with the most value, since it is the one the old one-per-core default got wrong — still have never returned a limit, and Windows `GlobalMemoryStatusEx` has never been run: the Windows job does not print it. There is no admission control either: the pool is sized once, up front, and a worker heavier than its estimate is not throttled mid-run.	Open, cgroup unverified
The packages are smoke-tested, not measured	Since 0.19.2 `crete` and `crete-gui` are published for openSUSE, Fedora, Debian, Ubuntu and Arch, and the only check inside each package build is that the binary runs and reports the version it was packaged as. The Homebrew `crete` formula, since 0.19.3, carries a test that measures a 20 s 1 kHz sine at −6 dBFS and asserts DR0, a −6.02 dBFS sample peak and −6.01 LUFS — but Homebrew runs it only on `brew test`, not on install. Since 0.19.4 releases are published from Jenkins, which smoke-builds the CLI tier from the release tarball and bumps the tap only after all 15 OBS repositories publish — but it has no macOS agent, so the formulae are never built on a Mac before users get them. The Windows build, since 0.20.0, runs the synthetic self-checks and a JSON round trip on its JSON build; the shipped `crete.exe` is checked only for its version and its DLL imports, and nothing runs `crete-gui.exe`. For 0.22.0 it did not run at all: the agent’s Application Control policy blocked MSYS2’s `bash.exe`, so that release has no Windows zip. The Linux packages are built with crête’s own flags — distribution `CFLAGS`, LTO and debug builds all off — so bit-identity with the gated builds holds by construction, not by comparison. No figure on this site was measured with a packaged binary: the census is weekly #106, on 0.22.0. The packaged `crete` is the FFmpeg tier without `-f json`.	Open
The >4 GB allocation is untested	RF64 / BW64 / Wave64 64-bit sizes are honoured — proven by a file carrying a chunk after the data chunk, which a naive reader swallows. crête loads a whole file before decoding, so a genuine >4 GB decode needs a larger host than any agent has.	Open, host-limited
The reference cannot score short tracks	Below about 30 seconds MAAT declines to compute DR at all, needing ten 3-second windows. One DSD256 album logs 14 of 17 tracks; the suite pins that exemption to exactly those counts, conditionally, so a complete local run still passes. 0.20.0 measured MAAT’s limit directly, on 28 synthetic PCM tracks of 3 to 45 s, at exactly 10.000 s — which does not match the figure above, so why this album’s three tracks go unscored is not established. The exemption stays pinned to its exact counts either way.	Exempted by design
No full-length lossless-carrier parity fixture	The clip-based pair is exact, but the ideal fixture has no clip-boundary tail — and a full 43-minute 5.1 96 kHz title decodes to roughly 12 GB of float64 per side. Needs a memory plan first.	Open
DFF coverage is one album	Deliberate: the suite proves a property of the code — that the DSF reader and the DFF deinterleaver emit the same samples — not a property of any album. A second multi-gigabyte capture would re-prove the same invariant.	Deliberate
Two reference rows nothing reproduces	On one DTS:X stem crête and MAAT agree to within 0.21 dB on ten of twelve tracks; two rows match nothing, including the peaks that gate them. Quarantined as named anomalies rather than absorbed into a tolerance.	Quarantined
The GUI takes defaults on two axes	It has no control for the DR averaging mode or the block set and always uses the shipping defaults; both are CLI-only A/B switches.	By design
The new GUI bands were never looked at	0.16.1’s SACD, cue and stream-picker bands are verified by construction — both front-ends call one collector — and headlessly, which is not the same as having been seen. 0.17.0 adds warnings to that surface without adding a test that anyone looked at them either. The SACD row’s trailing hint is long, so narrow window widths are the case to check. The results-table footer clipping in 0.12.0 is precisely the class of defect a suite cannot catch. 0.22.0’s BAND control, live at 176.4 and 352.8 kHz only, is compiled and option-checked, not exercised by hand.	Open
Release packaging is not built	Windows-on-ARM needs a second toolchain; Linux builds must be native rather than cross-compiled from macOS; the macOS leg has no agent and needs signing. Decided, scoped, not started.	Parked