Problem
src/conformance/runner.py:544-559 allows a PCM scenario to pass if the client's hash merely matches a prefix of the source PCM:
prefix_hasher = FloatPcmHasher()
prefix_hasher.update_from_pcm_bytes(prefix_pcm, bit_depth=audio["bit_depth"])
prefix_hash = prefix_hasher.hexdigest()
if prefix_hash == received_hash:
missing_samples = fixture.frame_count * audio["channels"] - received_sample_count
return True, f"PCM prefix matches; trailing samples omitted={missing_samples}"
This is deliberate tolerance for adapters that disconnect slightly early. But:
- The passed reason "trailing samples omitted" ships in the report with status "passed", not a distinct yellow state.
- A client that quietly truncates audio under normal conditions will appear fully green.
- No threshold exists — a client that receives 1 sample and correctly hashes the first sample would pass.
Proposed fix
- Introduce a bounded tolerance (e.g. "fail if truncation exceeds 5% of source frames" or "only allowed if final chunk was cut off mid-transmission").
- Surface a distinct result state in the report (e.g.
passed_with_truncation) so truncation shows differently than a full match.
- Consider removing the prefix fallback entirely if the intent is to prove full-stream reception.
Context
Prefix tolerance was a pragmatic escape hatch when adapters had racy shutdowns. Now that the harness has deterministic stop sequences, the cost of silent truncation > the convenience of tolerating it.
Problem
src/conformance/runner.py:544-559allows a PCM scenario to pass if the client's hash merely matches a prefix of the source PCM:This is deliberate tolerance for adapters that disconnect slightly early. But:
Proposed fix
passed_with_truncation) so truncation shows differently than a full match.Context
Prefix tolerance was a pragmatic escape hatch when adapters had racy shutdowns. Now that the harness has deterministic stop sequences, the cost of silent truncation > the convenience of tolerating it.