Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions docs/AGENT_TASK_BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,75 @@ Gate:
ctest --test-dir build --output-on-failure -j4
```

### D3. ALC tolerance — model + decoder sweep

Goal: validate (and measure) whether the modem decodes through realistic
HF transmitter ALC compression. Today there is **no ALC model** in OTASim,
**no ALC tolerance test**, and **no documented evidence** the modem
works through real radio ALC. Operator feedback (KC3VPB 2026-05-08,
`feedback_kc3vpb_alc_handling`) was explicit: design for ALC, don't
dodge it. The current 0.7×/0.8× WAV pre-attenuation is the wrong fix.

Full spec: `docs/ALC_TOLERANCE_WORKSTREAM.md`.

Scope:

- `src/ota_channel_core/models.{cpp,hpp}` — new `AlcCompressor`
class (peak detect, exponential attack/release in dB domain,
configurable threshold / ratio / attack / release / overshoot).
- `src/ota_channel_core/session_context.cpp::advanceSessionClock`
— wire ALC into the mixer chain BEFORE channel noise (ALC is in
the transmitter; noise is in the channel; order matters).
- `proto/ota_simulator.proto` + `tools/cli_simulator.cpp::setOtaChannel`
+ `tools/otasim_ctl.cpp` — extend channel model enum and
`SetChannel` plumbing for `alc` model.
- `tests/test_alc_compressor.cpp` — unit test for the compressor
itself (steady-state gain, attack/release time, multi-tone IMD).
- `tests/test_decoder_alc_tolerance.cpp` — sweep (modulation, code
rate, compression ratio, drive above threshold) and produce a
decode-success-rate table per cell.

Acceptance:

- Compressor unit test passes with all measured parameters within
±20% of configured values.
- Decoder sweep produces a numerical baseline table for the
production mode ladder (DQPSK R1/4 / R1/2 / R2/3 / R3/4, QPSK
R1/2 / R2/3, D8PSK R2/3) at compression ratios {1:1, 2:1, 4:1,
8:1, ∞:1} and drives {threshold, +6 dB, +12 dB}.
- DQPSK R1/2 (the production baseline for most QSOs) **must**
decode at 4:1 / +6 dB with ≥95% success. If it doesn't, STOP
and flag as a PHY issue — that's the real-radio deployment
blocker.
- Baseline table committed as a regression gate for future ALC
model or modulator changes.
- `docs/INVARIANTS.md` updated with "decoder must tolerate ALC at
X ratio Y compression" once baseline is established.

Out of scope:

- PAPR reduction in the modulator (clip-and-filter, ACE) — separate
workstream, becomes interesting *after* baseline numbers exist.
- Changing the default `tx_drive` value — per-radio operator choice.
- Adding ALC to the Mac↔Pi5 cable rig — would need a Pi-side
software ALC injector; outside the OTASim contract.

Gate:

```bash
ctest --test-dir build -R "AlcCompressor|DecoderAlcTolerance" --output-on-failure -j1
```

Risks:

- Decoder may fail at modest ALC (≤4:1) on higher-order modulations
(QPSK, D8PSK). If so, the conclusion is "production-safe modes
are DQPSK only" — a real finding worth surfacing, not a test
failure to suppress.
- Compressor parameters must match real-radio behavior; if model
is too gentle, we miss real failures. Pin parameters to documented
HF transceiver specs (FT-891, IC-7300, K3 — common rig classes).

## Lane E: Security And Agent Governance

### E1. Secret/Artifact Leak Gate
Expand Down
241 changes: 241 additions & 0 deletions docs/ALC_TOLERANCE_WORKSTREAM.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,241 @@
# ALC tolerance — workstream spec

**Status:** specified, not started
**Owner:** ProjectUltra core
**Origin:** KC3VPB OTA feedback 2026-05-08 + audit conversation 2026-05-18
**Memory link:** `feedback_kc3vpb_alc_handling.md`

## Why this exists

ProjectUltra targets real HF radios. Every SSB transmitter in the
deployment chain has an Automatic Level Control (ALC) loop that
compresses audio peaks to protect the final amp. The modem's TX
audio passes through ALC **on the way out of the operator's radio**
before it ever reaches the antenna, the propagation path, or a
remote receiver.

Today the modem has **zero validation** that it tolerates ALC
compression:

- OTASim channel models are AWGN + Watterson only. Perfectly linear.
- Mac↔Pi5 cable rig has no radio in the loop.
- No CTest gate, no invariant, no fixture exercises compression.
- The only ALC exposure was the KC3VPB 2026-05-07 OTA session, where
decode was poor and the response was to **pre-attenuate the WAV**
(the wrong fix, per KC3VPB's pushback).

Until this workstream lands, "does the modem work on a real radio"
is a guess.

## Scope

This workstream is **two pieces**:

1. **An ALC compression channel model** in OTASim, configurable
per session, exposed via `SetChannel` / `InjectEffect`.
2. **A decoder tolerance test sweep** covering the production
mode ladder (DQPSK R1/4 → DQPSK R1/2 → DQPSK R2/3 → DQPSK R3/4
→ QPSK R1/2 → QPSK R2/3 → D8PSK R2/3) at realistic ALC
parameters.

**Out of scope** (do not pull these in):

- Adding PAPR reduction (clip-and-filter, ACE) in the modulator.
That's a separate workstream that becomes interesting *after*
we measure ALC tolerance.
- Changing the default `tx_drive` value. Operator-tunable per radio.
- Adding ALC to the cable rig (Mac↔Pi5). Outside the OTASim
contract; would need a Pi-side software ALC injector.

## Background: how SSB ALC actually behaves

**Detector**: peak detector with fast attack. Some radios mix peak
+ short-window RMS to be less twitchy on syllabic speech, but for
continuous-power data modes the peak detector dominates.

**Threshold**: set by the manufacturer to clamp at roughly the
"safe drive" point (the radio's nominal full output). User's "mic
gain" knob moves the input level, not the threshold.

**Ratio**: 4:1 to 10:1 for normal compression range; effectively
∞:1 (limiter) above ALC range — radio refuses to output more
power regardless of input level.

**Attack**: 1-10 ms typical. Has to catch a syllabic peak that
lasts ~50-200 ms. Fast attack prevents over-power on the first
peak of a new syllable.

**Release**: 50-500 ms typical. Slow release keeps gain reduced
through the rest of a syllable.

**Behavior on data modes** (continuous-power, no syllabic structure):

- ALC engages on first sample above threshold.
- Stays engaged because release is much slower than the modem's
symbol period.
- After ~1-2 ms attack settles, the output is **steady-state
compressed** — a near-constant gain reduction across the entire
TX burst.
- The transient at TX onset (first ~1-2 ms) is uncompressed,
producing a brief "punch" — sometimes called the "ALC overshoot"
— which can saturate the final stage on hot drive.
- Multi-tone signals (OFDM) suffer **intermodulation distortion**:
ALC's nonlinear gain creates IM products at carrier sum/difference
frequencies. Splatter.

## Implementation: ALC channel-model node

**Location**: `src/ota_channel_core/models.cpp` + `models.hpp`.

**Class shape**:

```cpp
struct AlcConfig {
float threshold_dbfs = -1.0f; // hard threshold in dBFS (default -1 dBFS)
float ratio = 4.0f; // compression ratio above threshold
float attack_ms = 5.0f; // ms to reach 90% gain reduction
float release_ms = 150.0f; // ms to release after below threshold
float makeup_gain_db = 0.0f; // post-compressor gain (default 0)
bool include_overshoot = true; // model the ~1-2 ms transient
};

class AlcCompressor {
public:
explicit AlcCompressor(AlcConfig config, float sample_rate);
void process(std::span<float> samples); // in-place, mono
void reset(); // for new TX burst
};
```

Implementation notes:

- Peak detector with exponential attack/release in dB domain.
- Gain reduction = `(input_db - threshold_db) * (1 - 1/ratio)`,
clamped to non-negative.
- Overshoot model: don't apply gain reduction for the first
`attack_ms` of a new TX burst (detected by `reset()` call on
TX start).
- Run before AWGN noise in the OTASim mixer chain — ALC is in
the *transmitter*, noise is in the *channel*. Order matters.

**Wire format**: add an `alc` model to `ChannelType` / `parseChannelType`.
Configurable via `InjectEffect` for mid-session changes, and via
`SetChannel` with a JSON params field for session-wide setting.

Estimated: ~150-200 LOC implementation + 50 LOC test for the
compressor itself.

## Tests

### Unit test (mechanical correctness)

`tests/test_alc_compressor.cpp` — drive a known sine + noise into
the compressor, verify:

- Steady-state output peak at threshold (±0.5 dB).
- Steady-state gain reduction matches `(input_db - threshold) * (1 - 1/ratio)`.
- Attack time within ±20% of configured value.
- Release time within ±20% of configured value.
- Overshoot present when configured, absent when disabled.
- Multi-tone IMD products appear at expected frequencies.

### Integration test (decoder tolerance)

`tests/test_decoder_alc_tolerance.cpp` (or extend
`test_otasim_serve_smoke` with an ALC scenario).

For each (modulation, code rate) pair in the production ladder,
run a CONNECT + 1 KB DATA + DISCONNECT scenario through OTASim
with ALC enabled at:

| Compression ratio | Drive above threshold | Expected outcome (initial) |
|------------------:|:---------------------:|:-----------------------------|
| 1:1 (no ALC) | n/a | baseline (must match) |
| 2:1 | +6 dB | decode (~no degradation) |
| 4:1 | +6 dB | decode (mild degradation) |
| 4:1 | +12 dB | likely decode for DQPSK; QPSK marginal |
| 8:1 | +12 dB | DQPSK ok, higher mods fail |
| ∞:1 (limiter) | +12 dB | DQPSK probably ok |
| ∞:1 (limiter) | +20 dB | broad failure expected |

Pass criterion: produce a table per (mode × rate × compression)
of decode success rate. This is a **measurement**, not a binary.
First run sets the baseline; subsequent runs gate against regression.

### Hardware OTA (out of CTest, in `agents/run_hardware_smoke.sh`)

Re-run KC3VPB-style real-radio test with the default `tx_drive`
deliberately set hot enough to engage ALC, decode rate measured.
Compare to OTASim ALC-on prediction.

## Milestones

1. **M1 — Compressor core** (~1 day): `AlcCompressor` class + unit
test. Builds, tests pass.
2. **M2 — Wire into OTASim** (~half day): channel-model node,
`SetChannel` plumbing, integration with mixer order
(ALC → channel noise).
3. **M3 — Decoder sweep** (~1 day): integration test, run the
table above, commit the baseline numbers as a regression gate.
4. **M4 — Document** (~half day): CHANGELOG entry, update
`docs/INVARIANTS.md` with "decoder must tolerate ALC at X
ratio Y compression" once baseline is established.
5. **M5 — Hardware confirm** (deferred, requires Mac↔real-radio):
re-run KC3VPB scenario, document.

Total: ~3 dev-days to M4. M5 needs operator availability.

## Anti-stall rules

- **No PAPR-reduction work in this round.** That's tempting once
we see decoder fails — but the right sequence is *measure first,
fix second*. PAPR reduction in the modulator becomes its own
workstream after we have baseline numbers.
- **No default-value tweaks** (`tx_drive`, `output_gain`, etc.) in
this round. The fix is decoder tolerance, not avoidance.
- **If decoder fails at 4:1 / +6 dB on DQPSK R1/2** (the production
baseline for most QSOs): STOP and flag as a real PHY issue, not
a test-tuning issue.

## Three-perspective check

**PHY theorist**: ALC introduces *amplitude nonlinearity* into a
DPSK system. DPSK is amplitude-insensitive at the symbol-decision
level (only phase matters), but pilot-based channel estimation
in OFDM assumes roughly stable amplitudes. Mild compression
(2:1, ≤3 dB) should be invisible to the demodulator. Heavy
compression (∞:1, >6 dB) creates IMD products at carrier
sum/difference frequencies — for OFDM with 50+ carriers, this
spreads across the band. Real impact depends on whether the
demod's per-carrier SNR estimate sees the IMD as noise (probably
yes) or interprets it as signal (probably no, since pilots
calibrate amplitude per carrier).

**DSP systems engineer**: a sample-rate-accurate peak-detect
+ exponential gain follower is a textbook ~50-line implementation.
Numerical care: dB conversion (avoid `log(0)`), attack/release
filter coefficients computed from time constants, proper handling
of TX-burst boundaries (call `reset()` on new burst to model the
overshoot transient).

**Veteran HF operator**: this is the missing piece that separates
"works on the cable" from "works on the radio". KC3VPB called it
out 10 days ago. Until this lands, every test result is suspect
for real-radio deployment claims. The right operator answer to
"how do I drive?" is "set ALC to barely flicker" — but they
should EXPECT it to flicker, and the modem should EXPECT to see
some compression. This workstream validates that expectation.

## References

- `feedback_kc3vpb_alc_handling.md` (memory) — 2026-05-08 operator
pushback.
- `src/ota_channel_core/models.cpp` — current channel models, the
add-on point.
- `src/ota_channel_core/session_context.cpp::advanceSessionClock` —
the mixer call order (ALC must run BEFORE noise).
- `tools/cli_simulator.cpp::setOtaChannel` — pattern for the
`SetChannel` plumbing.
- `proto/ota_simulator.proto` — `SetChannelRequest`, `InjectEffectRequest`.
- `docs/CHANGELOG.md` 2026-05-14 "SimulatedChannel AWGN is
continuous RX noise" — precedent for non-trivial channel work.
Loading