test: prove subprocess and production coverage (Fixes #534)

Verify exact idle/info child profiles and real handler/transport/writer counter increases. Add conservative production/changed-line diagnostics without altering existing raw coverage gates, and measure native macOS against the exact base on one runner.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: karthiknadig

.github/workflows/coverage-baseline.yml

@@ -27,6 +27,9 @@ jobs:
     steps:
       - name: Checkout
         uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          fetch-depth: 0
+          persist-credentials: false
 
       - name: Set Python to PATH
         uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
@@ -181,6 +184,21 @@ jobs:
         env:
           RUST_BACKTRACE: 1
           RUST_LOG: trace
+          PET_SUBPROCESS_COVERAGE_PROOF: ${{ github.workspace }}/subprocess-coverage.json
+        shell: bash
+
+      - name: Verify Isolated Server Coverage
+        if: always()
+        run: python scripts/coverage_detail.py proof --manifest subprocess-coverage.json --output subprocess-coverage
+        shell: bash
+
+      - name: Report Production and Changed Coverage
+        if: always()
+        run: >-
+          python scripts/coverage_detail.py report
+          --lcov lcov.info
+          --base "HEAD"
+          --output production-coverage
         shell: bash
 
       - name: Validate Coverage Baseline
@@ -194,8 +212,14 @@ jobs:
         shell: bash
 
       - name: Upload Coverage Artifact
+        if: always()
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
         with:
           name: coverage-baseline-${{ matrix.os }}
-          path: lcov.info
+          path: |
+            lcov.info
+            coverage-baseline-report.md
+            production-coverage/
+            subprocess-coverage/
+            subprocess-coverage.json
           retention-days: 90

.github/workflows/coverage-macos.yml

@@ -0,0 +1,92 @@
+name: Native macOS Coverage
+
+on:
+  pull_request:
+    branches: [main, 'release*', 'release/*', 'release-*']
+  push:
+    branches: [main, 'release*', 'release/*', 'release-*']
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  coverage:
+    runs-on: macos-14
+    timeout-minutes: 35
+    steps:
+      - name: Checkout
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          fetch-depth: 0
+          persist-credentials: false
+
+      - name: Set Python to PATH
+        uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+        with:
+          python-version: '3.12'
+
+      - name: Rust Tool Chain setup
+        uses: dtolnay/rust-toolchain@stable
+        with:
+          toolchain: stable
+          components: llvm-tools-preview
+
+      - name: Install cargo-llvm-cov
+        uses: taiki-e/install-action@cargo-llvm-cov
+
+      - name: Validate Coverage Reporting
+        run: python -m unittest discover -s scripts/tests -p 'test_*.py' -v
+
+      - name: Collect Native macOS Coverage
+        run: cargo llvm-cov --workspace --lcov --output-path lcov.info -- --test-threads=1
+        env:
+          PET_SUBPROCESS_COVERAGE_PROOF: ${{ github.workspace }}/subprocess-coverage.json
+
+      - name: Verify Isolated Server Coverage
+        if: always()
+        run: python scripts/coverage_detail.py proof --manifest subprocess-coverage.json --output subprocess-coverage
+
+      - name: Report Production and Changed Coverage
+        if: always()
+        env:
+          BASE: ${{ github.event.pull_request.base.sha || github.sha }}
+        run: python scripts/coverage_detail.py report --lcov lcov.info --base "$BASE" --output production-coverage
+
+      - name: Measure Exact PR Base on the Same Runner
+        if: github.event_name == 'pull_request'
+        env:
+          BASE: ${{ github.event.pull_request.base.sha }}
+          CARGO_TARGET_DIR: ${{ runner.temp }}/coverage-base-target
+        run: |
+          git worktree add --detach "$RUNNER_TEMP/coverage-base" "$BASE"
+          cd "$RUNNER_TEMP/coverage-base"
+          cargo llvm-cov --workspace --lcov --output-path "$GITHUB_WORKSPACE/baseline-lcov.info" -- --test-threads=1
+
+      - name: Compare Exact Base Coverage
+        if: always() && github.event_name == 'pull_request'
+        run: >-
+          python scripts/quality_snapshot.py coverage
+          --current lcov.info --baseline baseline-lcov.info --platform macOS
+          --report coverage-report.md --summary "$GITHUB_STEP_SUMMARY"
+
+      - name: Validate Main Coverage
+        if: github.event_name != 'pull_request'
+        run: >-
+          python scripts/quality_snapshot.py coverage
+          --current lcov.info --baseline lcov.info --platform macOS
+          --report coverage-report.md --summary "$GITHUB_STEP_SUMMARY"
+
+      - name: Upload Native macOS Coverage
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: coverage-macos-native
+          path: |
+            lcov.info
+            baseline-lcov.info
+            coverage-report.md
+            production-coverage/
+            subprocess-coverage/
+            subprocess-coverage.json
+          retention-days: 30

.github/workflows/coverage.yml

@@ -32,6 +32,9 @@ jobs:
     steps:
       - name: Checkout
         uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          fetch-depth: 0
+          persist-credentials: false
 
       - name: Post Coverage Started Comment
         if: github.event.pull_request.head.repo.full_name == github.repository
@@ -200,6 +203,21 @@ jobs:
         env:
           RUST_BACKTRACE: 1
           RUST_LOG: trace
+          PET_SUBPROCESS_COVERAGE_PROOF: ${{ github.workspace }}/subprocess-coverage.json
+        shell: bash
+
+      - name: Verify Isolated Server Coverage
+        if: always()
+        run: python scripts/coverage_detail.py proof --manifest subprocess-coverage.json --output subprocess-coverage
+        shell: bash
+
+      - name: Report Production and Changed Coverage
+        if: always()
+        run: >-
+          python scripts/coverage_detail.py report
+          --lcov lcov.info
+          --base "${{ github.event.pull_request.base.sha }}"
+          --output production-coverage
         shell: bash
 
       - name: Wait for Exact PR Base Coverage
@@ -248,6 +266,9 @@ jobs:
           path: |
             lcov.info
             coverage-report.md
+            production-coverage/
+            subprocess-coverage/
+            subprocess-coverage.json
           if-no-files-found: ignore
 
       - name: Post Coverage Comment

crates/pet/tests/jsonrpc_server_test.rs

@@ -38,13 +38,20 @@ struct RawRpcClient {
 
 impl RawRpcClient {
     fn spawn() -> Self {
+        Self::spawn_with_profile(None)
+    }
+
+    fn spawn_with_profile(profile: Option<&Path>) -> Self {
         let mut command = Command::new(env!("CARGO_BIN_EXE_pet"));
         command
             .arg("server")
             .stdin(Stdio::piped())
             .stdout(Stdio::piped())
             .stderr(Stdio::inherit());
         jsonrpc_client::configure_isolated_pet_environment(&mut command);
+        if let Some(profile) = profile {
+            command.env("LLVM_PROFILE_FILE", profile);
+        }
         let mut child = command.spawn().expect("raw fixture must spawn PET");
         let stdout = child.stdout.take().expect("PET stdout must be piped");
         let (sender, responses) = mpsc::channel();
@@ -819,6 +826,89 @@ fn invalid_and_oversize_framing_terminates_with_bounded_diagnostics() {
     }
 }
 
+#[test]
+fn normal_shutdown_records_pid_unique_server_profiles() {
+    let Some(proof_path) = std::env::var_os("PET_SUBPROCESS_COVERAGE_PROOF") else {
+        return;
+    };
+    let inherited = PathBuf::from(
+        std::env::var_os("LLVM_PROFILE_FILE")
+            .expect("coverage proof requires cargo-llvm-cov instrumentation"),
+    );
+    let directory = inherited.parent().expect("profile must have a directory");
+    assert!(
+        directory.is_absolute(),
+        "profile directory must be absolute"
+    );
+    assert!(
+        inherited
+            .file_name()
+            .unwrap()
+            .to_string_lossy()
+            .contains("%p"),
+        "cargo-llvm-cov must use PID-unique profiles"
+    );
+    let mut profiles = serde_json::Map::new();
+    for (name, request) in [("idle", false), ("info", true)] {
+        let prefix = format!("pet-proof-{}-{name}-", std::process::id());
+        let pattern = directory.join(format!("{prefix}%p-%m.profraw"));
+        let mut client = RawRpcClient::spawn_with_profile(Some(&pattern));
+        let prefix = format!("{prefix}{}-", client.child.id());
+        assert!(
+            fs::read_dir(directory).unwrap().all(|entry| !entry
+                .unwrap()
+                .file_name()
+                .to_string_lossy()
+                .starts_with(&prefix)),
+            "child profile must not predate this process exit"
+        );
+        if request {
+            client.send(json!({"jsonrpc": "2.0", "id": "profile-proof", "method": "info"}));
+            let reply = client.receive();
+            assert_eq!(reply["id"], "profile-proof");
+            assert_eq!(reply["result"]["petVersion"], env!("CARGO_PKG_VERSION"));
+        }
+        client.child.stdin.take();
+        let status = jsonrpc_client::wait_for_exit(&mut client.child, Duration::from_secs(4))
+            .expect("profile proof requires graceful exit, not kill fallback");
+        assert!(
+            status.success(),
+            "profile probe exited unsuccessfully: {status}"
+        );
+        jsonrpc_client::join_reader(client.reader.take().unwrap(), Duration::from_secs(4)).unwrap();
+        let raw: Vec<PathBuf> = fs::read_dir(directory)
+            .unwrap()
+            .map(|entry| entry.unwrap().path())
+            .filter(|path| {
+                path.file_name()
+                    .unwrap()
+                    .to_string_lossy()
+                    .starts_with(&prefix)
+            })
+            .collect();
+        assert!(
+            !raw.is_empty(),
+            "normal server exit did not flush its own profile"
+        );
+        for path in &raw {
+            assert!(
+                fs::metadata(path).unwrap().len() > 0,
+                "empty server profile"
+            );
+        }
+        profiles.insert(name.into(), json!(raw));
+    }
+    fs::write(
+        proof_path,
+        serde_json::to_vec_pretty(&json!({
+            "binary": env!("CARGO_BIN_EXE_pet"),
+            "profiles": profiles,
+        }))
+        .unwrap(),
+    )
+    .expect("write subprocess coverage evidence");
+}
+
 #[test]
 fn stdin_eof_after_exchange_exits_cleanly_within_one_second() {
     let client = PetJsonRpcClient::spawn().unwrap();

docs/QUALITY_SNAPSHOTS.md

@@ -113,6 +113,53 @@ for the new client clocks.
 
 Linux and Windows line and function coverage are compared with the exact base commit. A decrease greater than 0.01 percentage points blocks the pull request. Coverage artifacts and comments remain available for inspection even when the comparison fails.
 
+### Production-focused and subprocess evidence
+
+The raw workspace percentages still include inline tests and retain the same exact-base
+0.01 percentage-point line/function gate. Supplemental `production-coverage/report.md` and
+schema-1 `details.json` separate executable production/test lines, list uncovered production
+lines, and intersect added/modified Rust lines with executable production lines. Changed files
+without instrumentation are listed explicitly, never assumed covered. These diagnostics do not
+introduce a fabricated baseline, change the raw denominator, or replace regression protection.
+
+Classification excludes integration-test/benchmark directories and Rust items explicitly marked
+`#[cfg(test)]` or `#[test]`, including inline modules and test-only helper functions. It masks
+strings, raw/byte strings, characters, and nested comments before matching item boundaries.
+Helpers outside those boundaries and complex conditional attributes remain conservatively in the
+production category; this is a source-focused diagnostic, not full Rust conditional-compilation
+analysis. Invalid/missing LCOV, missing source, inconsistent hit summaries, and source-line
+mismatches fail the reporting step. LLVM summaries can include more entries in `LF`/`LH`
+than the unique `DA` source lines (observed in real Windows exports). That deficit is reported
+per file (including unmatched summary hits) and conservatively retained as uncovered production,
+never dropped from the denominator or silently assigned coverage.
+Changed lines without `DA` records are listed separately in JSON, including non-executable syntax;
+they are not silently considered covered.
+
+Every coverage job opts into `normal_shutdown_records_pid_unique_server_profiles` through
+`PET_SUBPROCESS_COVERAGE_PROOF`. The test requires cargo-llvm-cov's absolute, PID-unique output
+pattern, launches idle and known-`info` PET subprocesses, closes stdin, and requires successful
+bounded exit and nonempty profiles for those exact child PIDs. The raw profiles stay in the normal
+cargo-llvm-cov collection directory and are included in the workspace report. The verifier also
+merges each child's profiles separately with the matching Rust LLVM tools and proves zero idle
+versus positive `info` execution at the real handler, transport dispatch, and response writer.
+The uploaded `subprocess-coverage/proof.json` and isolated LCOV exports retain that evidence;
+a killed child, missing profile, or absent execution witness fails rather than appearing covered.
+
+Native ARM64 macOS coverage runs workspace default-feature and native process/transport tests,
+including Darwin-specific process ownership paths. It intentionally does not compare this workload
+with Linux/Windows's installed-manager `ci` workload. For each macOS PR, the exact base revision is
+built and measured separately on the same runner with the same compiler and feature selection;
+the unchanged line/function comparator gates those comparable artifacts. This works on the first
+PR without silently accepting an absent macOS baseline. Main/manual runs publish the native
+measurement and proof for inspection. Existing functional macOS installed-manager jobs remain.
+
+Stable Rust line instrumentation does not provide condition/branch outcomes. Reports show LCOV
+`BRDA` totals when supplied, otherwise explicitly report branch data as unavailable (not 100%).
+Native malformed-frame/envelope, EOF, broken-output, saturation, and descendant tests provide
+behavioral failure-path evidence, but line coverage cannot prove both sides of every condition.
+Nightly `cargo llvm-cov --branch` can be used as a separate experiment; its unstable toolchain
+and differing denominator are not substituted into the stable cross-platform gate.
+
 ## Running locally
 
 The comparator requires Python 3.10 or newer.