Diff the corpus exports without prep-diff.php

Both sides share the corpus, the PHP binary, and the exporter, so the
export is already deterministic. prep-diff.php reconciles exports from
different environments: it zeroes line numbers, strips global-namespace
prefixes, and sorts collections. Measured against three open PRs, the
raw diff was equal or smaller in every case and kept each change at its
source location, while the collection sort relocated changed records
and the erasures hid classes of real change. Diff the exports as
emitted.

Author: sirreal

.github/workflows/corpus-diff.yml

@@ -1,15 +1,17 @@
 # Parses a pinned corpus of WordPress core source with the parser at the base
-# branch and with the PR merged into it, normalizes both JSON outputs with
-# prep-diff.php, and diffs them. Anything in the diff is a behavior change this
-# PR makes: every hunk must be either intended (and explained in the PR) or a
-# regression.
+# branch and with the PR merged into it, and diffs the two JSON exports.
+# Anything in the diff is a behavior change this PR makes: every hunk must be
+# either intended (and explained in the PR) or a regression.
 #
 # Policy decisions, deliberate:
 # - The corpus is pinned to one WordPress tag so diffs are reproducible.
-# - The head checkout's tools/export-corpus.php and prep-diff.php drive both
-#   sides, so tooling changes never masquerade as parser changes. When
-#   prep-diff.php itself changes, its effect on normalization shows up in the
-#   diff and is reviewed like any other change.
+# - The head checkout's tools/export-corpus.php drives both sides, so tooling
+#   changes never masquerade as parser changes.
+# - The exports are diffed as emitted, without prep-diff.php. Both sides share
+#   the corpus, the PHP binary, and the exporter, so the export is already
+#   deterministic; prep-diff.php exists to reconcile exports from different
+#   environments, and its erasures (line numbers, global-namespace prefixes)
+#   and collection sorting would hide or scatter real changes here.
 # - Non-blocking: the job succeeds even when the diff is non-empty. The diff
 #   is published as an artifact and summarized. Make it blocking only after
 #   the signal has proven trustworthy.
@@ -100,12 +102,10 @@ jobs:
           ls -l head.json
           [ "$(wc -c < head.json)" -ge 1000000 ]
 
-      - name: Normalize and diff
+      - name: Diff
         run: |
-          php -d memory_limit=4G prep-diff.php < base.json > base.norm.json
-          php -d memory_limit=4G prep-diff.php < head.json > head.norm.json
           # diff exits 1 on differences (expected) and 2 on trouble (fail).
-          diff -u --label base --label head base.norm.json head.norm.json > corpus.diff || [ $? -eq 1 ]
+          diff -u --label base --label head base.json head.json > corpus.diff || [ $? -eq 1 ]
           if [ -s corpus.diff ]; then
             hunks=$(grep -c '^@@' corpus.diff)
             lines=$(wc -l < corpus.diff)

README.md

@@ -53,9 +53,9 @@ wp parser create /path/to/source/code --user=<id|login>
 
 ## Corpus diff
 
-Unit tests do not cover every shape of real-world documentation, so changes to the parser are also checked against a corpus of WordPress core source. The same corpus is parsed with the parser at two refs — the base branch and the pull request merged into it — both JSON outputs are normalized with `prep-diff.php`, and the two are diffed. Everything in that diff is a behavior change the pull request makes: every hunk must be either intended and explained, or it is a regression.
+Unit tests do not cover every shape of real-world documentation, so changes to the parser are also checked against a corpus of WordPress core source. The same corpus is parsed with the parser at two refs, the base branch and the pull request merged into it, and the two JSON exports are diffed. Everything in that diff is a behavior change the pull request makes: every hunk must be either intended and explained, or it is a regression.
 
-`.github/workflows/corpus-diff.yml` runs this on every pull request. The corpus is `wp-includes` from a pinned WordPress tag (`WP_CORPUS_TAG` in the workflow), so diffs are reproducible. The job is non-blocking: it uploads the diff as a `corpus.diff` artifact and reports the hunk count in the job summary. The head checkout's `tools/export-corpus.php` and `prep-diff.php` drive both sides, so tooling changes never masquerade as parser changes; when `prep-diff.php` itself changes, its effect on normalization shows up in the diff and is reviewed like any other change.
+`.github/workflows/corpus-diff.yml` runs this on every pull request. The corpus is `wp-includes` from a pinned WordPress tag (`WP_CORPUS_TAG` in the workflow), so diffs are reproducible. The job is non-blocking: it uploads the diff as a `corpus.diff` artifact and reports the hunk count in the job summary. The head checkout's `tools/export-corpus.php` drives both sides, so tooling changes never masquerade as parser changes. The exports are diffed as emitted: both sides share the corpus, the PHP binary, and the exporter, so the output is already deterministic. `prep-diff.php` is for comparing exports from different environments; its line-number and namespace-prefix erasure and its collection sorting would hide or scatter real changes here.
 
 To run it locally, get the pinned corpus:
 
@@ -74,15 +74,13 @@ composer install
 
 On a branch this compares against the commit the branch left `master` at. CI runs on the pull request merged into `master`, so it compares against the `master` tip; merge `master` into the branch first to get the same comparison.
 
-Export both sides over the same corpus, normalize, and diff. `export-corpus.php` takes the parser root and the corpus directory, and writes JSON to stdout:
+Export both sides over the same corpus and diff. `export-corpus.php` takes the parser root and the corpus directory, and writes JSON to stdout:
 
 ```bash
 export LC_ALL=C
 php -d memory_limit=4G tools/export-corpus.php base WordPress-7.0.4/wp-includes > base.json
 php -d memory_limit=4G tools/export-corpus.php . WordPress-7.0.4/wp-includes > head.json
-php -d memory_limit=4G prep-diff.php < base.json > base.norm.json
-php -d memory_limit=4G prep-diff.php < head.json > head.norm.json
-diff -u base.norm.json head.norm.json > corpus.diff
+diff -u base.json head.json > corpus.diff
 ```
 
 An empty `corpus.diff` means the change has no effect on parser output.