Code snippets: Support "Outputs:" comments in runnable snippets #50

Workflow file for this run

.github/workflows/corpus-diff.yml at 40d6833

	# Parses a pinned corpus of WordPress core source with the parser at the base
	# branch and with the PR merged into it, and diffs the two JSON exports.
	# Anything in the diff is a behavior change this PR makes: every hunk must be
	# either intended (and explained in the PR) or a regression.
	#
	# Policy decisions, deliberate:
	# - The corpus is pinned in `.github/corpus-version` to one WordPress tag so
	# diffs are reproducible.
	# - The head checkout's tools/export-corpus.php drives both sides, so tooling
	# changes never masquerade as parser changes.
	# - The exports are diffed as emitted, without prep-diff.php. Both sides share
	# the corpus, the PHP binary, and the exporter, so the export is already
	# deterministic; prep-diff.php exists to reconcile exports from different
	# environments, and its erasures (line numbers, global-namespace prefixes)
	# and collection sorting would hide or scatter real changes here.
	# - Non-blocking: the job succeeds even when the diff is non-empty. The diff
	# is published as an artifact, summarized, and posted as a PR comment that
	# is updated in place on every run. Make it blocking only after the signal
	# has proven trustworthy.
	# - The comment needs a token that can write to the PR. pull_request runs for
	# forks and for Dependabot get a read-only token, so those PRs get the
	# summary and the artifact only.
	# - PHP 8.4, the newest runtime in the unit-test matrix. Both sides run under
	# the same binary, so the PHP version never shows up as a parser change.

	name: Corpus Diff

	on:
	pull_request:

	# A new push to the PR supersedes any run still in flight for it.
	concurrency:
	group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
	cancel-in-progress: true

	permissions: {}

	jobs:
	corpus-diff:
	name: WordPress corpus regeneration diff
	runs-on: ubuntu-latest
	permissions:
	contents: read

	env:
	CORPUS_PIN_FILE: .github/corpus-version
	LC_ALL: C

	steps:
	- name: Checkout
	uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
	with:
	fetch-depth: 0
	persist-credentials: false

	- name: Read corpus pin
	run: \|
	set -euo pipefail
	WP_CORPUS_TAG="$(cat "${CORPUS_PIN_FILE}")"
	if [[ ! "${WP_CORPUS_TAG}" =~ ^[0-9]+\.[0-9]+(\.[0-9]+)?$ ]]; then
	echo "::error::${CORPUS_PIN_FILE} does not contain a usable WordPress version (got: '${WP_CORPUS_TAG}')."
	exit 1
	fi
	echo "Pinned corpus tag: ${WP_CORPUS_TAG}"
	echo "WP_CORPUS_TAG=${WP_CORPUS_TAG}" >> "${GITHUB_ENV}"

	- name: Set up PHP
	uses: shivammathur/setup-php@f3e473d116dcccaddc5834248c87452386958240 # 2.37.2
	with:
	php-version: "8.4"
	coverage: none

	# The key carries the corpus layout: bump the suffix when the download
	# step changes what it keeps.
	- name: Cache corpus
	id: cache-corpus
	uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
	with:
	path: corpus
	key: wp-corpus-${{ env.WP_CORPUS_TAG }}-php-all

	# The whole release: wp-admin, wp-includes, the root files, and
	# wp-content with the bundled themes and plugins. Only PHP files are
	# kept; the parser reads nothing else.
	- name: Download corpus
	if: steps.cache-corpus.outputs.cache-hit != 'true'
	run: \|
	curl -sSfL -o wordpress.zip "https://github.com/WordPress/WordPress/archive/refs/tags/${WP_CORPUS_TAG}.zip"
	unzip -q wordpress.zip
	mv "WordPress-${WP_CORPUS_TAG}" corpus
	find corpus -type f ! -name '*.php' -delete
	rm wordpress.zip

	# An empty or miscached corpus would make both exports emit `[]` and the
	# diff trivially empty, a green job that checked nothing. A release has
	# well over 1500 PHP files on any supported tag.
	- name: Verify corpus
	run: \|
	count=$(find corpus -name '*.php' \| wc -l)
	echo "Corpus PHP files: ${count}"
	[ "$count" -ge 1500 ]

	# On pull_request, HEAD is the PR merged into the base branch, so its
	# merge base with the base branch is the base branch tip, not the commit
	# the PR branched from. That is the intended comparison: base as it is
	# now against base plus exactly this PR, with no hunks from other work
	# that landed since the branch point.
	- name: Check out base
	env:
	BASE_REF: ${{ github.base_ref }}
	run: git worktree add base "$(git merge-base -- "origin/${BASE_REF}" HEAD)"

	- name: Install Composer dependencies (head)
	run: composer install --no-interaction --no-security-blocking

	- name: Install Composer dependencies (base)
	run: composer --working-dir=base install --no-interaction --no-security-blocking

	# The size floor guards the same failure mode as Verify corpus: a real
	# export is tens of megabytes of JSON.
	- name: Export corpus (base)
	run: \|
	php -d memory_limit=4G tools/export-corpus.php base corpus > base.json
	ls -l base.json
	[ "$(wc -c < base.json)" -ge 1000000 ]

	- name: Export corpus (head)
	run: \|
	php -d memory_limit=4G tools/export-corpus.php . corpus > head.json
	ls -l head.json
	[ "$(wc -c < head.json)" -ge 1000000 ]

	- name: Diff
	run: \|
	# diff exits 1 on differences (expected) and 2 on trouble (fail).
	diff -u --label base --label head base.json head.json > corpus.diff \|\| [ $? -eq 1 ]
	echo "Corpus diff: $(grep -c '^@@' corpus.diff \|\| true) hunks, $(wc -l < corpus.diff) lines"

	- name: Upload diff
	id: upload
	if: always()
	uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
	with:
	path: corpus.diff
	archive: false
	if-no-files-found: ignore

	# Render on failure too: the comment is updated in place, so skipping
	# this step would leave the previous push's report standing as current.
	- name: Render report
	if: ${{ !cancelled() }}
	env:
	JOB_STATUS: ${{ job.status }}
	HEAD_SHA: ${{ github.event.pull_request.head.sha }}
	ARTIFACT_URL: ${{ steps.upload.outputs.artifact-url }}
	RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
	run: \|
	if [ "${JOB_STATUS}" = "success" ]; then
	BASE_SHA="$(git -C base rev-parse HEAD)" tools/corpus-diff-comment.sh corpus.diff > comment.md
	else
	printf '<!-- corpus-diff -->\n### Corpus diff\n\nRun failed at `%s`; no diff was produced for this push. See the [run log](%s).\n' \
	"${HEAD_SHA:0:7}" "${RUN_URL}" > comment.md
	fi
	tail -n +2 comment.md >> "$GITHUB_STEP_SUMMARY"

	- name: Upload rendered report
	if: always()
	uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
	with:
	path: comment.md
	archive: false
	if-no-files-found: ignore

	comment:
	name: Publish corpus diff report
	needs: corpus-diff
	if: ${{ !cancelled() && github.event.pull_request.head.repo.full_name == github.repository && github.actor != 'dependabot[bot]' }}
	runs-on: ubuntu-latest
	permissions:
	pull-requests: write # Create or update the corpus diff comment.

	steps:
	- name: Download rendered report
	uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
	with:
	name: comment.md

	# Create the comment on the first run and update it on every later one,
	# so the PR carries one report that reflects the latest push.
	- name: Comment on the pull request
	env:
	GH_TOKEN: ${{ github.token }}
	REPO: ${{ github.repository }}
	PR: ${{ github.event.pull_request.number }}
	run: \|
	ids=$(gh api "repos/${REPO}/issues/${PR}/comments" --paginate \
	--jq '.[] \| select(.user.login == "github-actions[bot]") \| select(.body \| startswith("<!-- corpus-diff -->")) \| .id')
	id=${ids%%$'\n'*}
	if [ -n "$id" ]; then
	gh api --silent -X PATCH "repos/${REPO}/issues/comments/${id}" -F body=@comment.md
	else
	gh api --silent -X POST "repos/${REPO}/issues/${PR}/comments" -F body=@comment.md
	fi

Navigation Menu

Code snippets: Support "Outputs:" comments in runnable snippets #50

Workflow file

Code snippets: Support "Outputs:" comments in runnable snippets #50

Uh oh!

Workflow file for this run