Skip to content

fix(#2093): retry empty PR file list before refusing to approve - #1196

Open
shairevivo wants to merge 2 commits into
fullsend-ai:mainfrom
shairevivo:fix/2093-post-review-empty-files
Open

fix(#2093): retry empty PR file list before refusing to approve#1196
shairevivo wants to merge 2 commits into
fullsend-ai:mainfrom
shairevivo:fix/2093-post-review-empty-files

Conversation

@shairevivo

Copy link
Copy Markdown
Contributor

What

Fixes the review agent failing genuinely non-empty PRs when the forge returns an empty changed-files list.

post-review.sh refuses to approve a PR when it cannot establish what changed (a safety net against blind approvals). But the file list can come back empty transiently: right after a merge-commit update, GitHub has not finished computing the diff, so forge_get_pr_files returns nothing and the agent aborts on a PR that actually has changes.

Changes

  • scripts/lib/github-review-ops.lib.shforge_get_pr_files now reads the paginated /pulls/{n}/files REST endpoint instead of the asynchronously-populated gh pr view --json files summary field. The files endpoint reflects the computed diff more directly.
  • scripts/post-review.src.sh — the empty-result guard now retries once (::notice:: + short sleep + re-fetch) before refusing to approve. The retry recovers from the transient race; a genuinely empty result still refuses to approve. This lives at the call site, so it protects every forge (GitHub + GitLab) regardless of which forge_get_pr_files implementation runs.
  • scripts/post-review-test.sh — mock gh updated for the new endpoint; added empty-pr-files-retry-recovers and empty-pr-files-retry-still-fails integration tests.
  • Regenerated the post-review.sh and pre-review.sh bundles (make script-build).

Verification

  • make script-build + make check-bundle — bundles in sync
  • bash scripts/post-review-test.sh — 106 pass, incl. both new tests
  • make lint (skillsaw) — Grade A, 0 errors/warnings
  • pre-commit run on changed files — shellcheck + secret scan pass

The remaining harness-jira-test.sh failures under make script-test are pre-existing and env-dependent (unset JIRA_* vars); they reproduce on a clean main and are unrelated to this change.

Note: the tracking issue lives in fullsend-ai/fullsend#2093, but the bug and fix are in this repo (the review scripts).

🤖 Generated with Claude Code

post-review.sh refuses to approve a PR when it cannot establish what
changed, guarding against blind approvals. But the file list can come
back empty transiently: right after a merge-commit update GitHub has
not finished computing the diff, so the review agent fails a genuinely
non-empty PR.

Two changes:

- forge_get_pr_files (GitHub) now reads the paginated
  /pulls/{n}/files REST endpoint instead of the async-populated
  `gh pr view --json files` summary field, which reflects the computed
  diff more directly.

- The call site retries once (notice + short sleep + re-fetch) before
  the empty-result guard. The retry recovers from a transient race; a
  genuinely empty result still refuses to approve. This applies to all
  forges since the guard is forge-agnostic.

Adds integration tests for retry-recovers and retry-still-fails, and
regenerates the post-review.sh / pre-review.sh bundles.

Fixes fullsend-ai/fullsend#2093

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Shai Revivo <srevivo@redhat.com>
@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown

Functional tests did not run

Functional tests run automatically for org/repo members and collaborators on pull requests.

For other contributors, a maintainer must add the ok-to-test label after the latest push.

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Retry transient empty PR file lists before blocking approval

🐞 Bug fix 🧪 Tests 🕐 20-40 Minutes

Grey Divider

AI Description

• Fetches GitHub changed files from the paginated REST endpoint.
• Retries transiently empty file lists once while preserving fail-closed approval behavior.
• Tests both successful recovery and persistent-empty rejection paths.
Diagram

graph TD
  B["Initial fetch"] --> C["Files endpoint"] --> D{"Files empty?"}
  D -- "Yes" --> E["Wait 10 seconds"] --> F["Retry fetch"] --> H{"Still empty?"}
  D -- "No" --> G["Approval checks"]
  H -- "No" --> G
  H -- "Yes" --> I["Refuse approval"]
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Retry inside each forge adapter
  • ➕ Keeps retry details close to forge-specific retrieval logic.
  • ➕ Could apply forge-specific delays and retry policies.
  • ➖ Duplicates safety behavior across GitHub and GitLab implementations.
  • ➖ Risks inconsistent fail-closed semantics between forges.
2. Poll with bounded backoff
  • ➕ Better tolerates forge diff computation lasting longer than ten seconds.
  • ➕ Supports multiple transient failures without immediately blocking approval.
  • ➖ Adds latency, configuration, and test complexity.
  • ➖ Could unnecessarily delay genuinely empty or consistently failing responses.

Recommendation: Keep the PR's single call-site retry. It provides bounded recovery across all forges, preserves the fail-closed guard, and avoids the complexity and approval latency of generalized polling. Using GitHub's paginated files endpoint also directly addresses the less reliable summary field.

Files changed (5) +172 / -8

Bug fix (4) +41 / -6
github-review-ops.lib.shFetch GitHub PR files through the paginated REST endpoint +7/-2

Fetch GitHub PR files through the paginated REST endpoint

• Replaces the asynchronously populated 'gh pr view --json files' summary with '/pulls/{n}/files'. Pagination and filename extraction provide a more direct changed-file source.

scripts/lib/github-review-ops.lib.sh

post-review.shBundle resilient file fetching and approval retry logic +17/-2

Bundle resilient file fetching and approval retry logic

• Regenerates the executable post-review bundle with the REST endpoint integration. Empty file lists now trigger a notice, ten-second delay, and one retry before approval is refused.

scripts/post-review.sh

post-review.src.shRetry empty PR file lists before failing closed +10/-0

Retry empty PR file lists before failing closed

• Adds one delayed, forge-agnostic retry when approval processing receives no changed files. The existing refusal remains in place when the second request is also empty.

scripts/post-review.src.sh

pre-review.shRegenerate pre-review bundle with REST file retrieval +7/-2

Regenerate pre-review bundle with REST file retrieval

• Updates the generated pre-review script to include the revised GitHub adapter. Changed files are now read from the paginated pull-request files endpoint.

scripts/pre-review.sh

Tests (1) +131 / -2
post-review-test.shCover transient and persistent empty file-list responses +131/-2

Cover transient and persistent empty file-list responses

• Updates the 'gh' mock for the REST files endpoint and supports different initial and retry responses. Adds integration tests proving transient emptiness recovers while repeated emptiness still blocks approval.

scripts/post-review-test.sh

@qodo-code-review

qodo-code-review Bot commented Sep 6, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (1)

Grey Divider


Action required

1. Protected scripts need human review 📜 Skill insight § Compliance
Description
scripts/post-review.src.sh changes the approval guard and its generated bundles, while every
modified file is under the protected scripts/ path. Issue #2093 explains the modification, but the
changed approval and forge-integration logic still reaches governance infrastructure reserved for
human review.
Code

scripts/post-review.src.sh[R242-244]

+    echo "::notice::PR files came back empty; retrying once in case of a transient forge data race (forge_get_pr_files)" >&2
+    sleep 10
+    PR_FILES=$(forge_get_pr_files)
Relevance

●●● Strong

Recent post-review precedents accept governance-path safeguards and configuration findings; issue
justification does not negate protected-script review requirements.

PR-#569

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
Compliance rule 1538392 designates scripts/ as a protected governance or infrastructure path and
requires a finding whenever such files change. The cited regions show changes to the source approval
guard, forge integration, tests, and generated scripts; the linked issue supplies justification but
does not remove the human-approval requirement.

scripts/post-review.src.sh[236-245]
scripts/lib/github-review-ops.lib.sh[53-60]
scripts/post-review-test.sh[1687-1791]
scripts/post-review.sh[651-660]
scripts/pre-review.sh[88-95]
Skill: pr-review

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
This PR modifies protected review scripts and therefore must not be approved automatically.

## Issue Context
The linked issue justifies the changes, but compliance rule 1538392 still requires human approval for modifications under `scripts/`.

## Fix Focus Areas
- scripts/post-review.src.sh[236-245]
- scripts/lib/github-review-ops.lib.sh[53-60]
- scripts/post-review-test.sh[1687-1791]
- scripts/post-review.sh[651-660]
- scripts/pre-review.sh[88-95]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

2. Review tests waste 30 seconds per run ✓ Resolved 🐞 Bug ➹ Performance
Description
The new integration cases invoke POST_SCRIPT after arranging an empty first response, but the
shared mock directory provides no sleep executable, so production's sleep 10 runs for real. The
recovery test, persistent-empty test, and pre-existing empty-list test each hit that branch, adding
at least 30 seconds to every serial test-suite run.
Code

scripts/post-review-test.sh[R1711-1714]

+    # First files call returns empty, the retry returns a real file.
+    export MOCK_PR_FILES_ON_RETRY="src/main.go"
+    export MOCK_FILES_CALL_MARKER="${TMPDIR}/marker-${test_name}"
+    bash "${POST_SCRIPT}"
Relevance

●●● Strong

Real sleeps make integration tests unnecessarily slow; recent scripts test precedents accept fixes
for inaccurate or incomplete harness behavior.

PR-#1109

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The integration harness prepends only MOCK_BIN to PATH, and its setup creates a gh mock but no
sleep mock. The production branch sleeps for ten seconds at scripts/post-review.src.sh:242-244,
while all three empty-list scenarios invoke that script at scripts/post-review-test.sh:1665-1666,
1711-1714, and 1764-1765.

scripts/post-review.src.sh[236-245]
scripts/post-review-test.sh[417-433]
scripts/post-review-test.sh[1643-1667]
scripts/post-review-test.sh[1691-1715]
scripts/post-review-test.sh[1745-1766]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The post-review integration tests execute the production ten-second retry delay instead of replacing `sleep` with a no-op, adding at least 30 seconds to each test-suite run.

## Issue Context
Three test cases produce an initially or persistently empty PR file list. Each invokes `post-review.sh`, whose retry branch calls `sleep 10`, while the test's mock `PATH` contains no mock `sleep` command.

## Fix Focus Areas
- scripts/post-review-test.sh[417-433]
- scripts/post-review-test.sh[1643-1667]
- scripts/post-review-test.sh[1691-1715]
- scripts/post-review-test.sh[1745-1766]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
✅ Compliance rules (platform): 57 rules
✅ Skills: 4 invoked
  code-review
  code-implementation
  pr-review
  docs-review
Review mode: ⚖️ Balanced: This changes approval-safety behavior and GitHub API file retrieval across runtime scripts, creating meaningful correctness and integration risk, but not enough independent logic for extended review.

Grey Divider

Tip of the day
💡 Did you know, you can show, collapse, or hide each part of a finding: code, evidence, and all

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread scripts/post-review.src.sh
Comment thread scripts/post-review-test.sh
The empty-PR-files retry branch calls `sleep 10` before re-fetching.
The integration tests prepend only MOCK_BIN to PATH and provided no
`sleep` mock, so the real sleep ran in all three empty-list cases
(retry-recovers, retry-still-fails, and the pre-existing
safety-net-independent test), adding ~30s to every serial suite run.

Add a no-op `sleep` mock to MOCK_BIN. The retry logic doesn't depend
on real elapsed time, so stubbing it keeps the tests deterministic and
fast without changing what they assert.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Shai Revivo <srevivo@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant