Skip to content

Commit 7cdac4e

Browse files
rfayclaude
andauthored
fix: fetch only new Drupal core commits on top of the seed cache (#262)
Co-authored-by: Claude <noreply@anthropic.com>
1 parent b46b234 commit 7cdac4e

4 files changed

Lines changed: 58 additions & 22 deletions

File tree

‎.github/workflows/drupal-integration-test.yml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -235,6 +235,16 @@ jobs:
235235
fi
236236
echo "OK: anonymous request redirected to Coder login (site is private by default)"
237237
238+
- &dump-setup
239+
# The checks above only report an exit code; the cause is in the
240+
# workspace's setup log, which is gone once the workspace is deleted.
241+
name: Dump setup status
242+
if: failure()
243+
run: |
244+
coder ssh ${{ env.WORKSPACE_NAME }} -- cat /home/coder/SETUP_STATUS.txt || true
245+
echo "--- last 60 lines of setup log ---"
246+
coder ssh ${{ env.WORKSPACE_NAME }} -- tail -60 /tmp/drupal-setup.log || true
247+
238248
- &delete-workspace
239249
name: Delete workspace
240250
if: always()
@@ -389,6 +399,7 @@ jobs:
389399
fi
390400
echo "OK: anonymous request reached the public Drupal site (drupal_site_public=true works)"
391401
402+
- *dump-setup
392403
- *delete-workspace
393404

394405
- name: Release staging box lock
@@ -576,6 +587,7 @@ jobs:
576587
fi
577588
echo "OK: anonymous request redirected to Coder login (site is private by default)"
578589
590+
- *dump-setup
579591
- name: Delete workspace
580592
if: always()
581593
run: coder delete ${{ env.WORKSPACE_NAME }} --yes || true
@@ -778,6 +790,7 @@ jobs:
778790
fi
779791
echo "OK: anonymous request redirected to Coder login (site is private by default)"
780792
793+
- *dump-setup
781794
- name: Delete workspace
782795
if: always()
783796
run: coder delete ${{ env.WORKSPACE_NAME }} --yes || true

‎docs/content/self-hosting/server-setup.md‎

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -880,10 +880,9 @@ This deploys three templates:
880880
The `drupal-core` template can provision workspaces faster using a **seed cache** on the host. The cache is a **bare** git clone of drupal/drupal. When a workspace starts, the startup script:
881881
882882
1. Creates a fresh git repo in the workspace directory
883-
2. Adds the bare cache as a local remote and fetches from it (fast — reuses local objects, no network)
884-
3. Removes the cache remote
885-
4. Fetches from `git.drupalcode.org` for any commits newer than the cache
886-
5. Checks out `main` (or the issue branch)
883+
2. Fetches the bare cache's branches and tags as `origin`'s (fast — reuses local objects, no network)
884+
3. Fetches from `git.drupalcode.org` for any commits newer than the cache
885+
4. Checks out `main` (or the issue branch), from the cache's refs if `git.drupalcode.org` can't be reached
887886
888887
This avoids downloading hundreds of MB per workspace and eliminates working-tree issues that a non-bare cache causes.
889888
@@ -897,7 +896,7 @@ The seed directory IS the bare clone root — no subdirectory nesting. No workin
897896
898897
### Install the hourly update timer
899898
900-
The update script runs `git fetch --all --prune` to keep the bare cache current. Because there is no working tree, no `git merge` is needed. Install it as an hourly systemd timer:
899+
The update script fetches every branch and tag into the bare cache to keep it current. Because there is no working tree, no `git merge` is needed. Install it as an hourly systemd timer:
901900
902901
```bash
903902
REPO=~/workspace/coder-ddev # adjust if your repo is elsewhere
@@ -1023,10 +1022,10 @@ The `update-drupal-cache` script handles both bare and non-bare repos automatica
10231022
**Update script fails:**
10241023
10251024
```bash
1026-
git -C ~/cache/drupal-core-seed fetch --all --prune
1025+
git -C ~/cache/drupal-core-seed fetch --prune origin '+refs/heads/*:refs/heads/*' '+refs/tags/*:refs/tags/*'
10271026
# If this fails, check network connectivity or re-clone:
10281027
rm -rf ~/cache/drupal-core-seed
1029-
git clone https://git.drupalcode.org/project/drupal.git ~/cache/drupal-core-seed
1028+
git clone --bare https://git.drupalcode.org/project/drupal.git ~/cache/drupal-core-seed
10301029
```
10311030
10321031
---

‎drupal-core/scripts/update-drupal-cache‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,8 @@
99
# the real remote, avoiding several hundred MB of network transfer per workspace.
1010
#
1111
# Using a bare repo is intentional: there is no working tree to keep in sync and
12-
# no "Your branch is behind" noise. git fetch --all --prune is all that is needed.
12+
# no "Your branch is behind" noise. Branches and tags are fetched straight into
13+
# refs/heads and refs/tags (a bare clone has no fetch refspec of its own).
1314
#
1415
# Initial setup (one-time, run as the seed-directory owner):
1516
# git clone --bare https://git.drupalcode.org/project/drupal.git ~/cache/drupal-core-seed
@@ -70,7 +71,15 @@ echo "Started: $(date)"
7071
echo ""
7172

7273
echo "Fetching Drupal core git objects..."
73-
git -C "$SEED_DIR" fetch --all --prune
74+
if [ "$IS_BARE" = "true" ]; then
75+
# `git clone --bare` configures no fetch refspec, so a plain
76+
# `git fetch --all` only writes FETCH_HEAD and leaves every branch where
77+
# the initial clone put it. Fetch branches and tags into place explicitly.
78+
git -C "$SEED_DIR" fetch --prune origin \
79+
'+refs/heads/*:refs/heads/*' '+refs/tags/*:refs/tags/*'
80+
else
81+
git -C "$SEED_DIR" fetch --all --prune
82+
fi
7483

7584
if [ "$IS_BARE" = "false" ]; then
7685
echo "Fast-forwarding local main branch..."

‎drupal-core/template.tf‎

Lines changed: 28 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -552,27 +552,36 @@ STATUS_HEADER
552552
_t=$SECONDS
553553
DRUPAL_REMOTE="https://git.drupalcode.org/project/drupal.git"
554554
if git -C "$CACHE_SEED" rev-parse --is-bare-repository 2>/dev/null | grep -q true; then
555-
# Bare cache: init repo, add all remotes (cache, origin, issue fork if
556-
# applicable), fetch cache first (fast local copy of all objects), then
557-
# fetch origin and issue fork for only the delta.
555+
# Bare cache: init repo, add origin (and the issue fork if applicable),
556+
# copy the cache's branches and tags in as origin's refs (fast local
557+
# copy of all objects), then fetch origin and issue fork for only the
558+
# delta.
558559
log_setup "Initialising Drupal core repo (bare cache + origin delta)..."
559560
mkdir -p "$DRUPAL_DIR"
560561
git -C "$DRUPAL_DIR" init -q >> "$SETUP_LOG" 2>&1
561-
git -C "$DRUPAL_DIR" remote add drupalcache "$CACHE_SEED"
562562
git -C "$DRUPAL_DIR" remote add origin "$DRUPAL_REMOTE"
563563
if [ "$USING_ISSUE_FORK" = "true" ] && [ -n "$ISSUE_FORK" ]; then
564564
# --no-tags: the fork's tags duplicate core's ~800 tags from origin.
565565
git -C "$DRUPAL_DIR" remote add --no-tags issue "https://git.drupalcode.org/issue/drupal-$${ISSUE_FORK}.git"
566566
fi
567567
# Fetches use -q so the log isn't flooded with one line per branch/tag
568568
# (core has ~800 refs); errors still go to stderr and land in the log.
569+
#
570+
# The cache's refs land directly under refs/remotes/origin/*, not under
571+
# a separate cache remote. That matters twice:
572+
# - The origin fetch below can only skip objects that local refs point
573+
# to. Refs under a temporary remote vanish with `git remote remove`,
574+
# which left nothing to negotiate with, so every workspace downloaded
575+
# all of core's history again (3-10 minutes, and sometimes failed).
576+
# - When the origin fetch fails, origin/main and the version branches
577+
# still exist from the cache, so the checkout below works.
569578
_f=$SECONDS
570-
git -C "$DRUPAL_DIR" fetch -q drupalcache >> "$SETUP_LOG" 2>&1
571-
log_setup " Fetched local cache ($((SECONDS - _f))s)"
572-
# Remove the local cache remote now — objects are in the local repo and
573-
# keeping it causes ambiguous-ref errors when checking out branches that
574-
# exist in both drupalcache and origin.
575-
git -C "$DRUPAL_DIR" remote remove drupalcache
579+
if git -C "$DRUPAL_DIR" fetch -q --no-tags "$CACHE_SEED" \
580+
'+refs/heads/*:refs/remotes/origin/*' '+refs/tags/*:refs/tags/*' >> "$SETUP_LOG" 2>&1; then
581+
log_setup " Fetched local cache ($((SECONDS - _f))s)"
582+
else
583+
log_setup " ⚠ Fetch from local cache failed — fetching everything from origin"
584+
fi
576585
_f=$SECONDS
577586
if git -C "$DRUPAL_DIR" fetch -q origin >> "$SETUP_LOG" 2>&1; then
578587
log_setup " Fetched origin ($((SECONDS - _f))s)"
@@ -587,7 +596,13 @@ STATUS_HEADER
587596
log_setup " ⚠ Fetch from issue fork failed"
588597
fi
589598
fi
590-
git -C "$DRUPAL_DIR" checkout -b main --track origin/main >> "$SETUP_LOG" 2>&1
599+
if ! git -C "$DRUPAL_DIR" checkout -b main --track origin/main >> "$SETUP_LOG" 2>&1; then
600+
# Without this, a failed checkout left an empty working tree that
601+
# setup reported as cloned, and the failure only surfaced later as
602+
# "Failed to install Drupal".
603+
log_setup "✗ Failed to check out main (no origin/main from cache or origin)"
604+
SETUP_FAILED=true
605+
fi
591606
elif [ -d "$CACHE_SEED/.git" ]; then
592607
# Non-bare (legacy) cache: use as --reference hint to avoid re-downloading
593608
# objects that already exist locally.
@@ -598,11 +613,11 @@ STATUS_HEADER
598613
log_setup "Cloning Drupal core..."
599614
git clone -q "$DRUPAL_REMOTE" "$DRUPAL_DIR" >> "$SETUP_LOG" 2>&1
600615
fi
601-
if [ -d "$DRUPAL_DIR/.git" ]; then
616+
if [ "$SETUP_FAILED" != "true" ] && [ -d "$DRUPAL_DIR/.git" ]; then
602617
log_setup "✓ Drupal core cloned ($((SECONDS - _t))s)"
603618
update_status "✓ Git clone: Success"
604619
else
605-
log_setup "✗ Failed to clone Drupal core"
620+
[ -d "$DRUPAL_DIR/.git" ] || log_setup "✗ Failed to clone Drupal core"
606621
update_status "✗ Git clone: Failed"
607622
SETUP_FAILED=true
608623
fi

0 commit comments

Comments
 (0)