Skip to content

Update and Improve the CDN Publishing Workflow #80

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
cabutlermit wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Update and Improve the CDN Publishing Workflow #80

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4b1c723
Update and Improve CDN Publish
cabutlermit May 22, 2026
f011a62
Enhance the CDN Publishing Workflow
cabutlermit May 26, 2026
98f7d25
Ensure Backwards Compatibility
cabutlermit May 29, 2026
2a4e2ba
Validate and Construct sync Parameters
cabutlermit May 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
263 changes: 189 additions & 74 deletions .github/workflows/cdn-shared-publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,24 +5,47 @@ on:
workflow_call:
inputs:
AWS_REGION:
required: true
description: "Region for AWS resources."
required: false
type: string
GHA_ROLE:
required: true
default: us-east-1
Comment thread
qltysh[bot] marked this conversation as resolved.
DOMAIN:
description: "Indicates the standard CDN or a custom domain for the URL to the CDN (only standard or custom accepted)."
required: false
type: string
default: standard
ENVIRONMENT:
description: "The AWS environment where the resources will be deployed."
required: true
type: string
S3URI:
GHA_ROLE:
description: "The IAM Role linked to the OIDC connection."
required: true
type: string
DOMAIN:
S3URI:
description: "Legacy (deprecated) full S3 URI for the sync target in AWS."
required: false
type: string
default: standard
SOURCE_PATH:
description: "The path in the caller repository containing the files to sync to the S3 bucket."
required: false
type: string
default: .
SYNC_PARAMS:
description: "Additional parameters for the aws s3 sync command, specific to the caller repository."
required: false
type: string
TARGET_PATH:
description: "The prefix in the S3 bucket to which the repository files should be synced (must start with slash)."
required: false
type: string
default: /


permissions:
# These are the minimum permissions to allow for OIDC connection to AWS
id-token: write
contents: read

# Set defaults
defaults:
Expand All @@ -31,97 +54,189 @@ defaults:

jobs:
publish:
name: Publish content to CDN
# Start with validating the inputs from the caller workflow and prepping
# environment variables for the synchronization job.
name: Publish
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read

steps:
- uses: actions/checkout@v4

- name: DEV Configure AWS credentials
# Only run this step if the environment is "dev"
if: ${{ inputs.ENVIRONMENT == 'dev' }}
uses: aws-actions/configure-aws-credentials@v4
- uses: actions/checkout@v6
Comment thread
qltysh[bot] marked this conversation as resolved.
with:
role-to-assume: arn:aws:iam::${{ secrets.AWS_ACCT_DEV }}:role/${{ inputs.GHA_ROLE }}
aws-region: ${{ inputs.AWS_REGION }}

- name: STAGE Configure AWS credentials
# Only run this step if the environment is "stage"
if: ${{ inputs.ENVIRONMENT == 'stage' }}
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::${{ secrets.AWS_ACCT_STAGE }}:role/${{ inputs.GHA_ROLE }}
aws-region: ${{ inputs.AWS_REGION }}
persist-credentials: false

- name: PROD Configure AWS credentials
# Only run this step if the environment is "prod"
if: ${{ inputs.ENVIRONMENT == 'prod' }}
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::${{ secrets.AWS_ACCT_PROD }}:role/${{ inputs.GHA_ROLE }}
aws-region: ${{ inputs.AWS_REGION }}

- name: Sync custom domain CDN S3 content
# Only run this step if this is custom domain content (e.g., a folder at the root of bucket)
if: ${{ inputs.DOMAIN == 'custom' }}
- name: Validate
# Verify that the DOMAIN & ENVIRONMENT inputs are using the correct
# values. Verify that the SOURCE_PATH and TARGET_PATH inputs are
# formatted correctly and ensure this supports legacy caller workflows.
# Validate the SYNC_PARAMS input to only allow `--exclude` and
# `--include` parameters and then construct a single VALID_SYNC_PARAMS
# environment variable with proper quoting for use in the sync step.
id: validate
env:
DOMAIN: ${{ inputs.DOMAIN }}
ENVIRONMENT: ${{ inputs.ENVIRONMENT }}
SOURCE_PATH: ${{ inputs.SOURCE_PATH }}
S3URI: ${{ inputs.S3URI }}
TARGET_PATH: ${{ inputs.TARGET_PATH }}
SYNC_PARAMS: ${{ inputs.SYNC_PARAMS }}
run: |
if [ '${{ inputs.SYNC_PARAMS }}' != '' ]; then
aws s3 sync . ${{ inputs.S3URI }} --delete --exclude ".github/*" --exclude ".git/*" --exclude ".gitignore" ${{ inputs.SYNC_PARAMS }}
case "$DOMAIN" in
standard|custom)
echo "Valid DOMAIN=$DOMAIN input, proceed."
;;
*)
echo "Invalid DOMAIN=$DOMAIN input, exiting."
exit 1
;;
esac
case "$ENVIRONMENT" in
dev|stage|prod)
echo "Valid ENVIRONMENT=$ENVIRONMENT input, proceed."
;;
*)
echo "Invalid ENVIRONMENT=$ENVIRONMENT input, exiting."
exit 1
;;
esac
if [[ "${SOURCE_PATH:0:1}" == "." ]]; then
echo "Valid SOURCE_PATH=$SOURCE_PATH, proceed."
else
aws s3 sync . ${{ inputs.S3URI }} --delete --exclude ".github/*" --exclude ".git/*" --exclude ".gitignore"
echo "Invalid SOURCE_PATH=$SOURCE_PATH, exiting."
exit 1
fi
echo "Content is synchronized to ${{ inputs.S3URI }}" >> $GITHUB_STEP_SUMMARY

- name: Sync standard CDN S3 content
# Only run this step if this is standard content (e.g., a subfolder of the cdn/ folder)
if: ${{ inputs.DOMAIN == 'standard' }}
run: |
if [ '${{ inputs.SYNC_PARAMS }}' != '' ]; then
aws s3 sync ./$(echo ${{ inputs.S3URI }} | awk -F/ '{print $5}') ${{ inputs.S3URI }} --delete --exclude ".github/*" --exclude ".git/*" --exclude ".gitignore" ${{ inputs.SYNC_PARAMS }}
if [[ "$S3URI" == "" ]]; then
if [[ "${TARGET_PATH:0:1}" == "/" ]]; then
echo "Valid TARGET_PATH=$TARGET_PATH, proceed."
else
echo "Invalid TARGET_PATH=$TARGET_PATH, exiting."
exit 1
fi
else
aws s3 sync ./$(echo ${{ inputs.S3URI }} | awk -F/ '{print $5}') ${{ inputs.S3URI }} --delete --exclude ".github/*" --exclude ".git/*" --exclude ".gitignore"
echo "Legacy caller workflow that passed an S3_URI value."
if [[ "$DOMAIN" == "standard" ]]; then
echo "LEGACY_TARGET_PATH=/$(echo "$S3URI" | awk -F/ '{print $5}')" >> $GITHUB_ENV
echo "LEGACY_SOURCE_PATH=$(echo "$S3URI" | awk -F/ '{print $5}')" >> $GITHUB_ENV
else
echo "LEGACY_TARGET_PATH=/$(echo "$S3URI" | awk -F/ '{print $4}')" >> $GITHUB_ENV
fi
echo "LEGACY=true" >> $GITHUB_ENV
fi
echo "Content is synchronized to ${{ inputs.S3URI }}" >> $GITHUB_STEP_SUMMARY

- name: Invalidate cache
run: |
if [ '${{ inputs.DOMAIN }}' == 'standard' ]; then
aws cloudfront create-invalidation --distribution-id $(aws ssm get-parameter --name "/tfvars/libraries-website/standard-cdn-id" --query 'Parameter.Value' --output text) --paths "/*"
echo "The cache for the $(echo ${{ inputs.S3URI }} | awk -F/ '{print $5}') folder has been cleared." >> $GITHUB_STEP_SUMMARY
if [[ -n "$SYNC_PARAMS" ]]; then
temp_params="${SYNC_PARAMS//--include/}"
temp_params="${temp_params//--exclude/}"
# If there's still a -- in there, it's an invalid flag
if [[ $temp_params =~ -- ]]; then
echo "Invalid SYNC_PARAMS: only --include and --exclude parameters are allowed, exiting."
exit 1
fi
echo "Valid SYNC_PARAMS, proceed."
echo "VALID_SYNC_PARAMS=--exclude \".github/*\" --exclude \".git/*\" --exclude \".gitignore\" $SYNC_PARAMS" >> $GITHUB_ENV
else
aws cloudfront create-invalidation --distribution-id $(aws ssm get-parameter --name "/tfvars/libraries-website/custom-cdn-id" --query 'Parameter.Value' --output text) --paths "/$(echo ${{ inputs.S3URI }} | awk -F/ '{print $5}')/*"
echo "The cache for the $(echo ${{ inputs.S3URI }} | awk -F/ '{print $4}') site has been cleared." >> $GITHUB_STEP_SUMMARY
echo "VALID_SYNC_PARAMS=--exclude \".github/*\" --exclude \".git/*\" --exclude \".gitignore\"" >> $GITHUB_ENV
fi

- name: Generate DEV Summary
# Only run this step if the environment is "dev"
if: ${{ inputs.ENVIRONMENT == 'dev' }}
- name: Set Environment
# Prepare environment variables for the synchronization job.
id: env
env:
AWS_DEV_ACCT: ${{ secrets.AWS_ACCT_DEV }}
AWS_STAGE_ACCT: ${{ secrets.AWS_ACCT_STAGE }}
AWS_PROD_ACCT: ${{ secrets.AWS_ACCT_PROD }}
ENVIRONMENT: ${{ inputs.ENVIRONMENT }}
GHA_ROLE: ${{ inputs.GHA_ROLE }}
run: |
case "$ENVIRONMENT" in
dev)
echo "AWS_ROLE=arn:aws:iam::$AWS_DEV_ACCT:role/$GHA_ROLE" >> $GITHUB_ENV
echo "CDN_DOMAIN=dev1.mitlibrary.net" >> $GITHUB_ENV
echo "AWS_ROLE and CDN_DOMAIN set for synchronization job to Dev1"
;;
stage)
echo "AWS_ROLE=arn:aws:iam::$AWS_STAGE_ACCT:role/$GHA_ROLE" >> $GITHUB_ENV
echo "CDN_DOMAIN=stage.mitlibrary.net" >> $GITHUB_ENV
echo "AWS_ROLE and CDN_DOMAIN set for synchronization job to Stage-Workloads"
;;
prod)
echo "AWS_ROLE=arn:aws:iam::$AWS_PROD_ACCT:role/$GHA_ROLE" >> $GITHUB_ENV
echo "CDN_DOMAIN=libraries.mit.edu" >> $GITHUB_ENV
echo "AWS_ROLE and CDN_DOMAIN set for synchronization job to Prod-Workloads"
;;
esac

- name: Configure AWS Credentials
id: aws_credentials
uses: aws-actions/configure-aws-credentials@v6
Copy link
Copy Markdown

@qltysh qltysh Bot Jun 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unpinned action reference [zizmor:zizmor/unpinned-uses]

Copy link
Copy Markdown
Contributor Author

@cabutlermit cabutlermit Jun 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't pin to SHAs, just to major versions.

with:
aws-region: ${{ inputs.AWS_REGION }}
role-to-assume: ${{ env.AWS_ROLE }}

- name: Set S3 Target URI
# Set the correct S3 URI for the synchronization job
id: s3_target
env:
AWS_REGION: ${{ inputs.AWS_REGION }}
DOMAIN: ${{ inputs.DOMAIN }}
TARGET_PATH: ${{ env.LEGACY && env.LEGACY_TARGET_PATH || inputs.TARGET_PATH }}
run: |
if [ '${{ inputs.DOMAIN }}' == 'standard' ]; then
echo "The updates to https://cdn.dev1.mitlibrary.net/$(echo ${{ inputs.S3URI }} | awk -F/ '{print $5}') are now available" >> $GITHUB_STEP_SUMMARY
BUCKET=$(aws ssm get-parameter \
--region "$AWS_REGION" \
--name "/tfvars/libraries-website/cdn-origin-bucket-name" \
--query 'Parameter.Value' \
--output text)
if [[ "$DOMAIN" == "standard" ]]; then
echo "DISTRIBUTION_ID=$(aws ssm get-parameter \
--name "/tfvars/libraries-website/standard-cdn-id" \
--query 'Parameter.Value' \
--output text)" >> $GITHUB_ENV
echo "S3_URI=s3://$BUCKET/cdn$TARGET_PATH/" >> $GITHUB_ENV
else
echo "The updates to https://$(echo ${{ inputs.S3URI }} | awk -F/ '{print $4}').dev1.mitlibrary.net site are now available" >> $GITHUB_STEP_SUMMARY
echo "DISTRIBUTION_ID=$(aws ssm get-parameter \
--name "/tfvars/libraries-website/custom-cdn-id" \
--query 'Parameter.Value' \
--output text)" >> $GITHUB_ENV
echo "S3_URI=s3://$BUCKET$TARGET_PATH/" >> $GITHUB_ENV
fi

- name: Generate STAGE Summary
# Only run this step if the environment is "stage"
if: ${{ inputs.ENVIRONMENT == 'stage' }}
- name: Sync To CDN S3 Bucket
env:
S3_URI: ${{ env.S3_URI }}
SOURCE_PATH: ${{ env.LEGACY && env.LEGACY_SOURCE_PATH || inputs.SOURCE_PATH }}
VALID_SYNC_PARAMS: ${{ env.VALID_SYNC_PARAMS }}
run: |
if [ '${{ inputs.DOMAIN }}' == 'standard' ]; then
echo "The updates to https://cdn.stage.mitlibrary.net/$(echo ${{ inputs.S3URI }} | awk -F/ '{print $5}') are now available" >> $GITHUB_STEP_SUMMARY
echo "### Content synchronization to $S3_URI." >> $GITHUB_STEP_SUMMARY
if [[ "$S3_URI" == *"cdn/"* ]]; then
echo "Standard CDN content is synchronizing"
else
echo "The updates to https://$(echo ${{ inputs.S3URI }} | awk -F/ '{print $4}').stage.mitlibrary.net site are now available" >> $GITHUB_STEP_SUMMARY
echo "Custom CDN content is synchronizing"
fi
cd "$GITHUB_WORKSPACE"
eval "aws s3 sync \"$SOURCE_PATH\" \"$S3_URI\" \
--delete \
$VALID_SYNC_PARAMS"
echo "Content is synchronized to $S3_URI." >> $GITHUB_STEP_SUMMARY

- name: Generate PROD Summary
# Only run this step if the environment is "prod"
if: ${{ inputs.ENVIRONMENT == 'prod' }}
- name: Invalidate cache
env:
CDN_DOMAIN: ${{ env.CDN_DOMAIN }}
DISTRIBUTION_ID: ${{ env.DISTRIBUTION_ID }}
DOMAIN: ${{ inputs.DOMAIN }}
TARGET_PATH: ${{ env.LEGACY && env.LEGACY_TARGET_PATH || inputs.TARGET_PATH }}
run: |
if [ '${{ inputs.DOMAIN }}' == 'standard' ]; then
echo "The updates to https://cdn.libraries.mit.edu/$(echo ${{ inputs.S3URI }} | awk -F/ '{print $5}') are now available" >> $GITHUB_STEP_SUMMARY
echo "### CDN cache invalidation" >> $GITHUB_STEP_SUMMARY
echo "Start CDN Cache invalidation."
INVALIDATION_ID=$(aws cloudfront create-invalidation \
--distribution-id "$DISTRIBUTION_ID" \
--paths "$TARGET_PATH" \
--query 'Invalidation.Id' \
--output text)
aws cloudfront wait invalidation-completed \
--distribution-id "$DISTRIBUTION_ID" \
--id "$INVALIDATION_ID"
echo "The cache has been cleared." >> $GITHUB_STEP_SUMMARY
if [[ "$DOMAIN" == "standard" ]]; then
echo "The updates to https://cdn.$CDN_DOMAIN$TARGET_PATH are now available." >> $GITHUB_STEP_SUMMARY
else
echo "The updates to https://$(echo ${{ inputs.S3URI }} | awk -F/ '{print $4}').libraries.mit.edu site are now available" >> $GITHUB_STEP_SUMMARY
echo "The updates to the https://$TARGET_PATH.$CDN_DOMAIN site are now available." >> $GITHUB_STEP_SUMMARY
fi
Loading