Update and Improve the CDN Publishing Workflow#80
Open
cabutlermit wants to merge 4 commits into
Open
Conversation
❌ 1 blocking issue (1 total)
|
![qltysh[bot]](https://avatars.githubusercontent.com/in/890766?s=60&v=4){kind=small}
|
|
||
| - name: Configure AWS Credentials | ||
| id: aws_credentials | ||
| uses: aws-actions/configure-aws-credentials@v6 |
Contributor
Author
There was a problem hiding this comment.
We don't pin to SHAs, just to major versions.
Why these changes are being made: This started as just a simple "update the version of third party actions" and then turned into a refactor. The refactor cleans up the logic and DRYs the code a bit and requires NO changes from any caller workflow. How this addresses that need: * Update actions/checkout and aws-actions/configure-aws-credentials to the most recent versions * Clean up usage of environment variables in the various bash commands throughout the workflow * Add a "polling" step (using the `aws cloudfront wait` command) to ensure that the cache invalidation completes before the workflow finishes
Why these changes are being introduced: The previous commit was a simple cleanup of the shared CDN publishing workflow. This commit extends that work with a full refactor of the workflow that remains compatible with our existing caller workflows and sets the stage for future repositories that might need to publish content to the CDN (in particular, the future work to publish the NDE-tacos content to the CDN). How this addresses that need: * Create additional inputs to allow for more options for the the source and target of the `aws s3 sync` command * Update existing inputs with corrected "required" versus "optional" and default values * Update all the inputs with descriptions * Add a "validation" step to ensure that some of the freeform inputs are valid and fail the job quickly if they are not * Refine the "environment" and "domain" verifications and set environment variables with the correct combination of environment, domain, S3 bucket name, and CloudFront domain name (this also allows future caller workflows to not have to pass the bucket name when calling this workflow) * Using the new envrionment variables, simplify both the `aws s3 sync` command and the cache invalidation command Side effects of this change: None Relevant ticket(s): * https://mitlibraries.atlassian.net/browse/NDE-96
Why these changes are being made: Eventually, we will update the caller workflows, but for now we want to ensure that the old caller workflows still work even with all the changes to this shared workflow. How these changes are implemented: * Add a block to the validation step to parse the deprecated S3_URI input to set "legacy" environment variables for later steps and set a boolean to indicate if this workflow has been called by a legacy caller workflow * Update later steps with a conditional for the SOURCE_PATH variable to set it correctly for legacy and non-legacy workflows Side effects: None.
cabutlermit
force-pushed
the
update-cdn-publish
branch
from
adb854d to
a14748c
Compare
Why these changes are being introduced: First, we want to ensure that the only additional parameters supplied by the caller workflow are `--include` and `--exclude` parameters. Second, since we pass the parameters via an `env:` block, we have to be extra careful with double-quotes (which are required by the `aws s3 sync` command, but typically get stripped when expanded in a bash script). How these changes are implemented: * Add a block to the validation step to throw an error if there is any other parameter in the SYNC_PARAMS input outside of `--include` and `--exclude` * If the SYNC_PARAMS is valid, merge it together with the other stock `--exclude` parameters and set one string in the GITHUB_ENV with the full list of inclues and excludes * Add the `eval` command to properly expand the `aws s3 sync` command and preserve the double-quotes where they are required Side effects: None.
cabutlermit
force-pushed
the
update-cdn-publish
branch
from
a14748c to
2a4e2ba
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose and background context
Overview
The end result is a shared workflow that is backward compatible with the existing caller workflows from our "static site" repositories AND is ready to use for any future repositories that need to publish objects to paths in our CDN. It fails fast on invalid input values, has flexibility for both source paths (in the caller repository) and target paths (in the S3 bucket backing the CDN), waits until the cache invalidation is complete before finishing the workflow run, and has cleaner output to the Action summary page.
For more details about the CloudFront Distributions, see the Reference: CDN, CloudFront, Web Content document in Confluence.
Details
The
actions/checkoutandaws-actions/configure-aws-credentialsare updated to the latest versions and their parameters are updated for clarity and security (e.g., no persistinggitcredentials after the initial checkout).The whole workflow received a number of updates, from cleaning up and expanding the caller workflow inputs, to a new step to run through validation on a number of the inputs to ensure that the values are usable and if they are not, the workflow will fail fast. At the same time, the logic of the workflow was completely revamped to DRY out the code and move input & variable expansion from the
run:blocks intoenv:blocks (this prevents injection attacks). Polling was added to the cache invalidation step to only let the workflow finish after the cache invalidation step is complete.Knowing that this workflow will be used by future repositories to push content into the CDN, the whole concept of source & target for the
aws s3 sync ...command was reworked from the old assumption that the source was always the root of the repository. Moving forward, this shared workflow expectsSOURCE_PATHandTARGET_PATHinput variables instead.Finally, extra bits were added to the workflow to ensure that our existing caller workflows that still pass just the
S3_URIvalue will still work as expected. Those caller workflows will eventually need to be updated, but for now, they will still work properly.How can a reviewer manually see the effects of these changes?
During the development of these changes, the caller workflows from future-of-libraries-static and web-images-static were run to verify compatibility. Each of those repositories has a feature branch named
test-publish-workflowthat updates thedev-cdn-publish.ymlworkflow to point to this feature branch for the shared workflow.A review can visit one of those two repositories, select the Actions tab, select the DEV Publish workflow, and manually run that workflow from the
test-publish-workflowbranch. That will call the shared workflow from this branch.Includes new or updated dependencies?
YES: New versions of third-party Actions
Changes expectations for external applications?
NO: This is 100% backwards compatible for existing caller workflows
Code review