Skip to content

update test-and-build.yml according to reusable-docker-ecr.yml - #679

Merged
jtherrmann merged 3 commits into
developfrom
update-build-workflow
Nov 5, 2025
Merged

update test-and-build.yml according to reusable-docker-ecr.yml#679
jtherrmann merged 3 commits into
developfrom
update-build-workflow

Conversation

@jtherrmann

Copy link
Copy Markdown
Contributor

Updates test-and-build.yml to more closely align with reusable-docker-ecr.yml. Here are the remaining differences between those files:

--- reusable-docker-ecr.yml	2025-11-04 14:50:43.729847440 -0900
+++ test-and-build.yml	2025-11-04 14:53:25.997248249 -0900
@@ -1,33 +1,40 @@
+name: Test and build
+
 on:
-  workflow_call:
-    inputs:
-      version_tag:
-        required: true
-        type: string
-      ecr_registry:
-        required: true
-        type: string
-      aws_region:
-        required: false
-        default: us-west-2
-        type: string
-      file:
-        required: false
-        default: Dockerfile
-        type: string
-    secrets:
-      AWS_ACCESS_KEY_ID:
-        required: true
-      AWS_SECRET_ACCESS_KEY:
-        required: true
-
-concurrency:
-  group: ${{ inputs.version_tag }}
-  cancel-in-progress: true
+  push:
+    branches:
+      - develop
+    tags:
+      - 'v*'
+  pull_request:
+    branches:
+      - main
+      - develop
+
+env:
+  HYP3_REGISTRY: 845172464411.dkr.ecr.us-west-2.amazonaws.com
+  AWS_REGION: us-west-2
+  AWS_ACCESS_KEY_ID: ${{ secrets.V2_AWS_ACCESS_KEY_ID }}
+  AWS_SECRET_ACCESS_KEY: ${{ secrets.V2_AWS_SECRET_ACCESS_KEY }}
 
 jobs:
+  call-pytest-workflow:
+    uses: ASFHyP3/actions/.github/workflows/reusable-pytest.yml@v0.20.0
+    with:
+      local_package_name: hyp3_gamma
+      python_versions: >-
+        ["3.10"]
+
+  call-version-info-workflow:
+    uses: ASFHyP3/actions/.github/workflows/reusable-version-info.yml@v0.20.0
+    with:
+      python_version: '3.10'
+
   dockerize:
+    needs: call-version-info-workflow
     runs-on: ubuntu-latest
+    env:
+      SDIST_VERSION: ${{ needs.call-version-info-workflow.outputs.version_tag }}
     steps:
       - uses: actions/checkout@v5
         with:
@@ -35,6 +42,7 @@
 
       - name: set environment variables
         run: |
+          git fetch origin +refs/tags/*:refs/tags/*
           echo "CI_JOB_TIMESTAMP=$(date --utc --rfc-3339=seconds)" >> $GITHUB_ENV
 
       - name: Set up Docker Buildx
@@ -42,44 +50,45 @@
 
       - uses: aws-actions/configure-aws-credentials@v5
         with:
-          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
-          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
-          aws-region: ${{ inputs.aws_region }}
+          aws-access-key-id: ${{ env.AWS_ACCESS_KEY_ID }}
+          aws-secret-access-key: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          aws-region: ${{ env.AWS_REGION }}
 
       - name: Login to Amazon ECR
         uses: aws-actions/amazon-ecr-login@v2
 
-      - name: Lowercase repo for container registry
+      - name: Fetch GAMMA
         run: |
-          echo "REPO=${GITHUB_REPOSITORY,,}" >> ${GITHUB_ENV}
+          aws s3 cp s3://hyp3-software/GAMMA_SOFTWARE-20240627_MSP_ISP_DIFF_LAT.linux64_ubuntu2204.tar.gz .
+          tar -zxvf GAMMA_SOFTWARE-20240627_MSP_ISP_DIFF_LAT.linux64_ubuntu2204.tar.gz
 
       - name: Build, tag, and push image to Amazon ECR
         uses: docker/build-push-action@v6
         with:
           context: .
-          file: ./${{ inputs.file }}
-          push: ${{ ! github.event.pull_request.head.repo.fork }}
+          no-cache: true
+          push: true
           tags: |
-            ${{ inputs.ecr_registry }}/${{ env.REPO }}:${{ inputs.version_tag }}
+            ${{ env.HYP3_REGISTRY }}/${{ github.event.repository.name }}:${{ env.SDIST_VERSION }}
           labels: |
             org.opencontainers.image.created=${{ env.CI_JOB_TIMESTAMP }}
-            org.opencontainers.image.version=${{ inputs.version_tag }}
+            org.opencontainers.image.version=${{ env.SDIST_VERSION }}
             org.opencontainers.image.revision=${{ github.sha }}
 
       - name: Add test tag
         if: ${{ github.event_name != 'pull_request' && contains(inputs.version_tag, '.dev') }}
         uses: akhilerm/tag-push-action@v2.2.0
         with:
-          src: ${{ inputs.ecr_registry }}/${{ env.REPO }}:${{ inputs.version_tag }}
-          dst: ${{ inputs.ecr_registry }}/${{ env.REPO }}:test
+          src: ${{ env.HYP3_REGISTRY }}/${{ github.event.repository.name }}:${{ env.SDIST_VERSION }}
+          dst: ${{ env.HYP3_REGISTRY }}/${{ github.event.repository.name }}:test
 
       - name: Add latest tag
         if: ${{ github.event_name != 'pull_request' && ! contains(inputs.version_tag, '.dev') }}
         uses: akhilerm/tag-push-action@v2.2.0
         with:
-          src: ${{ inputs.ecr_registry }}/${{ env.REPO }}:${{ inputs.version_tag }}
-          dst: ${{ inputs.ecr_registry }}/${{ env.REPO }}:latest
+          src: ${{ env.HYP3_REGISTRY }}/${{ github.event.repository.name }}:${{ env.SDIST_VERSION }}
+          dst: ${{ env.HYP3_REGISTRY }}/${{ github.event.repository.name }}:latest
 
       - name: Logout of Amazon ECR
         if: always()
-        run: docker logout ${{ inputs.ecr_registry }}
+        run: docker logout ${HYP3_REGISTRY}

@jtherrmann
jtherrmann requested review from a team as code owners November 5, 2025 00:08
@jtherrmann jtherrmann added the bumpless Changes to documentation, CI/CD pipelines, etc that don't affect the project's version label Nov 5, 2025
@jtherrmann jtherrmann mentioned this pull request Nov 5, 2025
4 tasks

@jhkennedy jhkennedy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This largely matches the corresponding changes to the reusable workflow, which you can see here:
https://github.com/ASFHyP3/actions/compare/7ee86554edfff43d176979ccdfe326319a704531..957d7e3fe9a13f6fbf869ade98120668f3a8acd6#diff-48114c661f46b500e4b1c564f490a5f266757f59430de4676b2cfb1724221d2d

The only thing missing is the inclusion of the concurrency:

concurrency:
  group: ${{ inputs.version_tag }}
  cancel-in-progress: true

The reasoning behind setting the concurrency is described in the actions PR description:
ASFHyP3/actions#288

@jtherrmann
jtherrmann merged commit 7e5536b into develop Nov 5, 2025
10 checks passed
@jtherrmann
jtherrmann deleted the update-build-workflow branch November 5, 2025 02:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bumpless Changes to documentation, CI/CD pipelines, etc that don't affect the project's version

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants