From 33c9c17108a58f46ab8fb2077105e7b044681322 Mon Sep 17 00:00:00 2001 From: Tim Smith Date: Fri, 28 Aug 2026 16:59:32 -0700 Subject: [PATCH 1/2] test: verify the plugin works with Test Kitchen in CI These plugins exist to be run by Test Kitchen, and nothing in CI ever ran them that way. The cucumber features drive `busser` directly, which covers the plugin but skips the whole verifier path -- how Test Kitchen installs busser, installs the plugin, transfers the suite and invokes it. This adds a `kitchen verify` job that runs the real thing: * **`driver: exec` with `transport: exec`**, both shipped with Test Kitchen, so the "machine under test" is the CI runner. No Docker, no VM, no credentials, a couple of seconds per run. * **The real `busser` verifier**, pointed at the running Ruby rather than Chef's omnibus and told not to sudo for a path inside the workspace. * **`kitchen-preinstall.sh`** puts this working tree into the Busser root first, so the job tests the branch rather than the last release from RubyGems. `test/integration/default/` holds a small suite in the plugin's own language. ## Two things worth knowing about the setup **It deliberately does not use bundler.** No `bundler-cache`, no `bundle exec`. The verifier shells out to `gem list` to decide what to install, and bundler in the environment makes that report the bundle's gems rather than the Busser root's -- so it skips installing busser and the run dies on a path that was never created. Test Kitchen is not run from inside a project's bundle in real use either. **The pre-install has to install busser itself and run the postinstall.** The verifier's check is `grep "^busser"` with no anchor at the end, so a plugin named `busser-*` satisfies it and busser would never be installed. And skipping `busser plugin install` also skips the plugin's postinstall, which is where a plugin installs the framework it drives. Verified both directions locally: a passing suite exits 0, and a deliberately failing test makes `kitchen verify` exit 20. The suite includes a `helper.sh`, which is not named `*_test.sh` or `*_spec.sh` and so must never run. If the glob ever widens, it fails the suite loudly. Signed-off-by: Tim Smith --- .github/workflows/integration.yml | 50 +++++++++++++++++++ .gitignore | 1 + kitchen-preinstall.sh | 55 +++++++++++++++++++++ kitchen.yml | 23 +++++++++ test/integration/default/bash/helper.sh | 5 ++ test/integration/default/bash/smoke_test.sh | 6 +++ 6 files changed, 140 insertions(+) create mode 100644 .github/workflows/integration.yml create mode 100755 kitchen-preinstall.sh create mode 100644 kitchen.yml create mode 100644 test/integration/default/bash/helper.sh create mode 100644 test/integration/default/bash/smoke_test.sh diff --git a/.github/workflows/integration.yml b/.github/workflows/integration.yml new file mode 100644 index 0000000..b043f2f --- /dev/null +++ b/.github/workflows/integration.yml @@ -0,0 +1,50 @@ +--- +name: "Test Kitchen" + +"on": + push: + branches: [main] + pull_request: + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + +jobs: + kitchen: + name: "kitchen verify on Ruby ${{ matrix.ruby }}" + runs-on: ubuntu-latest + strategy: + fail-fast: false + matrix: + # The ends of the supported range. The unit and feature suites already + # cover everything in between; this job is about the Test Kitchen + # integration, not Ruby compatibility. + ruby: ["3.2", "4.0"] + env: + BUSSER_ROOT: ${{ github.workspace }}/tmp/busser-kitchen + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + # Deliberately no bundler-cache, and nothing below runs under `bundle + # exec`. The verifier shells out to `gem list` to decide what to install, + # and bundler in the environment makes that report the bundle's gems + # rather than the Busser root's -- so it skips installing busser and the + # run dies on a path that was never created. Test Kitchen is not run from + # inside a project's bundle in real use either. + - uses: ruby/setup-ruby@95ef2b042f9d7a56d8268cba8559e2842e2ad01b # v1.321.0 + with: + ruby-version: ${{ matrix.ruby }} + + - name: Install Test Kitchen + run: gem install test-kitchen --no-document --version "~> 4.1" + + - name: Install this working tree into the Busser root + run: ./kitchen-preinstall.sh + + - name: kitchen verify + run: kitchen verify diff --git a/.gitignore b/.gitignore index c23d5fa..6f8b8be 100644 --- a/.gitignore +++ b/.gitignore @@ -14,3 +14,4 @@ spec/reports test/tmp test/version_tmp tmp +.kitchen/ diff --git a/kitchen-preinstall.sh b/kitchen-preinstall.sh new file mode 100755 index 0000000..4f2576d --- /dev/null +++ b/kitchen-preinstall.sh @@ -0,0 +1,55 @@ +#!/usr/bin/env bash +# Installs this working tree into the Busser root the Test Kitchen verifier +# will use, so `kitchen verify` exercises the code on this branch rather than +# the last release from RubyGems. +# +# The verifier's install script skips its own install when `gem list` already +# shows what it wants, so putting things in place first is all that is needed. +# Two consequences to handle: +# +# * It also has to install busser itself. That check is `grep "^busser"` with +# no anchor at the end, so a plugin named busser-* satisfies it and busser +# would otherwise never be installed at all. +# * Skipping `busser plugin install` also skips the plugin's postinstall, +# which is where a plugin installs the test framework it drives. So run the +# postinstall here, which is what the verifier would have done. +set -euo pipefail + +BUSSER_ROOT="${BUSSER_ROOT:-/tmp/busser-kitchen}" +gemspec="$(ls ./*.gemspec)" +gem_name="$(basename "${gemspec}" .gemspec)" + +install_opts=( + --install-dir "${BUSSER_ROOT}/gems" + --bindir "${BUSSER_ROOT}/bin" + --no-document +) + +mkdir -p "${BUSSER_ROOT}/gems" + +gem build "${gemspec}" --output "${BUSSER_ROOT}/${gem_name}.gem" >/dev/null + +if [ "${gem_name}" = "busser" ]; then + # No --local: a plugin's runtime dependencies are things its runner needs on + # the machine under test, and they have to be fetched into the Busser root + # like everything else. + gem install "${BUSSER_ROOT}/${gem_name}.gem" "${install_opts[@]}" >/dev/null +else + gem install busser "${install_opts[@]}" >/dev/null + # Not --ignore-dependencies: a plugin's runtime dependencies are things its + # runner needs on the machine under test, and they have to be in the Busser + # root like everything else. + # No --local: a plugin's runtime dependencies are things its runner needs on + # the machine under test, and they have to be fetched into the Busser root + # like everything else. + gem install "${BUSSER_ROOT}/${gem_name}.gem" "${install_opts[@]}" >/dev/null + + # The same environment the verifier exports, so the framework this plugin + # installs lands in the Busser root rather than the ambient gem home. + BUSSER_ROOT="${BUSSER_ROOT}" \ + GEM_HOME="${BUSSER_ROOT}/gems" \ + GEM_PATH="${BUSSER_ROOT}/gems" \ + "${BUSSER_ROOT}/bin/busser" plugin install "${gem_name}" --force-postinstall +fi + +echo "pre-installed ${gem_name} from the working tree into ${BUSSER_ROOT}" diff --git a/kitchen.yml b/kitchen.yml new file mode 100644 index 0000000..f295476 --- /dev/null +++ b/kitchen.yml @@ -0,0 +1,23 @@ +--- +# Runs the bash runner the way Test Kitchen actually runs it -- real verifier, +# real busser, real suite transfer -- but against the machine running the tests +# rather than a VM or container, so it works unchanged in CI. +driver: + name: exec + +provisioner: + name: dummy + +verifier: + name: busser + # The verifier defaults to Chef's omnibus Ruby, which is not what is running + # here, and to sudo, which is not needed for a path under the workspace. + ruby_bindir: <%= RbConfig::CONFIG["bindir"] %> + sudo: false + root_path: <%= ENV.fetch("BUSSER_ROOT", "/tmp/busser-kitchen") %> + +platforms: + - name: local + +suites: + - name: default diff --git a/test/integration/default/bash/helper.sh b/test/integration/default/bash/helper.sh new file mode 100644 index 0000000..9f896b0 --- /dev/null +++ b/test/integration/default/bash/helper.sh @@ -0,0 +1,5 @@ +#!/usr/bin/env bash +# Not named *_test.sh or *_spec.sh, so busser-bash must never run it. If the +# glob ever widens, this fails the suite loudly. +echo "helper.sh should not have been executed" +exit 1 diff --git a/test/integration/default/bash/smoke_test.sh b/test/integration/default/bash/smoke_test.sh new file mode 100644 index 0000000..31bfe59 --- /dev/null +++ b/test/integration/default/bash/smoke_test.sh @@ -0,0 +1,6 @@ +#!/usr/bin/env bash +# Runs on the machine under test through busser-bash. Exit status is the verdict. +set -euo pipefail + +test -n "$(command -v bash)" +echo "busser-bash ran this script" From c7a659752af746da9f0fe1b9bba157a7c996331f Mon Sep 17 00:00:00 2001 From: Tim Smith Date: Fri, 28 Aug 2026 17:04:33 -0700 Subject: [PATCH 2/2] fix(ci): keep the Busser root outside the checkout The runners require "bundler/setup", which walks up from the suite looking for a Gemfile. With the Busser root under ${{ github.workspace }} it found this project's Gemfile and tried to materialize that bundle inside the isolated gem home, which cannot work: ```text Could not find cookstyle-9.0.0, aruba-2.4.1, cucumber-11.1.1, ... in locally installed gems (Bundler::GemNotFound) ``` A real Busser root is /opt/busser -- never inside the project -- so this was an artefact of the CI layout rather than anything the plugins do. Moving it to /tmp/busser-kitchen matches reality and fixes it. It only showed up on CI because a root outside any Gemfile tree, which is what I had locally, makes bundler/setup a no-op. Signed-off-by: Tim Smith --- .github/workflows/integration.yml | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/.github/workflows/integration.yml b/.github/workflows/integration.yml index b043f2f..e37a7db 100644 --- a/.github/workflows/integration.yml +++ b/.github/workflows/integration.yml @@ -26,7 +26,12 @@ jobs: # integration, not Ruby compatibility. ruby: ["3.2", "4.0"] env: - BUSSER_ROOT: ${{ github.workspace }}/tmp/busser-kitchen + # Outside the checkout on purpose. The runners require "bundler/setup", + # which walks up from the suite looking for a Gemfile -- so a Busser root + # inside the workspace finds this project's Gemfile and tries to + # materialize that bundle inside the isolated gem home, which fails. A + # real Busser root is /opt/busser, never inside the project. + BUSSER_ROOT: /tmp/busser-kitchen steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1