Skip to content

test: add integration suites that create real Droplets - #141

Open
tas50 wants to merge 2 commits into
mainfrom
feat/integration-suites
Open

tas50 wants to merge 2 commits into
mainfrom
feat/integration-suites

Conversation

@tas50

@tas50 tas50 commented Aug 30, 2026

Copy link
Copy Markdown
Member

Why

The unit suite stubs HTTP at the wire. That proves the driver sends the right request — it says nothing about whether DigitalOcean accepts it. Nothing in CI has ever reached the v2 API.

PLATFORM_SLUG_MAP is the sharpest example. It is a hardcoded claim about what is in DigitalOcean's image catalogue, spec/readme_spec.rb checks it against the README, and neither of those checks it against DigitalOcean. Half the map names images that have almost certainly been retired (centos-7, debian-9, fedora-32, ubuntu-16), and the only way anyone finds out is a user's 404. The same is true of normalize_list: the unit suite proves "a, b" becomes ["a", "b"] in the request body, and the mock happily accepts either.

What this adds

Ten suites in integration/, each creating a real Droplet and asserting on the Droplet that the driver configured it as asked:

Suite What it proves
default Create, converge over SSH, destroy — and that ubuntu-24 resolves to an image that still exists.
image-slug A platform name not in the map (debian-13-x64) reaches the API untouched.
server-name An explicit server_name survives to the Droplet unchanged.
tags tags as a YAML list reach the Droplet.
tags-string tags as a delimited string are split, not sent as one tag with a comma in it.
user-data user_data reaches cloud-init verbatim and runs.
ipv6 ipv6: true gets a routable v6 address configured on the interface.
monitoring monitoring: true installs and starts do-agent.
size-region A non-default size and region are accepted together — the 422 case.
firewalls attach_firewalls places the Droplet behind a firewall without locking the transport out.

Assertions live in the shell provisioner, not a verifier. The script is transferred over the driver's own transport and executed on the Droplet, so reaching the machine is part of every assertion, a non-zero exit fails the suite, and there is no verifier licence to satisfy. Each script reads the metadata service, which is as close as you can get to reading back the request the driver sent, and needs no credentials on the Droplet.

export DIGITALOCEAN_ACCESS_TOKEN=dop_v1_...
export DIGITALOCEAN_SSH_KEY_IDS=12345678
bundle exec rake integration:list
bundle exec rake integration:test
bundle exec rake integration:destroy   # after a failed run

Not part of rake default — these cost money.

CI

.github/workflows/integration.yml runs weekly against main and on demand, never on a pull request: secrets are unavailable to forks and every run costs money. It also guards on github.repository, so a fork's scheduled run does not fire.

Setup is one repository secret, DIGITALOCEAN_ACCESS_TOKEN. Everything else is created for the run and deleted after it — an ed25519 key pair uploaded to the account, and a cloud firewall for the firewalls suite — so there is no long-lived SSH key sitting in the repository secrets.

Three operational details that matter more than they look:

  • Destroy everything runs with if: always(). A suite that leaks Droplets on failure turns a red build into a recurring bill.
  • A tag sweep runs after it, also if: always(). Every Droplet is tagged run-<run_id>, so if kitchen destroy could not run at all — a cancelled job, a crashed runner — DELETE /v2/droplets?tag_name= is the way back.
  • concurrency: digitalocean-integration with cancel-in-progress: false. Droplet limits are per account; two overlapping runs exhaust them and both fail.

A full run is ten Droplets, nine of them s-1vcpu-1gb, alive for a few minutes each — cents, not dollars.

Notes for review

  • I have not run these against a live account; I do not have a token for the org's DigitalOcean tenant. The configuration is verified as far as it can be offline — kitchen list resolves all ten instances and kitchen diagnose shows every setting landing where it should — but a maintainer should trigger the workflow once before trusting the schedule. I would expect the first run to shake out at least one stale slug, which is the point.
  • integration/kitchen.yml is linted as YAML by the shared workflow before ERB runs, so every ERB expression in it uses single quotes internally.
  • The shell provisioner uploads only the one file it is pointed at, so each script repeats a short preamble rather than sourcing a helper, and per-run values arrive as arguments:.

Verification

$ bundle exec cookstyle --chefstyle
Inspecting 5 files
.....

5 files inspected, no offenses detected

$ bundle exec rake test
175 examples, 0 failures

$ bundle exec yamllint .
(clean)

$ cd integration && bundle exec kitchen list
Instance                  Driver        Provisioner  Verifier  Transport  Last Action    Last Error
default-ubuntu-24         Digitalocean  Shell        Shell     Ssh        <Not Created>  <None>
image-slug-debian-13-x64  Digitalocean  Shell        Shell     Ssh        <Not Created>  <None>
server-name-ubuntu-24     Digitalocean  Shell        Shell     Ssh        <Not Created>  <None>
tags-ubuntu-24            Digitalocean  Shell        Shell     Ssh        <Not Created>  <None>
tags-string-ubuntu-24     Digitalocean  Shell        Shell     Ssh        <Not Created>  <None>
user-data-ubuntu-24       Digitalocean  Shell        Shell     Ssh        <Not Created>  <None>
ipv6-ubuntu-24            Digitalocean  Shell        Shell     Ssh        <Not Created>  <None>
monitoring-ubuntu-24      Digitalocean  Shell        Shell     Ssh        <Not Created>  <None>
size-region-ubuntu-24     Digitalocean  Shell        Shell     Ssh        <Not Created>  <None>
firewalls-ubuntu-24       Digitalocean  Shell        Shell     Ssh        <Not Created>  <None>

tas50 added 2 commits August 29, 2026 18:49
The unit suite stubs HTTP at the wire. That proves the driver sends the
right request; it says nothing about whether DigitalOcean accepts it.
PLATFORM_SLUG_MAP, in particular, is a claim about the image catalogue
that nothing in CI has ever checked.

Ten suites under integration/, each creating a real Droplet and asserting
on the Droplet that the driver configured it as asked: default,
image-slug, server-name, tags, tags-string, user-data, ipv6, monitoring,
size-region and firewalls.

Assertions live in the shell provisioner rather than a verifier. The
script is transferred over the driver's own transport and run on the
Droplet, so reaching the machine is part of every assertion, a non-zero
exit fails the suite, and there is no verifier licence to satisfy. The
assertions read the metadata service, which is the closest thing to
reading back the request the driver sent.

.github/workflows/integration.yml runs them weekly against main and on
demand, never on a pull request: secrets are unavailable to forks and
every run costs money. It needs one secret. The SSH key pair and the
cloud firewall are created for the run and deleted afterwards, so no
long-lived key is stored. Destroy and a tag sweep both run with
if: always(), because a suite that leaks a Droplet turns a red build into
a recurring bill.

Signed-off-by: Tim Smith <tim@mondoo.com>
…line

Signed-off-by: Tim Smith <tim@mondoo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant