fix: survive the API failures that used to leave Droplets running

[tas50]

Why

The driver's error handling assumes an API that always answers, answers promptly, and never changes its mind between two calls. The live one does none of those. Every problem below ends a run with a Droplet already created and billing, which is the expensive kind of bug.

I found these while writing the integration suites in #141 — reading the driver against droplet_kit's actual error surface rather than against the mocks, which encode the same assumptions the code does.

What was wrong

api_call only rescued DropletKit::Error. droplet_kit uses Faraday's net_http adapter, and faraday-net_http turns Errno::ECONNRESET, Net::OpenTimeout, OpenSSL::SSL::SSLError and friends into Faraday::ConnectionFailed / Faraday::TimeoutError / Faraday::SSLError. None of those is a DropletKit::Error, so they went straight past every rescue in the file:

• find_droplet's allow_missing: contract silently did not apply
• lookup_droplet's rescue Kitchen::ActionFailed did not catch it, so #status could raise out
• wait_until polls for up to 600 seconds; one reset connection anywhere in that window killed create or destroy

A 429 was a hard failure. DigitalOcean allows 250 requests a minute. wait_for_public_ip polls every 8s for up to 600s — ~75 GET /v2/droplets/:id per instance. A kitchen test across a dozen platforms is over a thousand requests inside a ten-minute window. Meeting the ceiling is routine, not exotic, and droplet_kit hands back #reset_at, #limit and #remaining on the exception, all of which the driver threw away.

destroy's delete omitted allow_missing: true. There is a window between the last find_droplet poll and the delete in which someone else — a colleague, a cleanup script, the control panel — can remove the Droplet. DELETE then returns 404, which raised, left :server_id in the state file, and made every later kitchen destroy fail the same way against a Droplet that was already gone.

doctor asked a different question from create_server. doctor used Array(config[:ssh_key_ids]).empty?; create_server used normalize_list. They disagree:

ssh_key_ids	required_config	doctor	actually sent
" "	passes	no problem	[] — no key
","	passes	no problem	[] — no key

DigitalOcean then mails a root password, the transport has no way in, and wait_until_ready hangs until it times out. That is precisely the failure the README's own troubleshooting section describes, so doctor is the thing that should catch it. doctor also reported an unreachable API as DigitalOcean rejected the configured access token: Failed to open TCP connection..., sending people off to regenerate a credential that was fine.

#status called super. Kitchen::Driver::Base#status does not exist before test-kitchen 4.1 — I checked 3.0.0, 3.7.0, 3.9.1 and v4.0.0 — while the gemspec says >= 3.0. Latent, because nothing below 4.1 calls driver.status, but the gem declared a floor its own advertised feature could not honour. Introduced by #138 after #137 set the floor.

vpcs is plural and takes one UUID. DigitalOcean's schema is vpc_uuid: {type: string}, singular. But the key is plural, and ssh_key_ids, tags and firewalls all accept YAML lists, so people write the obvious thing and get "vpc_uuid":["..."] on the wire → 422.

normalize_list returned nil into the request body. tags: {env: prod} in a kitchen.yml serialized as "tags": null, which DigitalOcean accepts and silently ignores. attach_firewalls warned about this case; create_server did not.

What changed

• api_call rescues Faraday::Error and retries with exponential backoff, and rescues DropletKit::RateLimitReached separately, sleeping until RateLimit-Reset says the window reopens (capped at a minute so a run does not sit silent).
• New idempotent: keyword. A Droplet create is never retried on a dropped connection — the request may well have arrived, and a second attempt bills for a Droplet Test Kitchen has no record of and will never destroy. It raises and tells you to check the account first. A 429 is retried on a create, because a rate-limited request definitively did not happen.
• destroy's delete passes allow_missing: true.
• doctor uses normalize_list, and reports an unreachable API as unreachable.
• #status answers for itself instead of delegating, and says what is actually unknown ("DigitalOcean does not know a Droplet <42>") rather than the base class's "does not support status checks", which is no longer true.
• vpcs accepts a list, uses the first, and warns when there is more than one.
• A list-shaped setting of an unreadable type warns instead of sending null.

24 new examples. The retry paths had no coverage at all before — no 429, RateLimit, Faraday, ConnectionFailed or Timeout appeared anywhere in spec/.

Not in this PR

PLATFORM_SLUG_MAP is badly stale — 15 of its 25 entries name images DigitalOcean has retired, including ubuntu-20 and every Debian below 13. That is a separate change and I will send it separately.

Verification

$ bundle exec cookstyle --chefstyle
Inspecting 5 files
.....

5 files inspected, no offenses detected

$ bundle exec rake test
199 examples, 0 failures

(175 before this PR.)

[tas50]

CI caught something worth recording, because it changes what the code is for.

The first push was green on Ruby 4.0 and red on 3.1, 3.2 and 3.4 — the same code, three different answers. The cause is dependency resolution, not Ruby: the gemspec allows droplet_kit >= 3.7, < 4.0, my lockfile had 3.19.0, and CI resolved 3.22.0. From 3.20 onwards droplet_kit installs a Faraday retry middleware of its own, and it was absorbing the 429s before the driver ever saw one, so the driver's sleep was never called and three assertions failed.

That middleware is not a substitute for this PR:

methods         = [:delete, :get, :head, :options, :put]   # no :post
retry_statuses  = [429]
max             = 3
interval        = 0.0
backoff_factor  = 1.0

It skips POST, so a rate-limited create is never retried by it. And interval = 0.0 with backoff_factor = 1.0 means three immediate retries inside the same rate-limited moment — which cannot succeed, because nothing has reset. This PR waits for the window RateLimit-Reset actually names.

So the fix was in the specs, not the driver. The three that assert how long the driver waits now go through the create, which droplet_kit's middleware never touches, so they hold on either side of 3.20. The one that asserts a 429 does not fail the run sends four in a row, enough to outlast the middleware's three. Both behaviours are now pinned on every droplet_kit the gemspec allows, rather than on whichever one happens to resolve. The reason is in a comment in api_call too, since it is the sort of thing that looks like redundant belt-and-braces to the next reader.

$ bundle exec rake test
199 examples, 0 failures

$ bundle exec cookstyle --chefstyle
5 files inspected, no offenses detected

Locally that is against droplet_kit 3.19.0; CI covers 3.22.0.