Bump elastic-agent-libs and enforce fips140=on at runtime for FIPS builds

[macdewee]

What is the problem this PR solves?

Fleet Server's FIPS builds already moved to Go's native FIPS 140-3 support (GOFIPS140), but two gaps remained compared to the same migration done in beats/elastic-agent:

• elastic-agent-libs was still pinned to v0.44.0, which predates a fix enforcing FIPS 140-3 peer-certificate key-type checks across all TLS verification modes. Fleet Server uses this library's tlscommon/httpcommon packages for all of its TLS configuration, so FIPS builds could accept non-compliant TLS certificates (e.g. RSA keys below 2048 bits).
• FIPS binaries never actually enforced fips140=on at runtime by default — checkFIPSBinary only verified the binary was compiled with GOFIPS140 available, not that it enforces FIPS mode when run.

How does this PR solve the problem?

• Bumped elastic-agent-libs to v0.46.0 in go.mod and testing/go.mod, regenerating NOTICE.txt/NOTICE-fips.txt.
• Added a //go:debug fips140=on directive in a new requirefips-gated file (godebug_fips.go), so FIPS binaries embed DefaultGODEBUG=fips140=on and enforce FIPS mode by default at runtime.
• Extended checkFIPSBinary in magefile.go to fail if a FIPS binary's build info doesn't show DefaultGODEBUG containing fips140=on.

How to test this PR locally

FIPS=true PLATFORMS=linux/amd64 mage build:local
go version -m bin/fleet-server | grep -E 'GOFIPS140|DefaultGODEBUG|-tags'

Confirm DefaultGODEBUG=fips140=on is present, and that a non-FIPS build (mage build:local without FIPS=true) shows none of these markers.

mage test:unit and mage check:imports check:headers check:fix check:notice all pass.

Design Checklist

• I have ensured my design is stateless and will work when multiple fleet-server instances are behind a load balancer.
• I have or intend to scale test my changes, ensuring it will work reliably with 100K+ agents connected.
• I have included fail safe mechanisms to limit the load on fleet-server: rate limiting, circuit breakers, caching, load shedding, etc.

Checklist

• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool

Related issues

[mergify]

This pull request does not have a backport label. Could you fix it @macdewee? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-./d./d is the label to automatically backport to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[mergify]

This pull request is now in conflicts. Could you fix it @macdewee? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b drosiek-fips-native-go upstream/drosiek-fips-native-go
git merge upstream/main
git push upstream drosiek-fips-native-go

[github-actions]

TL;DR

The two FIPS unit-test jobs are failing because the PR intentionally turns on FIPS behavior and bumps elastic-agent-libs to v0.46.0, but the existing tests still assert the pre-change TLS behavior. Update the affected expectations rather than retrying the build.

Remediation

• Update internal/pkg/config/output_test.go so the https and mixed-https expected tls.Config values tolerate or assert the new VerifyConnection hook added by tlscommon.LoadTLSConfig(...).ToConfig() under FIPS.
• Update internal/pkg/es/client_test.go:208-216 so TestConnectionTLS expects the RSA-1024 certificate rejection when FIPS mode is enabled by //go:debug fips140=on, not only when fips140.Enforced() is true.
• Validate with GOFIPS140=v1.0.0 go test -tags=grpcnotrace,requirefips ./internal/pkg/config ./internal/pkg/es and then rerun the two failed Buildkite FIPS unit-test steps.

Investigation details

Root Cause

This is a test expectation failure caused by the PR's FIPS/runtime behavior changes, not infrastructure.

The PR adds godebug_fips.go:5-10 with //go:build requirefips and //go:debug fips140=on, and bumps github.com/elastic/elastic-agent-libs from v0.44.0 to v0.46.0 in go.mod. With those changes applied locally, the same FIPS-tagged unit test command fails in two places:

1. internal/pkg/config/output_test.go:211 deeply compares the whole elasticsearch.Config. For the HTTPS cases, the actual TLS config now contains a non-nil VerifyConnection function, but the expected config still has nil.
2. internal/pkg/es/client_test.go:208-216 only expects the RSA-1024 TLS error when fips140.Enforced() is true. Under this PR's FIPS-tagged test binary, fips140=on plus the updated TLS library rejects the invalid RSA-1024 server certificate, so the test reaches the require.NoError(t, err) branch and fails.

Evidence

• Build: https://buildkite.com/elastic/fleet-server/builds/15382
• Jobs/steps: :smartbear-testexecute: Run unit tests with FIPS and :smartbear-testexecute: Run fips140=only unit tests
• Buildkite log excerpt from both failed jobs:

FAIL
Error: running "go test -tags=grpcnotrace,requirefips -v -race -coverprofile=build/coverage-linux.out ./..." failed with exit code 1

• Local reproduction with the PR's source-level FIPS changes applied:

--- FAIL: TestToESConfig (0.00s)
    --- FAIL: TestToESConfig/https (0.00s)
        output_test.go:211: mismatch (-want +got)
-           VerifyConnection:      <nil>,
+           VerifyConnection:      <func>,
    --- FAIL: TestToESConfig/mixed-https (0.00s)
        output_test.go:211: mismatch (-want +got)
-           VerifyConnection:      <nil>,
+           VerifyConnection:      <func>,

--- FAIL: TestConnectionTLS (0.01s)
    client_test.go:216: Received unexpected error:
        tls: certificate uses RSA-1024 public key which is not allowed by FIPS 140-3 (minimum 2048 bits)

No existing flaky-test issue was found for TestToESConfig or TestConnectionTLS in this repository.

Verification

Reproduced in a disposable copy with:

GOFIPS140=v1.0.0 go test -tags=grpcnotrace,requirefips ./...

---

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[ycombinator]

This would also catch fips140=only, which we probably don't want since that could lead to unexpected panics at runtime for FIPS-140 non-compliant uses. Consider a more structured approach like so:

case "DefaultGODEBUG":
    for _, entry := range strings.Split(setting.Value, ",") {
        if key, val, ok := strings.Cut(entry, "="); ok && key == "fips140" && val == "on" {
            foundFIPSDefault = true
            break
        }
    }

[macdewee]

good catch, I will fix it in beats and agent as well

[ycombinator]

Thanks for enforcing GODEBUG=fips140=on at a code/build level. To clarify, this was not a gap for Docker-based deployments before as we were enforcing this at the packaging level:

fleet-server/Dockerfile.fips

Line 59 in 7caed5a

ENV GODEBUG=fips140=on

This form of enforcement was, of course, only limited to Docker images since our goal at the time was FRH environment deployments in ESS.

All that said, the change in this PR is better as it enforces this now regardless of the packaging artifact used for deployment. Maybe we could remove the Dockerfile line now. Or we could leave it as-is for defense in depth. WDYT?

[snyk-io]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues
✅	Licenses	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

[macdewee]

All that said, the change in this PR is better as it enforces this now regardless of the packaging artifact used for deployment. Maybe we could remove the Dockerfile line now. Or we could leave it as-is for defense in depth. WDYT?

I would leave it. It is not a cost and gives as additional defense in depth