Skip to content

Add Redis-backed mixed-era vMCP e2e coverage - #6056

Merged
JAORMX merged 1 commit into
mainfrom
redis-vmcp-dual-era-e2e
Jul 28, 2026
Merged

Add Redis-backed mixed-era vMCP e2e coverage#6056
JAORMX merged 1 commit into
mainfrom
redis-vmcp-dual-era-e2e

Conversation

@jhrozek

@jhrozek jhrozek commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add live e2e tests for dual-era stateless proxy behavior #5981 removed two specs from the single-server dual-era k8s test because they cannot work on one MCPServer: a single go-sdk backend cannot be both stateless (serving Modern per-request clients) and session-issuing (serving Legacy clients) at the same time. The commit asked whoever landed the vMCP cross-generation bridge to re-add them as vMCP coverage.
  • Bridge era-mismatched client×backend cells in vMCP #6006 landed that bridge with live four-cell coverage at two tiers, but neither tier configures Redis. So the bridge is proven and its session-store dimension is not: nothing exercised a Legacy client's session metadata living in Redis while a Modern backend shared the group, and nothing exercised what a Redis outage does to a mixed-era deployment.
  • Adds an operator-tier spec over a mixed Legacy+Modern yardstick backend set behind one vMCP with sessionStorage: redis. The property it pins is the asymmetry a single-server test could not express: a Redis outage 503s the Legacy session path while the Modern stateless path keeps serving, because vMCP classifies client era in middleware ahead of the SDK session layer and dispatchModern never reads the session store.
  • Three specs: concurrent Legacy-session + Modern-stateless traffic with no cross-delivery; Redis holding a backend-session key for the Legacy backend only; and outage → 503 → recovery, with the wiped pre-outage session correctly reported terminated.
  • Also ports the scaleRedis helper deleted in 5fa67123b, name-parameterized this time so scaling one spec's Redis cannot disturb another's under --procs=8 (the original hardcoded a shared redis Deployment, which is why it needed its own namespace).

Closes #6008

Type of change

  • Other (describe): test-only — adds E2E coverage, no production code touched

Test plan

  • E2E tests (task test-e2e)
  • Linting (task lint-fix)
  • Manual testing (describe below)

Run against a Kind cluster created with kind-setup-e2e (the NodePort mappings are required):

  • Focused run green repeatedly: 3 Passed | 0 Failed, including after rebasing onto current main and rebuilding/redeploying all three images.
  • All three specs also passed inside a full --procs=8 suite run, so they do not collide with the other 146 specs.
  • task lint-fix: 0 issues repo-wide.

Negative checks — the part that matters. Both were run to confirm the specs are not passing vacuously:

Flip Result
STATELESS=false on the Modern backend Era gate fails precisely: modern backend …: want revision "2026-07-28", got "2025-11-25", both backends Legacy
TOOLHIVE_VMCP_MODERN_STATELESS=false The Modern tools/call leg fails

Without those two gates a forgotten env var would silently collapse the "mixed" set and every spec would still pass while testing one era twice — a failure mode that has already happened once in this area.

API Compatibility

  • This PR does not break the v1beta1 API, OR the api-break-allowed label is applied and the migration guidance is described above.

Changes

File Change
test/e2e/thv-operator/virtualmcp/virtualmcp_dual_era_redis_test.go New. Mixed-era fixture (Redis session storage, era gate, Modern dispatcher on) plus three specs: basic round-trip, concurrency + Redis metadata partition, outage/recovery.
test/e2e/thv-operator/virtualmcp/mcpserver_scaling_test.go Adds scaleRedis(name, replicas) next to the other Redis helpers.

Does this introduce a user-facing change?

No.

Special notes for reviewers

Deviation from the issue text: this runs at one replica, not multiple. WithSessionIdManager is wired unconditionally (pkg/vmcp/server/server.go:570), so one replica exercises the Redis store just as fully. Two replicas would add only the rehydration path, and that path is what makes the 503 nondeterministic: on a non-owning pod serveRehydrated returns 503 only if that pod happens to hold a cached rehydration, otherwise 404 Invalid session ID. Cross-pod Redis session sharing is already covered by virtualmcp_redis_session_test.go Context 2, so replicas=2 here would buy coverage that exists elsewhere and pay for it with port-forward pinning to work around a coin flip we'd have introduced ourselves. Happy to switch if reviewers disagree.

Two constraints cost a debug cycle each and are documented in the file — worth knowing before editing it:

  1. yardstick's echo tool schema-validates input against ^[a-zA-Z0-9]+$, and reports a violation as a successful JSON-RPC result with isError: true. So resp.Error == nil is not proof a call worked, and any nonce must be run-together alphanumeric (round0req3, never round-0-req-3).
  2. A go-sdk/mcpcompat client cannot open a Legacy session against a Modern-dispatch-enabled vMCP at all. go-sdk v1.7's Connect is Modern-first, so server/discover routes to dispatchModern (pkg/vmcp/server/classification.go:97-110), the client negotiates Modern, gets no session, and then 404s on the subscriptions/listen stream mcpcompat's Initialize unconditionally opens — surfaced as ErrSessionMissing after Initialize apparently succeeded. That rules out CreateInitializedMCPClient / WaitForExpectedTools / ToolsContainAll here, so all traffic goes through e2e.RawMCPClient, which pins the era per request. Shared helpers were left untouched; they remain correct for a dispatcher-off vMCP.

The Redis negative assertion (Not(HaveKey("vmcp.backend.session." + modernBackend))) is deterministic in both revision-cache states, and the comment explains why: the warm path skips the connect via isKnownModern (pkg/vmcp/session/factory.go:281-287), and even off that path populateBackendMetadata (factory.go:414-425) only writes the key when the backend reports a non-empty session ID, which a stateless backend never does.

Unrelated observation while verifying: the operator E2E suite is flaky under --procs=8 on a single-node Kind cluster independent of this change — a baseline run with this file removed failed more specs (8 vs 6) and on a different root spec. Not addressed here.

Generated with Claude Code

@github-actions github-actions Bot added the size/L Large PR: 600-999 lines changed label Jul 27, 2026
@codecov

codecov Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.27%. Comparing base (793eb55) to head (018da30).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6056      +/-   ##
==========================================
- Coverage   72.30%   72.27%   -0.04%     
==========================================
  Files         722      724       +2     
  Lines       75138    75330     +192     
==========================================
+ Hits        54327    54443     +116     
- Misses      16934    17009      +75     
- Partials     3877     3878       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

PR #5981 removed two specs from the single-server dual-era k8s test
because they cannot work on one MCPServer: a single go-sdk backend
cannot be both stateless (for Modern per-request clients) and
session-issuing (for Legacy clients) at once. PR #6006 then landed the
vMCP cross-generation bridge, but neither of its tiers configures
Redis, so the session-store dimension of the bridge stayed unproven:
nothing exercised a Legacy client's session metadata living in Redis
while a Modern backend shared the group, and nothing exercised what a
Redis outage does to a mixed-era deployment.

Add an operator-tier spec over a mixed Legacy+Modern yardstick backend
set behind one vMCP with Redis session storage. It asserts the
asymmetry a single-server test could not express: a Redis outage 503s
the Legacy session path while the Modern stateless path keeps serving,
because vMCP classifies client era in middleware ahead of the SDK
session layer and dispatchModern never reads the session store.

Coverage: concurrent Legacy-session and Modern-stateless traffic with
no cross-delivery; Redis holding a backend-session key for the Legacy
backend only; and outage to 503 to recovery, with the wiped pre-outage
session correctly reported terminated.

Two constraints are documented in the file because both cost a debug
cycle. yardstick's echo tool schema-validates input against
^[a-zA-Z0-9]+$ and reports a violation as a successful result with
isError set, so resp.Error being nil is not proof a call worked. And a
go-sdk/mcpcompat client cannot open a Legacy session against a
Modern-dispatch-enabled vMCP at all: Connect is Modern-first, so
server/discover routes to dispatchModern, the client negotiates Modern,
gets no session, and then 404s on subscriptions/listen. This spec
therefore drives all traffic through the raw client, which pins the era
per request.

Runs at one replica rather than the multiple replicas the issue
suggests. WithSessionIdManager is wired unconditionally, so one replica
exercises the Redis store just as fully; two would add only the
rehydration path, whose transient-error branch returns 404 instead of
503 unless the pod holds a cached rehydration, making the outage
assertion nondeterministic. Cross-pod Redis session sharing is already
covered by virtualmcp_redis_session_test.go.

Closes #6008

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@jhrozek
jhrozek force-pushed the redis-vmcp-dual-era-e2e branch from acff2cd to 018da30 Compare July 28, 2026 06:38
@github-actions github-actions Bot added size/L Large PR: 600-999 lines changed and removed size/L Large PR: 600-999 lines changed labels Jul 28, 2026
@JAORMX
JAORMX merged commit b5a4851 into main Jul 28, 2026
47 checks passed
@JAORMX
JAORMX deleted the redis-vmcp-dual-era-e2e branch July 28, 2026 07:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/L Large PR: 600-999 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Redis-backed mixed-era vMCP e2e coverage (session store + outage)

3 participants