fix(transports): share plugin context with tool, prompt and resource hooks on /mcp

[0717376]

📌 Summary

Closes #3879.

Tool calls, prompt fetches and resource reads made over the Streamable HTTP /mcp
transport never receive the plugin contexts produced by HTTP_PRE_REQUEST, so a plugin
cannot share state between its own hooks on the one path an MCP client actually uses. Every REST handler already
forwards them; /mcp is the only remaining gap.

🔁 Reproduction Steps

1. Register an http_pre_request + tool_pre_invoke plugin that stores a value in
   context.state during the first hook and reads it back in the second
   (e.g. a per-user authorization plugin that stashes the caller's token).
2. Call the tool over POST /servers/{server_id}/mcp/.
3. tool_pre_invoke observes an empty context and the stored value is gone.

The same plugin works when the tool is invoked over the REST API.

🐞 Root Cause

HttpAuthMiddleware records the results of the pre-request hooks on the request
state (mcpgateway/middleware/http_auth_middleware.py:224-227):

if context_table:
    request.state.plugin_context_table = context_table
if global_context:
    request.state.plugin_global_context = global_context

ToolService.invoke_tool() accepts both (plugin_context_table is documented as
"Optional plugin context table from previous hooks for cross-hook state sharing"),
and mcpgateway/main.py reads them off request.state at every REST call site.

call_tool() in mcpgateway/transports/streamablehttp_transport.py does not — it
calls invoke_tool() without either argument, so tool_service builds a fresh
GlobalContext and passes local_contexts=None to the hook manager. Anything a
plugin wrote during HTTP_PRE_REQUEST is dropped.

💡 Fix Description

call_tool(), get_prompt() and read_resource() now resolve the two contexts and
forward them to the matching service method — all three already accept the arguments.

The new _get_plugin_contexts_or_none() helper reads them from the ASGI scope
(scope["state"], which is what request.state writes into) rather than from a
ContextVar. This mirrors path 2 of the existing _get_request_context_or_default()
helper and, as its docstring notes, survives the task-group boundaries introduced by
the MCP SDK where ContextVars may be lost. Missing or unexpected values degrade to
(None, None), i.e. exactly today's behaviour.

Scope notes:

• Only the local invoke_tool() path is touched. invoke_tool_direct()
  (direct_proxy) does not take plugin contexts, and the Rust runtime is tracked
  separately in [FEATURE]: Port MCP plugin context sharing to the Rust MCP runtime #4119.

Relationship to the earlier attempt: #3915 fixed the same gap and was closed in favour
of a CPEX gated-extensions approach. Since then main has continued to standardise on
plugin_context_table — it is threaded through ~13 REST call sites in main.py and
middleware/rbac.py now uses it to share context with HTTP_AUTH_CHECK_PERMISSION —
while mcpgateway/transports/ still contains zero references to it. This PR closes
that inconsistency with the mechanism already in use. If maintainers would rather land
this as a CPEX gated extension, I'm glad to redo it that way.

📏 Reviewability

• This PR has one clear purpose
• The linked issue is not labeled triage
• Unrelated bugs or improvements are tracked in separate issues/PRs
• Tests are included with the code they validate
• If AI-assisted, I understand and can explain the generated changes

🧪 Verification

19 tests cover the change in tests/unit/mcpgateway/transports/test_streamablehttp_transport.py:
forwarding for all three handlers, every type-guard fallback of the helper, an
end-to-end run of the issue's repro against a real PluginManager with the shared
CrossHookContextPlugin fixture for each of TOOL_PRE_INVOKE / PROMPT_PRE_FETCH /
RESOURCE_PRE_FETCH, and a negative control per hook that reproduces the pre-fix
failure with local_contexts=None.

Check	Status
pytest tests/unit/mcpgateway/transports/ tests/unit/mcpgateway/plugins/ + prompt/resource service tests	1870 passed, 39 skipped
pytest --doctest-modules mcpgateway/transports/streamablehttp_transport.py	passed
pre-commit run --files <both files>	clean
pylint mcpgateway/transports/streamablehttp_transport.py	no new findings vs main

📐 MCP Compliance (if relevant)

• Matches current MCP spec
• No breaking change to MCP clients

✅ Checklist

• Code formatted (black, isort, ruff)
• No secrets/credentials committed
• DCO sign-off included

[lucarlig]

@araujof can you review this so that we can correctly use CPEX to share context safely. i think you mentioned https://contextforge-org.github.io/cpex/docs/extensions/ previously in this #3915 (comment)

[msureshkumar88]

Reviewed against issue #3879 end-to-end: root cause, fix wiring, and blast radius all check out.

• Root cause confirmed: call_tool() never read plugin_context_table/plugin_global_context off request.state and never forwarded them to invoke_tool(), so TOOL_PRE_INVOKE hooks on /mcp always saw empty context while REST paths worked fine.
• Fix verified correct, not just accepted as-is: traced the full chain — HttpAuthMiddleware writes request.state.plugin_context_table/plugin_global_context → new _get_plugin_contexts_or_none() reads the same keys back off scope["state"] (same technique already proven by _get_request_context_or_default() Path 2 and middleware/rbac.py) → call_tool() forwards both into invoke_tool() → tool_service.py threads plugin_context_table into local_contexts and merges plugin_global_context via _apply_tool_payload_to_global_context(), which enriches rather than overwrites (so server_id is always corrected regardless of what's forwarded).
• Minimal, scoped correctly: only touches call_tool(); invoke_tool_direct() (direct_proxy) and the Rust runtime path are explicitly out of scope, get_prompt/read_resource have the same gap but are reasonably deferred to a follow-up.
• No schema/migration needed — no persisted state involved.
• No breaking change: checked every shipped plugin implementing tool_pre_invoke (header_filter, schema_guard, vault, webhook_notification) against every plugin implementing http_pre_request (custom_auth_example, simple_token_auth) — zero overlap today, so no existing plugin's behavior changes. Fallback stays (None, None), identical to current behavior.
• Security note: this closes a real authz-relevant gap — a custom authorization plugin pairing these two hooks over /mcp was silently getting an empty context; if such a plugin fails open on missing state, that was a live access-control risk specific to this transport. Worth double-checking both the populated and empty-context paths on any custom authz plugin before relying on this in production.
• Tests: ran the new tests plus the full transport suite locally — all green, no regressions. Coverage is solid (forward path, empty-scope path, no-request-context path).
• Confirmed via gh pr diff that the change is scoped to exactly the two files it should touch, no unrelated changes.

Two purely optional, non-blocking suggestions for a future pass: a test covering the isinstance type-guard fallback for malformed scope state, and an e2e test modeled on the issue's own repro for extra confidence. Neither blocks this.

[ja8zyjits]

Hi @0717376,

Please check this review. Some of the reviews may be way out of line, if so, then comment about them here and ignore it. Others, please work on them if you think they are legit.

Summary

Blocking: 1 | Functionally Impacting: 1 | Suggestions: 3 | Minor: 2

PR adds _get_plugin_contexts_or_none() helper and two production lines in call_tool. Fix is correct and well-scoped. New helper reads from scope["state"] (verified: Starlette request.state._state IS scope["state"]). Forwarded args reach invoke_tool → PluginManager.invoke_hook with local_contexts set. Tests are non-synthetic; 9 new plugin-context tests pass; ruff clean.

---

🚨 Blocking

F-1: except Exception branch in _get_plugin_contexts_or_none is untested

# streamablehttp_transport.py:1715-1717
except Exception as exc:  # pylint: disable=broad-except
    logger.debug("Failed to resolve request context for plugin contexts: %s", exc)
    return None, None

No test triggers this path. The parametrized suite sets request_context to a property returning MagicMock — it never raises a non-LookupError. A test should set mcp_app.request_context to a property that raises RuntimeError and assert return is (None, None).

Why blocking: Broad-except is a deliberate defensive path. Without a test, a future refactor removing it (or changing it to re-raise) goes undetected — plugin crash on context access surfaces as unhandled 500 at /mcp transport layer.

---

⚠️ Functionally Impacting

F-2: get_prompt / read_resource context gap has no tracking marker

PR description notes:

get_prompt / read_resource on the same transport have the same gap. I left them out to keep this PR to one concern and am happy to file a follow-up issue.

No test verifies the absence of forwarding as a documented accepted gap (pytest.mark.xfail or comment marker). The CrossHookContextPlugin fixture already implements resource_pre_fetch and prompt_pre_fetch hooks — but no test exercises those paths through the transport.

Impact: A user configuring a plugin for RESOURCE_PRE_FETCH or PROMPT_PRE_FETCH expecting cross-hook state encounters the same empty-context failure as #3879, with no test alerting reviewers of the outstanding gap.

Recommendation: File follow-up issue; add # TODO(follow-up): resource/prompt context gap — see issue #XXXX at read_resource / get_prompt call sites in the transport, plus a pytest.mark.skip or comment block in the test file.

---

💡 Suggestions

S-1: _malformed_scope_cases() called twice at collection time
File: tests/.../test_streamablehttp_transport.py:4116

# Current — two calls at module load
@pytest.mark.parametrize(
    "case_id,scope,expect_global,expect_table",
    _malformed_scope_cases(),
    ids=[c[0] for c in _malformed_scope_cases()]  # second call
)

# Fix
_CASES = _malformed_scope_cases()
@pytest.mark.parametrize("case_id,scope,expect_global,expect_table", _CASES, ids=[c[0] for c in _CASES])

Minor overhead only; not a correctness issue.

S-2: _plugin_context_scope() used only by one test; _malformed_scope_cases() duplicates construction inline

Extracting both into the same fixture factory or reusing _plugin_context_scope() in _malformed_scope_cases reduces duplication. No functional impact.

S-3: Inconsistent client_host/client_port in E2E tests

test_cross_hook_plugin_fails_without_forwarded_contexts omits client_host/client_port; test_call_tool_cross_hook_sharing_end_to_end_with_real_plugin passes them. Both default to None — cosmetic only.

---

📝 Minor Notes

M-1: Triple request_context access in call_tool
call_tool already accesses mcp_app.request_context at line 1784; _get_request_context_or_default accesses it again internally; _get_plugin_contexts_or_none() adds a third. Consistent with existing pattern, no synchronization issue — worth noting for future consolidation.

M-2: Deprecation warnings in two new tests
File: tests/.../test_streamablehttp_transport.py:4298

DeprecationWarning: HttpPreRequestPayload.headers is deprecated; use extensions.http.headers instead.
DeprecationWarning: ToolPreInvokePayload.headers is deprecated; use extensions.http.headers instead.

Pre-existing deprecations in cpex, not introduced by this PR. Tests pass headers= via public API which routes through deprecated fields.

---

📋 Documentation Warnings

• docs/docs/using/plugins/index.md:943 — broad cross-hook state claim partially invalidated: "A single plugin can share state across all hooks in a request by using the PluginContext state dictionary." Only call_tool on /mcp now forwards context; get_prompt/read_resource still pass local_contexts=None. Fix: Add note that context forwarding on Streamable HTTP is currently TOOL_PRE_INVOKE only; RESOURCE_PRE_FETCH/PROMPT_PRE_FETCH tracked in follow-up issue.

• mcpgateway/transports/streamablehttp_transport.py:1745-1778 — call_tool docstring silent on plugin context forwarding. Fix: Add one sentence: "Plugin contexts recorded by HTTP_PRE_REQUEST are read from the ASGI scope via _get_plugin_contexts_or_none and forwarded to invoke_tool to enable cross-hook state sharing."

---

✅ Coverage Matrix

Branch / Line	Covered
LookupError from request_context	✅ test_get_plugin_contexts_returns_none_without_request_context
except Exception (non-LookupError)	❌ No test
scope not a dict	✅ scope_not_a_dict param
state missing from scope	✅ state_missing param
state not a dict	✅ state_not_a_dict param
global_context wrong type	✅ global_context_wrong_type param
context_table wrong type	✅ context_table_wrong_type param
Both slots None	✅ both_none param
Both slots populated and forwarded	✅ test_call_tool_forwards_plugin_contexts_from_scope
Plugin manager receives forwarded contexts	✅ E2E test
Pre-fix failure reproduced	✅ test_cross_hook_plugin_fails_without_forwarded_contexts
get_prompt context gap (known)	❌ No test; no tracking marker
read_resource context gap (known)	❌ No test; no tracking marker

---

[0717376]

@ja8zyjits thanks — this was a useful pass. Everything is in a82ff932a, rebased onto current main.

F-1 — added test_get_plugin_contexts_swallows_unexpected_request_context_errors: request_context raises RuntimeError, the helper returns (None, None) and logs at debug level.

F-2 — you were right that a tracking marker is the minimum, but it turned out to cost about as much as the fix: PromptService.get_prompt() and ResourceService.read_resource() already accept plugin_global_context / plugin_context_table, and main.py forwards them at the REST call sites (:7056, :6516) — so /mcp was the only place missing the hand-off for all three hooks, not just tools. get_prompt() and read_resource() now reuse _get_plugin_contexts_or_none(). Forwarding tests for both mirror the call_tool one, and the e2e suite now drives each handler through the real PluginManager with CrossHookContextPlugin (PROMPT_PRE_FETCH and RESOURCE_PRE_FETCH were already implemented in that fixture, as you noted). The negative control is parametrised over the three hooks. PR title/description updated accordingly; the follow-up-issue note is gone since there's nothing left to follow up.

S-1 — fixed, _MALFORMED_SCOPE_CASES is evaluated once.

S-2 — left as is, deliberately: _malformed_scope_cases() needs objects that are wrong on purpose (a non-dict scope, a string where a GlobalContext should be), while _plugin_context_scope() builds the valid happy-path request. Sharing a builder would hide which case is which. Happy to revisit if you feel strongly.

S-3 — both e2e paths now go through one _cross_hook_plugin_session() helper, so client_host/client_port are set identically.

M-1 — agreed, noted for a future consolidation; not touching it here.

M-2 — pre-existing in cpex; tool_service.py:5671 constructs ToolPreInvokePayload(headers=...) the same way, so the tests match production.

Docs — one sentence added to each of the three handler docstrings. The plugins/index.md caveat is no longer needed: the "share state across all hooks" claim is now true on Streamable HTTP too.

Locally: transports + plugins + prompt/resource service suites 1870 passed; doctests, pre-commit and pylint clean.