Skip to content

Commit b8f8253

Browse files
feat(mcp): capture model identifiers (#927)
* feat(mcp): capture model identifiers Capture model identifiers from recognized client metadata, with an SDK-owned self-report fallback for other clients. Preserve source provenance and fail closed when the application owns the field.\n\nVerify both MCP Python SDK 1.x and 2.x, including a 2026-07-28 wire-level call. * fix(mcp): preserve model capture opt-in and tool ownership Gate custom-dispatcher model resolution behind capture_model and copy object tools before model injection so repeated listings preserve ownership. Publish the completed ownership map without an intermediate empty state. Keep additionalProperties constraints in all analytics schema injectors. Document adapter-specific requiredness rather than requiring fields that standalone FastMCP strips before input validation. Share the eligibility predicate and update prepare helper documentation and the changeset. Verification: - MCP v1 suite: 267 passed. - MCP v2 suite: 245 passed, 13 expected skips. - Ruff check and format check passed repository-wide. - Repository mypy/baseline check passed (229 source files). - Public API snapshot and git diff --check passed. - Regression cases failed before the fix for disabled capture, object ownership and strict validation through a low-level tool-cache rebuild. Full non-MCP test suite not rerun locally; CI covers the broader matrix. * chore(mcp): address release note and test dependency feedback Clarify that preserving strict schemas during context and conversation-ID injection also affects existing users with model capture disabled. Declare jsonschema>=4.0 in the test extra and refresh only its direct dependency entries in uv.lock. Verification: python -m pytest posthog/test/mcp --timeout=30 -q passed (267 tests, MCP SDK 1.29.0); uv lock --check and git diff --check passed. Configured pre-commit checks skipped these non-Python files. CodeScene reported no applicable changes. MCP v2 was not rerun locally for this documentation and dependency-only update. * fix(mcp): preserve positional constructor compatibility Append model capture fields after the existing public dataclass fields. Existing positional MCPAnalyticsOptions calls keep their identity and callback bindings, and PreparedToolCall retains the fourth positional missing-capability flag. Add four parameterized regression cases and regenerate the public API snapshot. The cases failed before the field reorder and pass afterward. Verification: 271 MCP tests passed; public_api_snapshot and public_api_check passed; configured Ruff pre-commit checks passed; mypy baseline check found no issues in 230 source files; strict import and git diff checks passed. CodeScene checked both Python files with no issues. MCP v2 validation remains covered by CI.
1 parent e823b3e commit b8f8253

31 files changed

Lines changed: 954 additions & 56 deletions
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
pypi/posthog: minor
3+
---
4+
5+
Capture MCP model identifiers from client metadata or an SDK-owned self-report field.
6+
Model capture remains opt-in and preserves application-owned fields across repeated tool listings.
7+
8+
MCP context and conversation-ID injection now preserve `additionalProperties: false` in tool schemas, including when model capture is disabled. Servers that validate these schemas now reject undeclared arguments that earlier SDK versions allowed. Declared analytics fields remain valid.

‎posthog/mcp/README.md‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,75 @@ Request headers use the same identity and package version, so SDK Health can com
2121
Because `$lib` is a client-level identity, `instrument()` relabels every event sent by the client passed to it.
2222
Use a client dedicated to MCP analytics if the application also captures unrelated events.
2323

24+
## Capture the calling model
25+
26+
Model capture is off by default. Enable it for an instrumented MCP Python SDK 1.x or
27+
2.x server:
28+
29+
```python
30+
from posthog.mcp import MCPAnalyticsOptions, instrument
31+
32+
analytics = instrument(
33+
server,
34+
posthog,
35+
MCPAnalyticsOptions(capture_model=True),
36+
)
37+
```
38+
39+
The SDK records the best model identifier visible to the server as
40+
`$mcp_llm_model`. Recognized client metadata wins and sets
41+
`$mcp_llm_model_source` to `client_metadata`. The SDK also adds an `llm_model`
42+
string to each compatible tool schema as a fallback, recorded with source
43+
`self_reported`. It is required for custom dispatchers and the official high-level
44+
MCP SDK adapters. Raw low-level servers and standalone `fastmcp.FastMCP` advertise
45+
it as optional; the standalone adapter strips it before input validation, so
46+
requiring it would reject calls. Existing schema strictness is preserved when
47+
analytics fields are added.
48+
49+
The recognized metadata path is Codex's `x-codex-turn-metadata.model` field in
50+
request `_meta`. Other clients, including Claude Code, use the self-report path
51+
until they expose a stable model field. Missing, blank, and `unknown` values are
52+
not recorded.
53+
54+
MCP does not standardize or attest model identity. Both sources are unverified.
55+
Use them to compare tool behavior across models, not for billing or access
56+
control.
57+
58+
Model self-reporting only runs when PostHog can prove it owns the injected field.
59+
If a tool already declares `llm_model`, or uses a root `$ref`, `oneOf`, `allOf`, or
60+
`anyOf` schema, PostHog leaves the schema and argument untouched. Client metadata
61+
can still be captured in those cases.
62+
63+
For a custom dispatcher, use the same option on `PostHogMCP` and pass request
64+
metadata through explicitly:
65+
66+
```python
67+
from posthog.mcp import PostHogMCP
68+
69+
posthog = PostHogMCP("phc_...", capture_model=True)
70+
tools = posthog.prepare_tool_list(server_tools)
71+
original_tool = next(tool for tool in server_tools if tool["name"] == tool_name)
72+
call = posthog.prepare_tool_call(
73+
tool_name,
74+
raw_args,
75+
request_meta=request.get("params", {}).get("_meta"),
76+
original_tool=original_tool,
77+
)
78+
result = dispatch(tool_name, call.args)
79+
posthog.capture_tool_call(
80+
tool_name,
81+
llm_model=call.llm_model,
82+
llm_model_source=call.llm_model_source,
83+
)
84+
```
85+
86+
Passing `original_tool` keeps ownership accurate when `tools/list` and
87+
`tools/call` reach different server replicas. A persistent single-process
88+
dispatcher can omit it after calling `prepare_tool_list()`.
89+
Model injection copies tool objects instead of changing their original schemas.
90+
Always advertise the returned list and pass the original application tool to
91+
`prepare_tool_call()`. Repeatedly preparing the original list preserves ownership.
92+
2493
## Stateless / multi-pod servers
2594

2695
A stateless MCP server issues no session id, so `$session_id` fragments across pods

‎posthog/mcp/__init__.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,8 @@
7373
from .types import (
7474
CaptureEventData,
7575
MCPAnalyticsContextOptions,
76+
MCPAnalyticsModelOptions,
77+
MCPAnalyticsModelSource,
7678
MCPAnalyticsOptions,
7779
PreparedToolCall,
7880
UserIdentity,
@@ -85,6 +87,8 @@
8587
"PostHogMCP",
8688
"MCPAnalyticsOptions",
8789
"MCPAnalyticsContextOptions",
90+
"MCPAnalyticsModelOptions",
91+
"MCPAnalyticsModelSource",
8892
"UserIdentity",
8993
"CaptureEventData",
9094
"PreparedToolCall",

‎posthog/mcp/_capture.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,8 @@ def capture_event(
6363
"response": event_input.get("response"),
6464
"user_intent": event_input.get("user_intent"),
6565
"user_intent_source": event_input.get("user_intent_source"),
66+
"llm_model": event_input.get("llm_model"),
67+
"llm_model_source": event_input.get("llm_model_source"),
6668
"is_error": event_input.get("is_error"),
6769
"error": event_input.get("error"),
6870
"error_type": event_input.get("error_type"),

‎posthog/mcp/_context_parameters.py‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -79,10 +79,7 @@ def add_context_parameter_to_schema(
7979
if not isinstance(schema.get("properties"), dict):
8080
schema["properties"] = {}
8181

82-
# additionalProperties: false would reject the injected context — remove it
83-
# (the SDK adds this when converting Pydantic models to JSON Schema).
84-
if schema.get("additionalProperties") is False:
85-
schema.pop("additionalProperties", None)
82+
# The declared context property is allowed even under additionalProperties: false.
8683

8784
schema["properties"]["context"] = {
8885
"type": "string",

‎posthog/mcp/_conversation_id.py‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -57,8 +57,6 @@ def add_conversation_id_to_schema(
5757
schema = copy.deepcopy(schema)
5858
if not isinstance(schema.get("properties"), dict):
5959
schema["properties"] = {}
60-
if schema.get("additionalProperties") is False:
61-
schema.pop("additionalProperties", None)
6260
schema["properties"][CONVERSATION_ID_PARAM_NAME] = {
6361
"type": "string",
6462
"description": DEFAULT_CONVERSATION_ID_DESCRIPTION,

‎posthog/mcp/_instrument_fastmcp.py‎

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,11 @@
3939
start_tools_list_lifecycle,
4040
)
4141
from ._internal import MCPAnalyticsData
42+
from ._model_parameters import (
43+
can_inject_model_parameter,
44+
is_capture_model_enabled,
45+
request_meta_from_context,
46+
)
4247
from ._output_instructions import mirror_instructions_into_structured_content
4348
from .logger import log
4449
from .tools import get_more_tools_result_text, resolve_missing_capability_tool_name
@@ -90,6 +95,8 @@ async def wrapped(
9095
data,
9196
name=name,
9297
arguments=arguments,
98+
request_meta=request_meta_from_context(_tool_call_request_context(context)),
99+
allow_self_reported_model=_analytics_owns_model(server, data, name),
93100
mcp_session_id=mcp_session_id,
94101
token=token,
95102
client_name=client_name,
@@ -119,6 +126,8 @@ async def wrapped(
119126
server, name, "conversation_id"
120127
):
121128
strip_keys.add("conversation_id")
129+
if _analytics_owns_model(server, data, name):
130+
strip_keys.add("llm_model")
122131
if strip_keys:
123132
call_arguments = {
124133
k: v for k, v in arguments.items() if k not in strip_keys
@@ -251,7 +260,7 @@ async def list_handler(req: Any) -> Any:
251260
if data.options.report_missing:
252261
missing_name = resolve_missing_capability_tool_name(data.options)
253262
if not any(t.name == missing_name for t in tools):
254-
append_get_more_tools(result, missing_name)
263+
append_get_more_tools(result, missing_name, data)
255264
names.append(missing_name)
256265

257266
await lifecycle.record_result(
@@ -310,6 +319,16 @@ def _tool_owns_context(server: Any, name: str) -> bool:
310319
return _tool_owns_param(server, name, "context")
311320

312321

322+
def _analytics_owns_model(server: Any, data: MCPAnalyticsData, name: str) -> bool:
323+
if not is_capture_model_enabled(data.options.capture_model):
324+
return False
325+
try:
326+
tool = server._tool_manager.get_tool(name)
327+
return can_inject_model_parameter(getattr(tool, "parameters", None))
328+
except Exception: # noqa: BLE001 - model analytics must never break dispatch
329+
return data.tool_model_parameter_injected.get(name, False)
330+
331+
313332
def _tool_call_request_context(context: Any) -> Any:
314333
"""The request context behind a FastMCP ``Context``, or ``None``.
315334

‎posthog/mcp/_instrument_lowlevel.py‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@
3434
start_tools_list_lifecycle,
3535
)
3636
from ._internal import MCPAnalyticsData
37+
from ._model_parameters import request_meta_from_context
3738
from ._output_instructions import mirror_instructions_into_structured_content
3839
from .logger import log
3940
from .tools import get_more_tools_result_text, resolve_missing_capability_tool_name
@@ -97,6 +98,10 @@ async def handler(req: Any) -> Any:
9798
data,
9899
name=name,
99100
arguments=arguments,
101+
request_meta=request_meta_from_context(_request_context(server)),
102+
allow_self_reported_model=data.tool_model_parameter_injected.get(
103+
name, False
104+
),
100105
mcp_session_id=mcp_session_id,
101106
token=token,
102107
client_name=client_name,
@@ -127,7 +132,10 @@ async def handler(req: Any) -> Any:
127132
# tools/list and across stateless per-request server instances.
128133
if strip_injected and req.params.arguments:
129134
owned = await _tool_owned_injected_keys(high_level, name)
130-
for key in ("context", "conversation_id"):
135+
injected_keys = ["context", "conversation_id"]
136+
if data.tool_model_parameter_injected.get(name, False):
137+
injected_keys.append("llm_model")
138+
for key in injected_keys:
131139
if key not in owned:
132140
req.params.arguments.pop(key, None)
133141

@@ -289,7 +297,7 @@ async def handler(req: Any) -> Any:
289297
if data.options.report_missing:
290298
missing_name = resolve_missing_capability_tool_name(data.options)
291299
if not any(t.name == missing_name for t in tools):
292-
append_get_more_tools(result, missing_name)
300+
append_get_more_tools(result, missing_name, data)
293301
names.append(missing_name)
294302

295303
await lifecycle.record_result(

‎posthog/mcp/_instrument_v2.py‎

Lines changed: 34 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,11 @@
4747
start_tools_list_lifecycle,
4848
)
4949
from ._internal import MCPAnalyticsData
50+
from ._model_parameters import (
51+
can_inject_model_parameter,
52+
is_capture_model_enabled,
53+
request_meta_from_context,
54+
)
5055
from ._output_instructions import mirror_instructions_into_structured_content
5156
from .logger import log
5257
from .request_headers import get_request_headers
@@ -222,6 +227,18 @@ def _tool_owns_param_v2(high_level: Any, name: str, param: str) -> bool:
222227
return param in _tool_own_properties_v2(high_level, name)
223228

224229

230+
def _analytics_owns_model_v2(
231+
high_level: Any, data: MCPAnalyticsData, name: str
232+
) -> bool:
233+
if not is_capture_model_enabled(data.options.capture_model):
234+
return False
235+
try:
236+
tool = high_level._tool_manager.get_tool(name)
237+
return can_inject_model_parameter(getattr(tool, "parameters", None))
238+
except Exception: # noqa: BLE001 - model analytics must never break dispatch
239+
return data.tool_model_parameter_injected.get(name, False)
240+
241+
225242
# --- high-level: ToolManager.call_tool seam --------------------------------------
226243

227244

@@ -249,6 +266,8 @@ async def wrapped(
249266
data,
250267
name=name,
251268
arguments=arguments,
269+
request_meta=request_meta_from_context(ctx),
270+
allow_self_reported_model=_analytics_owns_model_v2(server, data, name),
252271
mcp_session_id=mcp_session_id,
253272
token=token,
254273
client_name=client_name,
@@ -281,6 +300,8 @@ async def wrapped(
281300
and "conversation_id" not in own_properties
282301
):
283302
strip_keys.add("conversation_id")
303+
if _analytics_owns_model_v2(server, data, name):
304+
strip_keys.add("llm_model")
284305
if strip_keys:
285306
call_arguments = {
286307
k: v for k, v in arguments.items() if k not in strip_keys
@@ -389,6 +410,10 @@ async def handler(ctx: Any, params: Any) -> Any:
389410
data,
390411
name=name,
391412
arguments=arguments,
413+
request_meta=request_meta_from_context(ctx),
414+
allow_self_reported_model=data.tool_model_parameter_injected.get(
415+
name, False
416+
),
392417
mcp_session_id=mcp_session_id,
393418
token=token,
394419
client_name=client_name,
@@ -504,7 +529,7 @@ async def handler(ctx: Any, params: Any) -> Any:
504529
if data.options.report_missing:
505530
missing_name = resolve_missing_capability_tool_name(data.options)
506531
if not any(t.name == missing_name for t in tools):
507-
_append_get_more_tools_v2(result, missing_name)
532+
_append_get_more_tools_v2(result, missing_name, data)
508533
names.append(missing_name)
509534

510535
await lifecycle.record_result(
@@ -520,14 +545,21 @@ async def handler(ctx: Any, params: Any) -> Any:
520545
_replace_handler(server, _LIST_METHOD, handler, entry.params_type)
521546

522547

523-
def _append_get_more_tools_v2(result: Any, name: str) -> None:
548+
def _append_get_more_tools_v2(result: Any, name: str, data: MCPAnalyticsData) -> None:
524549
descriptor = build_report_missing_descriptor(name)
525550
tool = mcp_types.Tool(
526551
name=descriptor["name"],
527552
description=descriptor["description"],
528553
input_schema=descriptor["inputSchema"],
529554
annotations=descriptor["annotations"],
530555
)
556+
mutate_tool_schema(
557+
data,
558+
tool,
559+
schema_attribute="input_schema",
560+
owns_context=True,
561+
context_required=True,
562+
)
531563
tools_list = getattr(result, "tools", None)
532564
if isinstance(tools_list, list):
533565
tools_list.append(tool)

0 commit comments

Comments
 (0)