Skip to content

Telemetry: record which surface asked (cli | mcp), and the client where the protocol offers it #8

Description

@cdevarenne

The gap

Events are emitted from service.py so the CLI and the MCP server produce identical documents — deliberate, per docs/specs/observability.md:65. That was right when the only consumer was a human at a terminal.

Now a foreign agent is a real consumer, and the log cannot tell the two apart. A concrete case: five events in var/telemetry.ndjson between the two Codex sessions in docs/CodexQandA.md were CLI verification runs, not agent traffic. The only way to attribute them was remembering what had been typed. Kibana cannot answer "what did the agent ask?" — the panels mix a foreign runtime's questions with my own testing.

Proposed

Add a surface field to the event: cli | mcp. Schema v3.

Where it comes from is the design question. service.py is shared on purpose and should not learn about its callers, so the two candidates are:

  1. The entry points declare it — cli.main() and mcp_server.main() set a module-level or context value that telemetry.record reads.
  2. lookup_field/ask take an optional surface argument, defaulted, and the two entry points pass it.

Option 1 keeps the signatures clean; option 2 keeps the state explicit. Pick one when implementing — the requirement is only that the shared answer-builder stays the single emit site.

Client identity — separate, and unverified

The MCP initialize handshake carries clientInfo (name and version), so "Codex CLI 0.149.0" is knowable in principle. Whether it is reachable from inside a @server.tool() handler in this SDK is not establishedclient_info appears in the SDK internals but no spike has been done. Treat it as a follow-on: land surface first, add client only if the SDK exposes it without contortions.

Blast radius

  • telemetry.pySCHEMA_VERSION 2 → 3, event construction. The mixed-version warning at telemetry.py:207 already degrades gracefully, so old events stay readable.
  • telemetry_index.py — mapping gets the new keyword field.
  • docs/specs/observability.md — the event schema is specified there; update before implementing.
  • docs/kibana/ — a "by surface" breakdown is the panel that makes this worth doing.
  • Tests for both the log and the projection.

Done when

A Kibana panel splits agent traffic from CLI traffic, and the eight Codex events in docs/CodexQandA.md could have been identified from the log alone.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Planned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions