CMF content parts cannot carry MCP annotations or _meta

[madhu-mohan-jaishankar]

CMF content parts drop two things every MCP ContentBlock carries, which has one functional consequence and one structural one.

1. Plugins cannot see annotations

MCP attaches Annotations { audience, priority, last_modified } to every content block. audience states who a message is for — e.g. ["assistant"] means model-only, never shown to the user.

CMF's ContentPart::Text { text }, ImageSource, and AudioSource carry no equivalent, so a policy plugin cannot express a rule like "redact anything the user will see." This is a capability gap independent of any particular consumer.

2. The CMF↔MCP round trip is lossy, so consumers must compare before rebuilding

Fields missing on the CMF side:

MCP	CMF	missing
TextContent { text, meta, annotations }	Text { text }	meta, annotations
ImageContent { data, mime_type, meta, annotations }	ImageSource { source_type, data, media_type }	meta, annotations
AudioContent { data, mime_type, meta, annotations }	AudioSource { …, duration_ms }	meta, annotations
EmbeddedResource { resource, meta, annotations }	Resource { … }	meta; blob is String (base64) vs Vec<u8>
Resource link — uri, name, title, description, mime_type, size, icons, meta, annotations	ResourceReference { resource_request_id, uri, name, resource_type, range_start, range_end, selector }	title, description, mime_type, size, icons, meta, annotations

Because reconstructing MCP content from CMF silently drops these, a consumer that writes plugin edits back cannot simply rebuild — it must compare each item against a re-projection of the original and rebuild only what changed, so untouched content is returned intact. Content a plugin did edit still loses these fields, with no way to recover them.

PipelineResult reports modification at payload granularity (modified_payload present or absent), so per-item change detection has to be derived by comparison. CMF types also don't derive PartialEq, so that comparison goes through serde_json::to_value.

Suggested

1. Add annotations (and ideally a _meta passthrough) to CMF content parts — addresses the capability gap on its own.
2. Fill the remaining ResourceReference fields and settle a blob representation, making the round trip lossless so consumers can rebuild unconditionally.

[araujof]

@madhu-mohan-jaishankar thanks for this issue.

I see some missing fields listed that already seem to existing in the CMF types. For example, see Resource.

Question: would it be reasonable to future proof this and consider introducing a _meta map in all content types so that consumers/producers can extend them with things like annotations, meta, etc.? To avoid tracking a specific protocol (e.g., MCP, an inference protocol, etc.).