Skip to content

feat(messaging): send the workflow version a template send by key was checked against - #127188

Draft
Silthus wants to merge 175 commits into
masterfrom
email-v2/12-send-graded-template-version
Draft

Silthus wants to merge 175 commits into
masterfrom
email-v2/12-send-graded-template-version

Conversation

@Silthus

@Silthus Silthus commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Moved from Silthus#585.

Part of the code-first rebuild of the email send stack; new in the rebuild, with no closed PR to replace: a template send delivers the content Django graded (decision D3 on map Silthus#461).

Problem

  • A template send on send/email/ right after an upload can deliver the previous content, while Django answered that the caller's template.hash is current.
  • Django grades the hash, but the email goes out from the workflow copy cdp-api and the worker cached, which can lag after a lost reload.
  • This is the Django half: it tells cdp-api which workflow version holds the graded content. The cdp-api and worker half is feat(cdp): send a template from the workflow version Django graded #127130.

Impact

Pays into Grow Workflows through billed workflows_emails actions per paying org: a developer who uploads a template and sends it at once gets the new content. In scope: the send/email/ route.

Merge Danger

Note

🟢 Merge risk: low · two-way door: a revert stops sending the version, and delivery works as today

Changes

Kind Files Lines
Production 5 +56 −8
Tests 2 +88 −7
  • A template send posts the version of its workflow that holds the graded content, as send.hogFlowVersion. Django reads it after grading the hash, from the writer database, so it is never older than the graded content.
  • A workflow deleted between the grade and the version read answers as the cdp-api 404 does today (Unavailable to the caller), without a stack trace in the log.
  • A send that the email pipeline can't deliver yet gets a 503 that says so. When cdp-api still holds an older workflow version than the graded one, the caller gets master's 503 delivery_unavailable shape with its own detail:
{
  "type": "server_error",
  "code": "delivery_unavailable",
  "detail": "This template's newest version hasn't reached the email pipeline yet, so nothing was sent. Retry in a few seconds, with the same Idempotency-Key if you sent one.",
  "attr": null
}
  • Nothing was sent in that case, so a retry is safe with or without a key. The internal reason, workflow_version_behind, never reaches the caller.
  • Every other 503 keeps master's body, on both routes.
  • Every send by template.key posts the version, with or without from, reply_to or capture_engagement, so it goes to the accepted-send route. Its default captureEngagement: true changes nothing: the project's setting still decides.
  • A send by template.id keeps master's route, which grades no hash and posts no version. One that sets capture_engagement (feat(messaging): send a stored template by key on send/email #127176), which takes workflows-email-from-code, goes to the accepted-send route and posts the version, like a send by template.key. Only a field the caller sent counts, so a form body that omits it keeps master's route.
  • No serializer or OpenAPI change, so no regenerated types.

How did you test this code?

Test seams: the DRF client on send/email/, asserting the body posted to cdp-api (test_transactional_email.py).

Test rationale, each run red first:

  • A send by template.key that posts no version, with or without the send options, or a version from before a save. A PATCH that saves new content is followed by a send that must post the new version.
  • A version read from a lagging read replica.
  • A workflow deleted before the version read that logs an exception.
  • The three rendered-content rows compare the whole send dict, which pins that the field is absent there.
  • A version-behind 503 from cdp-api that answers the generic 503, or that puts workflow_version_behind anywhere in the body.

Proofs (saved with fleet proof on ef44c9b):

Earlier live run, 2026-10-31

Django at 3a30f51c529, cdp-api and both CDP workers at #127130 (f9abed0841c), mail caught in maildev.

Send by template.key Result
Current hash, right after an upload changed the content 202, the new content delivered
Superseded hash 202 with the template_superseded warning, the new content delivered
A hash PostHog doesn't hold 409 template_outdated, nothing sent
With from, reply_to and capture_engagement 202, sent from support@example.com
posthog-node local template (PostHog/posthog-js#5513), new key 409, upload, then 202

Release status

  • This change is behind a feature flag and is not available to users

Docs update

None.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Claude Code, Claude Opus 5.5 (claude-opus-5-5)

Question Answer Who Where
Does a template send deliver what Django graded? Yes, through #127130 and this PR 👤 Human map Silthus#461, decision D3 of 2026-10-25
May the code carry comments? No; names carry the reasons 👤 Human map Silthus#461, 2026-10-25
What does a send get when the pipeline holds an older version? The contract's 503 delivery_unavailable with the version lag in detail; replaces a new public workflow_version_behind code 👤 Human approved (conductor, 2026-10-26)
Does a 503 carry why and fix? No; this stack reuses master's refusal shape, so the lag goes in detail 👤 Human map Silthus#461, 2026-10-28
Key the cache on the hash, or on a version floor? A version floor, read after grading from the writer; replaces D3's "key the cache on the hash" 👍 Accepted default approved (conductor, 2026-10-26)
Does a send by template.id with capture_engagement post the version? Yes, it takes the accepted-send route like a key send; other id sends keep master's route 🤖 Agent dj3 QA swarm, 2026-11-01
Must #127188 deploy with #127187? Yes; apart, a keyed retry of a key send answers 422. Kept as a stated condition, not code 🤖 Agent dj3 QA swarm, 2026-11-01
Must the retry reuse the key? Only if the caller sent one: nothing was sent, so a retry without a key is safe too 👍 Accepted default approved (conductor, 2026-10-26)

Open: a send by template.id posts no version, so one right after an upload can still use the cached copy. Recommendation: leave it; that send grades no hash.

Involvement: one decision from Michael and one from the map's conductor; review round 1 (GPT-6.1 Sol, Opus fresh eyes) fixed, then final reads by GPT-6.1 Sol; dj3 round 1 (GPT-6.1 Sol) fixed, its final read raised the cdp-api route order, answered in the merge risk; live run captured; cross-version proof in #127130. dj3 QA swarm (GPT-6.1 Sol review, Opus fresh eyes): one merge fix, two rejections; live run, shadow diff and cross-version on this head.

🤖 Generated with Claude Code

Silthus and others added 30 commits October 31, 2026 08:32
…endpoint

The single send endpoint now takes either a stored template (`template.id`)
or content the caller's code rendered (`template_key`, `subject`, `html` or
`text`, `from`, `reply_to`, `capture_engagement`). The code form is gated by
`workflows-email-from-code` and goes out through cdp-api's accepted-send
route, under one hidden workflow per key. A key that names a template uses
that template's transactional workflow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both SDKs send distinct_id as a top-level field with template_key, so the
code form refused their sends as an unknown field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s off template sends

- A code send the pipeline does not confirm answers 503 with a detail that says
  the email may still go out, so a caller checks before it sends again.
- capture_engagement has no schema default, so a generated client no longer adds
  it to a template send, which the server refuses.
- A template send honors a top-level distinct_id, like a code send does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The code send takes distinct_id as a top-level field, as the SDKs send it,
so the tool drops the nested identifiers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nt field

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A template uploaded from a repository needs to record who owns its
content, which file it came from, and a hash of what it publishes. This
adds those columns (all nullable) and the content hash function. No
route reads or writes them yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`PUT /api/projects/:id/messaging_templates/by_key/:key/` creates or
updates the template with that key from HTML kept in a repository. The
first upload makes it transactional with the project's default verified
sender; later uploads publish changed content at once, answer
`unchanged` for the same content and, with `if_not_superseded`,
`superseded` for content a newer upload replaced. Code sends under a
key move to the template's workflow once an upload claims the key.

The upload sits behind `workflows-email-from-code`, keeps the PSAK
bypass and workflow access checks of the send route, gets its own rate
limits, and reports a usage event. The MCP tool
`workflows-upload-email-template` calls it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ode sends

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… code

A template managed by code now refuses edits in PostHog with
`403 immutable` until someone releases it with `managed_by: gui`, which
captures a `message template released` usage event. A template made in
PostHog moves into code when someone gives it a `key`: the next upload
under that key keeps its id, workflow and sender, and takes over the code
sends under the key from then on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… template

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A template uploaded from code opens as a read-only preview with its key
and source. Edits refused by an upload wait in a banner that offers a
duplicate, and the menu can stop managing the template in code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`POST /api/projects/:id/messaging/send/email/` now takes
`template: {key, hash?}` for a template uploaded from code. A send by key
may set `from`, `reply_to` and `capture_engagement`; a send by
`template.id` still refuses them. With `template.hash`, a hash PostHog
doesn't hold answers `409 template_outdated` and sends nothing, and a
superseded hash sends the current content with a `template_superseded`
warning. An unknown key answers `404 template_not_found`, or
`409 template_outdated` when the send carries a hash. A code-managed
template without text sends the text its html holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…te source

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…izer

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e edit

A publish refused because an upload took the template back over already
shows the warning banner with "Duplicate with my changes". The global
loader toast also showed the server message, so the user saw two errors.
messageTemplateLogic now owns the `immutable` refusal of saveTemplate.
Every other saveTemplate failure keeps the toast.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A code send, or a send by `template.key`, that names a `from` on no
verified domain now answers `sender_not_verified` with `attr: "from"`, so
an SDK can show which field to fix. A send without `from` whose project
has no verified default sender still names no field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The send tool's Zod stripped unknown keys, so template.content_hash
instead of template.hash sent the email without the hash check. Orval
now generates strict bodies for the email_templates module only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Upload, update and make-transactional join path params and a body, so
they took the params' loose top level and stripped a misspelled key such
as if_not_superseded. The joined input is now strict when its module has
strict bodies. The design document stays open: its skeleton sets
additionalProperties, and the Orval post-step turns those objects into
looseObject, because Orval's strict mode ignores the flag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every open object the OpenAPI marks must now match a generated object, or
the generator fails and names the module, the operation and the path.
Unions with more than one non-null branch, colliding paths and dotted
property names fail the same way instead of loosening a closed sibling.
The update tool now tells the agent to keep the template id.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Base automatically changed from email-v2/11-first-code-send-event to master November 4, 2026 20:20
@trunk-io

trunk-io Bot commented Nov 4, 2026

Copy link
Copy Markdown

Merging to master in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

Silthus and others added 25 commits November 6, 2026 08:15
Master now holds the merged code send layers this branch was stacked on,
so the merge keeps master's code and re-applies only this layer's change:
code-managed fields on message templates and the content hash.

Master's messaging migrations end at 0016_first_code_send and already
hold this branch's 0012 to 0014 under other numbers, so those copies go,
and the code-managed fields migration becomes 0017 on top of
0016_first_code_send.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The template code fields layer now sits on master, so this merge keeps
master's code and re-applies only this layer's change: uploading email
templates from code by key.

Master moved on in a few places this layer touches, and the merge keeps
master's shape there:
- the code send sender helper is code_send_sender_id
- a key's workflow lookup takes the template, not its id
- send and make_transactional declare their scopes on the action, so
  only upload joins the write actions list
- make_transactional still drops content for a send-scoped key

OpenAPI types, MCP tools and tool snapshots are regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The template uploads layer now sits on master, so this merge keeps that
layer's code and re-applies only this layer's change: code-managed
templates refuse edits in PostHog, and a template made in PostHog moves
into code by key.

The update tool's description keeps master's note on the read-only
fields of a fetched template and adds this layer's lock and adoption
text. OpenAPI types, MCP tools and tool snapshots are regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Patch the workflow reload where master's hog flow model calls it, and
expect a key workflow's email step to carry its key when no upload
claimed the template that shares it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e-lock-adopt

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Master reads a template through read_object now, so the race test
claims the template there, and the workflow reload is patched where
the hog flow model calls it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e-send

# Conflicts:
#	posthog/plugins/plugin_server_api.py
#	products/messaging/backend/facade/contracts.py
#	products/messaging/backend/facade/templates.py
#	products/messaging/backend/facade/transactional_email.py
#	products/messaging/backend/migrations/max_migration.txt
#	products/messaging/backend/models/message_template.py
#	products/messaging/backend/models/transactional_email_key.py
#	products/messaging/backend/presentation/views/message_templates.py
#	products/messaging/backend/services/html_text.py
#	products/messaging/backend/services/transactional_email.py
#	products/messaging/backend/tests/api/test_transactional_email.py
#	products/messaging/backend/tests/test_html_text.py
#	products/messaging/frontend/generated/api.schemas.ts
#	products/messaging/frontend/generated/api.ts
#	products/messaging/frontend/generated/api.zod.ts
#	products/workflows/backend/facade/transactional_emails.py
#	products/workflows/backend/services/transactional_email_workflows.py
#	products/workflows/frontend/generated/api.zod.ts
#	products/workflows/mcp/email_templates.yaml
#	services/mcp/schema/generated-tool-definitions.json
#	services/mcp/schema/tool-definitions-all.json
#	services/mcp/src/api/generated.ts
#	services/mcp/src/generated/email_templates/api.ts
#	services/mcp/src/generated/workflows/api.ts
#	services/mcp/src/tools/generated/email_templates.ts
#	services/mcp/tests/unit/__snapshots__/tool-schemas/broadcasts-create.json
#	services/mcp/tests/unit/__snapshots__/tool-schemas/workflows-create.json
# Conflicts:
#	products/messaging/backend/facade/contracts.py
#	products/messaging/backend/facade/templates.py
#	products/messaging/backend/presentation/views/message_templates.py
#	products/messaging/backend/tests/api/test_transactional_email.py
#	products/messaging/frontend/generated/api.schemas.ts
#	products/messaging/frontend/generated/api.ts
#	services/mcp/tests/unit/__snapshots__/tool-schemas/workflows-send-email.json
…d template_hash

Take the key and the hash check as flat send fields beside master's template_id,
regenerate the OpenAPI types and MCP tool schemas, and keep the MCP send tool on
rendered content only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e-editor

# Conflicts:
#	frontend/src/initKea.ts
#	posthog/plugins/plugin_server_api.py
#	products/messaging/backend/facade/contracts.py
#	products/messaging/backend/facade/templates.py
#	products/messaging/backend/facade/transactional_email.py
#	products/messaging/backend/migrations/max_migration.txt
#	products/messaging/backend/models/message_template.py
#	products/messaging/backend/models/transactional_email_key.py
#	products/messaging/backend/presentation/views/message_templates.py
#	products/messaging/backend/services/html_text.py
#	products/messaging/backend/services/transactional_email.py
#	products/messaging/backend/tests/api/test_transactional_email.py
#	products/messaging/backend/tests/test_html_text.py
#	products/messaging/frontend/generated/api.schemas.ts
#	products/messaging/frontend/generated/api.ts
#	products/messaging/frontend/generated/api.zod.ts
#	products/workflows/backend/facade/transactional_emails.py
#	products/workflows/backend/services/transactional_email_workflows.py
#	products/workflows/frontend/TemplateLibrary/TransactionalEmailApiExample.tsx
#	products/workflows/frontend/generated/api.zod.ts
#	products/workflows/mcp/email_templates.yaml
#	services/mcp/schema/generated-tool-definitions.json
#	services/mcp/schema/tool-definitions-all.json
#	services/mcp/src/api/generated.ts
#	services/mcp/src/generated/email_templates/api.ts
#	services/mcp/src/generated/workflows/api.ts
#	services/mcp/src/tools/generated/email_templates.ts
#	services/mcp/tests/unit/__snapshots__/tool-schemas/broadcasts-create.json
#	services/mcp/tests/unit/__snapshots__/tool-schemas/workflows-create.json
# Conflicts:
#	frontend/src/initKea.ts
#	posthog/plugins/plugin_server_api.py
#	products/messaging/backend/facade/contracts.py
#	products/messaging/backend/facade/templates.py
#	products/messaging/backend/migrations/0016_first_code_send.py
#	products/messaging/backend/models/__init__.py
#	products/messaging/backend/models/message_template.py
#	products/messaging/backend/models/transactional_email_key.py
#	products/messaging/backend/presentation/views/message_templates.py
#	products/messaging/backend/services/html_text.py
#	products/messaging/backend/services/transactional_email.py
#	products/messaging/backend/tests/api/test_transactional_email.py
#	products/messaging/backend/tests/test_html_text.py
#	products/messaging/frontend/generated/api.schemas.ts
#	products/messaging/frontend/generated/api.ts
#	products/messaging/frontend/generated/api.zod.ts
#	products/workflows/backend/facade/api.py
#	products/workflows/backend/facade/transactional_emails.py
#	products/workflows/backend/services/transactional_email_workflows.py
#	products/workflows/frontend/TemplateLibrary/TransactionalEmailApiExample.tsx
#	products/workflows/frontend/generated/api.zod.ts
#	products/workflows/mcp/email_templates.yaml
#	services/mcp/schema/generated-tool-definitions.json
#	services/mcp/schema/tool-definitions-all.json
#	services/mcp/scripts/generate-tools.ts
#	services/mcp/scripts/lib/loosen-open-objects.mjs
#	services/mcp/src/api/generated.ts
#	services/mcp/src/generated/email_templates/api.ts
#	services/mcp/src/generated/workflows/api.ts
#	services/mcp/src/tools/generated/email_templates.ts
#	services/mcp/tests/unit/__snapshots__/tool-schemas/broadcasts-create.json
#	services/mcp/tests/unit/__snapshots__/tool-schemas/workflows-create.json
#	services/mcp/tests/unit/email-templates-generated.test.ts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	products/messaging/backend/services/transactional_email.py
#	products/messaging/backend/tests/api/test_transactional_email.py
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@posthog

posthog Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

👋 Visual changes detected for this PR.

Review and approve in PostHog Visual Review

If these changes are unexpected, they may be caused by a flaky test or a broken snapshot on master. Don't approve — rerun the job or wait for a fix.

Install the Visual Review Chrome extension to see visual review results at the top of your pull requests.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant