Skip to content

feat(data-warehouse): add 1 missing table to the E2B source - #124262

Merged
trunk-io[bot] merged 1 commit into
masterfrom
tom/dwh-endpoints-e2b
Oct 27, 2026
Merged

trunk-io[bot] merged 1 commit into
masterfrom
tom/dwh-endpoints-e2b

Conversation

@Gilbert09

Copy link
Copy Markdown
Member

Problem

  • E2B users cannot attribute a sandbox or a build to a template tag (for example latest) in the warehouse.
  • The endpoint-coverage audit listed 4 E2B endpoints with no table. Only one of them is warehouse-table material on the API we implement.

Changes

  • A new template_tags table lists every tag of every template, with the build that the tag points at.
  • It fans out from templates, one request per template, like template_builds.
  • The key is templateID plus tag, because the same tag name (for example latest) occurs on many templates.
  • The table has no partition key. A tag's createdAt changes when the tag moves to a new build.
  • Mechanical: canonical column descriptions, one line in the source doc, and the ticked box in COVERAGE_GAPS_APPENDIX.md.
Table Endpoint Sync mode
template_tags GET /templates/{templateID}/tags Full refresh

Full refresh only: the endpoint has no timestamp filter and no pagination.

Skipped endpoints

Each one was checked against the E2B OpenAPI spec. The appendix already records these reasons.

  • GET /teams: the spec allows only AuthProviderBearerAuth, so the team API key of this source cannot call it. The response also contains the team's live apiKey.
  • GET /teams/{teamID}/metrics/max: each call returns one {timestamp, value} object for one metric. That is one value, not a table, and it is the maximum of the existing team_metrics series.
  • GET /templates/{templateID}/builds/{buildID}/status: it needs a second fan-out hop with one request per build. Its only non-log fields already arrive in template_builds. The rest is build log lines.

How did you test this code?

  • Ran the E2B source tests and the shared source tests in sources/tests/ filtered to E2B. Both pass.
  • Ran mypy products/warehouse_sources products/data_warehouse and ruff on the changed files.
  • Not done: a sync against a live E2B account. The test data is invented.

Test rationale: test_template_tags_carry_their_template_id catches tag rows that lose their templateID. Two templates that share the tag latest would then collide on the composite key. The nearest test, test_fanout_checkpoints_under_its_own_slot_and_resumes_from_it, checks only resume state, not row shape.

Release status

  • No feature flag controls this change

Docs update

The source doc at e2b/e2b.md now mentions template tags. The table list renders from get_schemas and the canonical descriptions.

Add a template_tags table that reads GET /templates/{templateID}/tags
for each template. Skip /teams, /teams/{teamID}/metrics/max and the
build status endpoint for the reasons in the coverage appendix.
@trunk-io

trunk-io Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

😎 Merged successfully - details.

@Gilbert09 Gilbert09 added the stamphog Request AI approval (no full review) label Oct 27, 2026 — with Talyn App
@Gilbert09
Gilbert09 requested a review from a team October 27, 2026 17:28
@parameterai

parameterai Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Risk: No findings

This PR adds a template_tags table to the E2B warehouse source: a new endpoint config (GET /templates/{templateID}/tags, full refresh, composite key templateID+tag) that fans out from templates, plus canonical column descriptions, a doc line, an appendix tick, and a row-shape test. It follows the existing template_builds fanout pattern exactly, and I found no security risk: no new trust boundary, no sensitive fields in the endpoint's data, and no new request surface beyond the same-host, redirect-pinned, key-redacted session every other E2B endpoint already uses.

Sentinel reviewed bf1ed58 · Review settings

@Gilbert09 Gilbert09 added the missing-endpoint Adds tables for endpoints a warehouse source does not import yet label Oct 27, 2026

Copy link
Copy Markdown
Member Author

/trunk merge

@bill-ph bill-ph left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the PR description, diff, existing discussion, E2B source wiring, shared fan-out behavior, and the added regression test. Checked endpoint response shape and authentication against the E2B OpenAPI spec, composite-key preservation, pagination, full-refresh partitioning, data safety, performance, and scope/overengineering. The change reuses the existing source machinery and stays within scope. Tests, builds, and linters were not run, as requested.

No issues found.

— Robo Bill v2 (gpt-6.1-sol, high reasoning)

@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested a review from a team October 27, 2026 17:29

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved.

The ingestion change has current-head review assurance and an owning-team author. No unresolved substantive concerns or other refusal grounds were found.

  • Author wrote 50% of the modified lines and has 25 merged PRs in these paths (familiarity MODERATE).
  • PostHog Review (Standard) reviewed the current head: no findings.
  • bill-ph reviewed the current head.
Gate mechanics and policy version
Gate Result
prerequisites ✓ all clear
deny-list ✓ no deny categories matched
size ✓ 20L, 2F substantive, 46L/5F incl. docs/generated/snapshots — within ceiling
tier ✓ T1-agent / T1b-small (46L, 5F, single-area, feat)
stamphog 2.6.1 .stamphog/policy.yml @ bf1ed58 · reviewed head bf1ed58

@github-actions

Copy link
Copy Markdown
Contributor

Hey @Gilbert09! 👋

It looks like your git author email on this PR isn't your @posthog.com address (owerstom@gmail.com). Since you're on the PostHog team, it's worth pointing your local git author email at your @posthog.com address. Why it matters:

  • Consistent work identity in git history — internal tooling that attributes commits to team members keys off your @posthog.com address.
  • Keeps team contributions easy to tell apart from external community ones when scanning history.

You can fix it for this repo with:

git config user.email "you@posthog.com"

Or set it globally with git config --global user.email "you@posthog.com". No need to redo this PR — just a nudge for next time. 🙂

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

🤖 CI report

⚠️ Trunk lane — backend Python lane (py:product:warehouse_sources)

This PR is assigned to the backend Python lane (py:product:warehouse_sources). It runs backend Python tests and may merge in parallel with PRs in other lanes.

✅ Duplication (Python) — clean

New Python code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

✅ Duplication (TypeScript) — clean

New TypeScript code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

@trunk-io
trunk-io Bot merged commit 72c4ac7 into master Oct 27, 2026
252 checks passed
@trunk-io
trunk-io Bot deleted the tom/dwh-endpoints-e2b branch October 27, 2026 18:49
@deployment-status-posthog

deployment-status-posthog Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-10-27 19:23 UTC Run
prod-us ✅ Deployed 2026-10-27 20:13 UTC Run
prod-eu ✅ Deployed 2026-10-27 20:15 UTC Run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

missing-endpoint Adds tables for endpoints a warehouse source does not import yet stamphog Request AI approval (no full review)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants