Skip to content

chore(agents): move topic rules out of the root agents file - #123122

Open
Silthus wants to merge 6 commits into
PostHog:masterfrom
Silthus:chore/slim-root-agents
Open

Silthus wants to merge 6 commits into
PostHog:masterfrom
Silthus:chore/slim-root-agents

Conversation

@Silthus

@Silthus Silthus commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Problem

  • Every Claude Code and Codex session in this repo loads the root AGENTS.md in full before its first step, whatever the task.
  • The file is 46,010 bytes, roughly 17k tokens, and each session reads it again on every turn.
  • Most of it applies to one kind of work only: backend architecture, per-language code style, CI workflows, feature tracking.

Changes

  • The root file drops to 25,741 bytes. Agents now read the topic rules on demand.
  • A routing table before the topic sections names each guide and the work that needs it. It says that a diff review counts.
  • The table is the only trigger for each guide, so it is the one place to keep current.
  • Without the table, Haiku 5.5 and Sonnet 5.5 reviewed a diff without opening any guide and missed guide-only rules.
  • Each moved topic keeps its heading in the root file, with one line that names its guide.
  • The text moved word for word. Only heading levels and relative link paths changed.
Root section New home
Architecture guidelines .agents/architecture.md
Code Style .agents/code-style.md
CI / GitHub Actions .agents/ci.md
Feature usage tracking .agents/feature-usage-tracking.md
Agent automation (the ladder and hook rules) .agents/agent-automation.md
User-facing copy /writing-user-facing-copy, which already held every bullet. It gains "Keep one tone across surfaces", the one rule it lacked.
  • Mandatory skill invocation moves up from a subsection of Agent automation to its own section, unchanged.
  • These stay in the root file unchanged: the codebase map, commands, commit and PR rules, the public repo guidance, Security, and Mandatory skill invocation.
  • The guides sit beside .agents/security.md, the existing home for agent guides read on demand.
  • The guides keep their ownership: CODEOWNERS and the stamphog agent_steering category now list the five files with the root file.
  • test_agents_md_enforcement_tags.py checks tags in the root file and every .agents/*.md guide. It requires a tag on every rule in the architecture and code style guides.
  • A new test case fails when someone adds a rule under a root pointer heading, where the tag check would not see it.
  • The repo_invariants path filter in ci-backend.yml now matches .agents/*.md, so an edit to a guide still runs the test.
  • The frozen policy copy in the stamphog test_policy.py gains the new agent_steering pattern.
  • Mechanical: references to the moved rules now point at the guides. These are in three nested AGENTS.md files, six skills, the code-reviewer agent, two code comments, a semgrep message, the hooks warning, a dagster comment, .greptile/files.json, and .stamphog/README.md.
  • Boy scout: the conversations skill entry linked a path that no longer exists. It now names /organizing-conversations-code.

Warning

🟡 Merge risk: medium · two-way door (merge-risk rubric)

  • Every agent session changes: a rule an agent used to see up front now loads only when the pointer line sends it there. An agent that skips the pointer misses the rule.
  • The diff also touches CI path filters, CODEOWNERS and stamphog policy. Each edit only adds paths, so no existing check or approval goes away.
  • A revert restores the old file in full. No production code changes.

How did you test this code?

Test rationale: the change rewires the existing invariant test. No new test is needed.

  • test_agents_md_enforcement_tags.py and the stamphog test_policy.py pass on the new layout.
  • Three temporary edits each failed the tag test, then were reverted: an untagged rule in .agents/code-style.md, a [lint: no-such-rule] tag in .agents/ci.md, and a rule under the root ## Code Style heading.
  • A script checked that every line of the old file is in the new root file or a guide. The exceptions are the user-facing copy bullets, the reworded tag legend, and the fixed conversations link.
  • Every Markdown link in the root file, the guides, and the edited skills resolves.
  • Agent pickup: headless planning runs of invented tasks on Haiku 5.5, Sonnet 5.5, Opus 5.5 and GPT-6.1 Sol. On the final file, 38 of 40 guided runs opened the right guide, against 33 of 56 before the table. All four models now flag the review case's guide-only rule. Per-model results are in the confidence report comment.
  • Not checked: observed sessions after merge. The runs ask for a plan, not an implementation.
  • Not run: the semgrep devex scan. The pre-push preflight had no cached semgrep binary. The only semgrep file change is message text.

Release status

  • No feature flag controls this change

Docs update

None. The change only moves agent instructions.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Claude Code, Claude Opus 5.5 (claude-opus-5-5)

  • Skills invoked: /editing-agents-md, /writing-pr-descriptions, pr-description, merge-risk, leaving-pr-comments.
  • A later session measured guide pickup in headless runs and tested five wordings for the pointers. The routing table won. Rewording the pointers alone changed nothing, and the table's position in the file made no difference.
  • Review of the table: two Claude Opus 5.5 reads at high effort. They found triggers that the table and the old pointer lines disagreed on. The fix made the table the only trigger. Rejected: a second link to frontend/src/AGENTS.md, because .agents/code-style.md already sends frontend work there.
  • Review: one Claude Opus 5.5 review at high effort, then two final reads of the fixes. It found the stamphog frozen-policy test failure, seven stale references, two gaps in the tag test, and four missed pointer triggers. All are fixed. Rejected: adding the guides to .coderabbit.yaml, because CodeRabbit is retired here.
  • No duplicate: a search of open PRs with AGENTS.md in the title found none.
  • Public artifact: the body holds only size numbers about the file itself.
  • The PR could not be assigned: the author account has read access to this repo only.
  • The guides went into .agents/ and not into a new docs/agents/ tree, because .agents/security.md already sets that pattern.
  • Open question for the DRI: the root file is still above a 15 KB goal. Getting there means moving the "Invoke when in the area" skill list, or parts of the PR rules. This PR keeps both in place.

🤖 Generated with Claude Code

@trunk-io

trunk-io Bot commented Oct 25, 2026

Copy link
Copy Markdown

Merging to master in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here


- **Sentence case.** Capitalize only the first word and proper nouns. "Save as view", not "Save As View". "Product analytics", not "Product Analytics".
- **Be direct and friendly.** Say what happened and what to do next.
- **Be direct and friendly.** Use short sentences and keep one tone across surfaces. Say what happened and what to do next.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **Be direct and friendly.** Use short sentences and keep one tone across surfaces. Say what happened and what to do next.
- **Be direct and friendly.** Keep one tone across surfaces, say only what happened and what to do next.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Claude Opus 5.5 responding on behalf of Michael

Applied in 4d14ea9, thanks!

Comment thread .github/CODEOWNERS
/.agents/architecture.md @PostHog/team-devex
/.agents/ci.md @PostHog/team-devex
/.agents/code-style.md @PostHog/team-devex
/.agents/feature-usage-tracking.md @PostHog/team-devex

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏼

@parameterai

parameterai Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Risk: No findings

This increment is CI-only: wrapping uv sync in a 270s timeout, dropping a redundant ensure_migration_defaults call (the custom migrate command re-seeds it after every default-DB migrate), removing no-op checkout options (no LFS files exist in the repo, fetch-depth 1 is the default, fresh workspace makes clean: false inert), and gating dagster CI jobs to the main repo for push/schedule events. I traced each surface and found no security impact: no production code, no auth or trust-boundary change, and no check protecting the main repo is removed.

Sentinel reviewed 8a60702 · Review settings

@posthog-security-review-bot posthog-security-review-bot Bot added the security-review Request a security review label Oct 26, 2026
@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested a review from a team October 26, 2026 12:55
@pr-assigner-resolver-posthog

Copy link
Copy Markdown

👀 Auto-assigned reviewers

These soft owners were skipped because they only have minor changes here. Nothing blocks merge, so self-assign if you'd like a look:

  • @PostHog/team-growth (.agents/owners.yaml)

Soft owners come from each directory's owners.yaml and each product's product.yaml (resolved nearest-file-wins). For a skipped owner, the locator is the file that decided it. Generated files and lockfiles are ignored when deciding ownership.

@posthog-security-review-bot

posthog-security-review-bot Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

✅ Security review complete — 3fad41b6683b

No findings above the confidence threshold, in 6 min.

Add the security-review label to run again on the latest commit.

@posthog-security-review-bot posthog-security-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agent-driven security review - no findings.

@posthog-security-review-bot posthog-security-review-bot Bot removed the security-review Request a security review label Oct 26, 2026
@Silthus

Silthus commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

Note

🤖 Claude Opus 5.5 responding on behalf of Michael

Confidence report: do agents pick up the moved rules?

I took the recurring task shapes from my past Claude Code and Codex sessions in this repo and wrote 9 test cases from them. Each case needs a rule that now lives only in an .agents/*.md guide. All cases ran headless on Haiku 5.5, Sonnet 5.5, Opus 5.5 and GPT-6.1 Sol, against the base commit, the first version of this PR, five wording variants, and the final file. That makes 266 runs.

Verdict: with the routing table from 5619465 and e8cc8cb, I'm confident in this for all four models.

Model Opened the right guide: first version → final Review case: base / first version / final
Haiku 5.5 6/14 → 10/11 1/1 · 0/2 · 1/1
Sonnet 5.5 8/14 → 11/11 1/1 · 0/2 · 1/1
Opus 5.5 5/14 → 10/11 1/1 · 2/2 · 1/1
GPT-6.1 Sol 14/14 → 7/7 1/1 · 2/2 · 1/1
  • The final file applied the rule under test in 39 of 44 runs. Every miss was the double-submit check on the frontend button case, and one of them was a probe miss. Sonnet also missed that check on base (0/1), so it predates this PR.
  • No model opened a guide for the control question (a commit type for a docs typo fix), so the table does not cause extra reads.

What the first version missed

  • Ad-hoc reviews. The review case's diff calls posthoganalytics.capture() in a Celery task. On the first version, Haiku and Sonnet flagged it in 0 of 4 runs, against 2 of 2 on base. Neither model opened a guide, because a diff in the prompt did not read as "add or change code".
  • Guide reads on code tasks. Claude models opened the guide in about a third of the runs, and found the rule in the code in most of the rest.

What I tried

Haiku and Sonnet on the four weakest cases, 2 runs per cell:

Variant Review case flagged Guide opened Input tokens per run
First version of this PR 0/4 3/16 530k
"add, change, or review" in the pointer lines 0/4 0/8 n/a (2 cases only)
Pointer lines that name their traps 4/4 10/16 466k
One line at the top: "every task, a review included, starts by opening the guides" 4/4 16/16 593k
Routing table 4/4 16/16 538k
Base (old file in full) 2/2 n/a 661k (1 run per cell)
  • The routing table won, and its position does not matter: placed mid-file, it fixed 8 of 8 runs. That matches the null result on position in /editing-agents-md.
  • A review found that the table and the old pointer lines named different triggers for the same guide. Now each guide's row holds its full trigger, and each topic section only names its guide.

Other findings

  • Codex now sees the whole root file. Codex truncates project docs at 32 KiB. On base, it cut AGENTS.md at byte 32,643 of 46,010. So GPT never saw Comments, Tests, User-facing copy, Feature usage tracking, Agent automation, or Mandatory skill invocation. The new root file is 25,741 bytes and fits.
  • Sessions start lighter. On the Claude models, the first-turn context drops from about 40.0k to 32.1k tokens, 7.8k (20%) fewer. The table costs about 0.7k of the 8.6k the first version saved.
How I ran it
  • The cases are invented from task shapes, not copied from sessions: a Celery task, a Temporal activity, a per-team cap, a Python helper, a frontend button, a publish workflow, a Claude Code hook, a diff review, and a control question. The build prompts ask for a plan, not an implementation.
  • Claude: claude -p --effort high, project settings only (no user instructions, MCP servers, or plugins), subagents off, edit tools denied.
  • GPT: codex exec with a read-only sandbox, high reasoning, and a clean Codex home without a user AGENTS.md. Codex still listed user-level skills from the machine.
  • "Opened the guide" means a tool call named the guide's path. "Rule applied" means a regex matched the agent's messages, and I spot-checked the answers by hand.
  • I reran every run that hit a rate limit.
  • The sample is small: 1 or 2 runs per cell. A one-run difference is noise. The review gap and the reads the table restored are the consistent signals.

@feliperalmeida feliperalmeida left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approved .github/CODEOWNERS changes.

@Radu-Raicea
Radu-Raicea removed the request for review from a team October 27, 2026 13:31
@Silthus
Silthus force-pushed the chore/slim-root-agents branch from 4d14ea9 to 8a60702 Compare October 27, 2026 23:37
@signed-commit-check

signed-commit-check Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Commit signatures resolved

GitHub now verifies every commit signature in this PR. No further action is needed.

Silthus and others added 6 commits October 27, 2026 23:39
The root AGENTS.md loads in full in every agent session. Move the
architecture, code style, CI, feature usage tracking, and agent
automation rules into .agents/<topic>.md guides that agents read on
demand, and leave one pointer line under each original heading.
The user-facing copy bullets already live in /writing-user-facing-copy.

Point the enforcement-tag invariant test, its CI path filter, CODEOWNERS,
the stamphog agent_steering category, and every reference at the new
guides. Root file: 46,010 bytes to 22,404 bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add the new guides to the frozen stamphog policy copy, repoint the
remaining references to moved rules, and make the tag test read every
.agents guide and reject rules under the root pointer headings. Widen
three pointer lines to cover every trigger of the rules they replace,
and promote Mandatory skill invocation to its own section.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Haiku and Sonnet skipped the per-section pointers when a task only
showed a diff to review, and missed guide-only rules there. A table that
maps each area to its guide, and says a review counts, made every
measured run open the right guide.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The table and the old pointer lines named different triggers for the
same guide, so they would drift. Each guide's row now holds its full
trigger, and the section under each heading only names the guide. The
editing-agents-md skill says where a new trigger goes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Silthus
Silthus force-pushed the chore/slim-root-agents branch from 8a60702 to d2ddf4e Compare October 27, 2026 23:41

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants