feat(server): add authenticated HTTP parity for project merge

[eSagraAI]

📋 Pre-flight Checks

• I have searched existing issues and this is not a duplicate
• I understand this issue needs status:approved before a PR can be opened

🔍 Problem Description

Local clients that integrate directly with engram serve cannot invoke the existing project-merge capability without also starting MCP or using the interactive CLI consolidation flow.

The capability already exists through:

• MCP: mem_merge_projects
• CLI: engram projects consolidate
• Store: Store.MergeProjects

There is no equivalent HTTP endpoint. The existing POST /projects/migrate route is not an alternative project-merge API: it is a deprecated compatibility alias for the ownership-rescue handler.

This is an administrative transport-parity gap, not a request for new merge semantics.

mem_merge_projects is intentionally restricted to the MCP admin profile and is annotated as destructive, idempotent, non-read-only, and closed-world. Store.MergeProjects already provides the authoritative safety behavior:

• every source must normalize to the canonical project;
• all sources are validated before mutation;
• observations, sessions, prompts, pending sync mutations, and sync enrollment are handled in one transaction;
• unsafe mixed-source requests fail without partial mutation;
• repeated valid requests are idempotent.

HTTP clients should be able to use that existing behavior without reimplementing it or mounting MCP solely for this administrative operation.

💡 Proposed Solution

Expose a narrowly scoped, authenticated JSON endpoint from engram serve that delegates directly to Store.MergeProjects.

One possible HTTP shape is:

POST /projects/merge
Authorization: Bearer <ENGRAM_HTTP_TOKEN>
Content-Type: application/json

{
  "from": ["Engram", "ENGRAM"],
  "to": "engram",
  "confirmed": true
}

A successful response could use the existing store.MergeResult JSON shape:

{
  "canonical": "engram",
  "sources_merged": ["Engram", "ENGRAM"],
  "observations_updated": 12,
  "sessions_updated": 2,
  "prompts_updated": 4
}

The route and property names above are illustrative rather than a closed design decision. The final contract should follow maintainer preference and existing HTTP conventions.

Whichever contract is selected, the implementation should:

• protect the endpoint with requireConfiguredAuth, not optional requireAuth;
• return 503 when ENGRAM_HTTP_TOKEN is not configured;
• return 401 for missing or invalid bearer credentials;
• cap the request body at 8 KiB, matching the existing ownership-rescue admin endpoint;
• require an explicit confirmed: true;
• require at least one non-blank source and a non-blank canonical target;
• call Store.MergeProjects directly;
• keep normalization, source eligibility, atomicity, sync migration, and idempotency authoritative in the store;
• avoid raw SQL or duplicated merge rules in the HTTP handler;
• map store validation failures to 400 without relying on error-string matching or reimplementing those validations;
• return 200 for successful merges, including valid idempotent no-ops;
• return a generic 500 for unexpected store/infrastructure failures;
• call notifyWrite() after a successful merge so autosync can observe any record, journal, or enrollment changes;
• not gate notifyWrite() solely on observation/session/prompt counters, because sync identity or enrollment may change independently;
• preserve all existing MCP, CLI, sync, and HTTP behavior;
• document the final contract in DOCS.md.

Suggested acceptance criteria:

• A valid authenticated and confirmed request merges observations, sessions, and prompts and returns the existing merge result fields.
• Pending sync mutations and enrollment continue to follow the canonical project through Store.MergeProjects.
• Autosync is notified after a successful merge.
• A repeated valid request succeeds idempotently with no additional records moved.
• An unset server token returns 503 without mutation or autosync notification.
• Missing or invalid credentials return 401 without mutation or autosync notification.
• Missing/false confirmation returns 400 without invoking the merge.
• Invalid JSON, oversized payloads, missing targets, and empty source lists return 400.
• A source that does not normalize to the canonical project returns 400.
• A mixed valid/invalid source list fails closed with no partial mutation.
• Unexpected store errors return 500 without exposing infrastructure details.
• Existing MCP and CLI merge tests continue to pass unchanged.

📦 Affected Area

Other — HTTP API (engram serve)

🔄 Alternatives Considered

• Require HTTP clients to start MCP: this adds a second transport solely to access behavior already available in the same Engram process and store.
• Require the interactive CLI consolidation flow: this does not serve applications integrating programmatically with engram serve.
• Reimplement merge behavior in HTTP clients: rejected because validation, atomicity, and sync behavior would drift from Engram.
• Implement merge logic in the HTTP handler: rejected because Store.MergeProjects is already the authoritative domain/store operation.
• Reuse POST /projects/migrate: rejected because that route is now a deprecated alias for ownership rescue and has different semantics.
• Include this in feat(server): add HTTP parity for pinning and topic-key suggestions #1105: rejected because project merge is an administrative/destructive operation with stricter authentication, confirmation, payload, and sync requirements.

📎 Additional Context

Verified against current main at commit 9632b56f21e37b41fc40bdb9c8a23bc1d6e5ecf6.

Relevant implementation:

• internal/mcp/mcp.go
  • ProfileAdmin
  • mem_merge_projects registration and annotations
  • handleMergeProjects
  • queuedWriteHandler
• internal/store/store.go
  • MergeResult
  • Store.MergeProjects
  • projectMergeSourceVariants
  • migrateProjectSyncIdentityTx
  • backfillProjectSyncMutationsTx
• internal/server/server.go
  • Server.routes
  • requireConfiguredAuth
  • handleRescueProjectOwnership
  • notifyWrite
• cmd/engram/main.go
  • cmdProjectsConsolidate

Related work:

• feat(server): add HTTP parity for pinning and topic-key suggestions #1105 / PR feat(server): add HTTP parity for pinning and topic-key suggestions #1109 explicitly leave mem_merge_projects for a separate administrative issue.
• security(http-api): add optional local auth for export, import, and destructive endpoints #200 / PR fix(server): optional local auth for destructive HTTP endpoints #429 established bearer-token protection for sensitive HTTP operations.
• projects consolidate over-merges unrelated projects (data-loss risk); cwd-based detection creates duplicate projects for one logical project #578 / PR fix(projects): prevent unsafe consolidation merges #810 hardened project consolidation and its sync behavior.
• How to force a full re-sync of a project to self-hosted cloud after an in-place project merge/rename? #595 documented sync divergence after project merge/rename.
• bug(store): MergeProjects skips sources that normalize to the canonical name #749 and projects consolidate / mem_merge_projects: case-only name pairs merge 0 records #911 covered historical case-only merge failures.
• bug(cli): projects consolidate ignores piped stdin and proceeds with the full candidate set — "none" included #1007 / PR fix(cli): require dry-run consolidation selection #1063 established fail-closed confirmation behavior for CLI consolidation.

No existing issue or PR was found for authenticated HTTP exposure of Store.MergeProjects.