|
| 1 | +# Architecture |
| 2 | + |
| 3 | +How `@corbits/artifacts` is put together, and where its boundaries are. For |
| 4 | +install, the mount snippet, the route table and the response contracts, see the |
| 5 | +[package README](./packages/artifacts/README.md) — this document is about |
| 6 | +structure and reasoning, and does not repeat them. |
| 7 | + |
| 8 | +## The shape of the thing |
| 9 | + |
| 10 | +A **library, not a service**. It creates no app, starts no background work, and |
| 11 | +opens no pool unless asked. A host calls two functions: |
| 12 | + |
| 13 | +- `runArtifactMigrations(db)` — once at boot, before serving. |
| 14 | +- `mountArtifacts(app, opts)` — registers the artifact routes on a Hono app the |
| 15 | + host already built. |
| 16 | + |
| 17 | +## Where the routes are served |
| 18 | + |
| 19 | +The core registers root-relative paths (`/artifacts*`, |
| 20 | +`/instances/:id/mail-attachments`) and takes no base path, so the *mount point* |
| 21 | +is the host's decision. The convention every `@corbits/*-core` package |
| 22 | +documents, and every example here demonstrates, is **`/api`** — the same prefix |
| 23 | +Interchange serves its own routes under (`app.route("/api/me", …)`, |
| 24 | +`app.route("/api/tenants", …)`). No `/v1` segment, no vendor prefix. |
| 25 | + |
| 26 | +```ts |
| 27 | +const api = new Hono<AppEnv>(); |
| 28 | +mountArtifacts(api, { db, contentStore, resolvePrincipal }); |
| 29 | +app.route("/api", api); |
| 30 | +``` |
| 31 | + |
| 32 | +which serves `/api/artifacts`, `/api/artifacts/:id`, |
| 33 | +`/api/artifacts/:id/versions`, `/api/artifacts/:id/download`, and |
| 34 | +`/api/instances/:instanceId/mail-attachments`. Nesting rather than teaching the |
| 35 | +core a base path keeps the frozen `mountX<E extends Env>(app, opts) => Hono<E>` |
| 36 | +seam untouched. |
| 37 | + |
| 38 | +Everything else it needs arrives through `opts`. Nothing is reached for; the |
| 39 | +dep-guard enforces that mechanically. |
| 40 | + |
| 41 | +## The mount seam |
| 42 | + |
| 43 | +`mountArtifacts<E extends Env>(app: Hono<E>, opts): Hono<E>` is generic over the |
| 44 | +host's Hono `Env`, so it composes with an app that carries its own environment |
| 45 | +rather than requiring a bare `Hono`. |
| 46 | + |
| 47 | +Three options have no sensible default — `db`, `contentStore`, |
| 48 | +`resolvePrincipal` — and the rest degrade a *feature*, never safety, when |
| 49 | +omitted. The README's table of what a minimal host passes is the reference; what |
| 50 | +matters architecturally is that every default **fails closed**: no `adminAuthz` |
| 51 | +means nobody is an admin, no `identity` means no directory and no cross-tenant |
| 52 | +reads, no `provenance` means no decoration. |
| 53 | + |
| 54 | +What the package does **not** require of a host: no auth middleware, no session |
| 55 | +library, no UI. What it DOES require: Interchange's control plane — |
| 56 | +`public.tenant` and `public.principal` must exist before the migrations run, |
| 57 | +because the tables carry hard foreign keys into them. |
| 58 | + |
| 59 | +`resolvePrincipal`'s signature is identical across the Corbits cores, so a host |
| 60 | +mounting more than one passes the same function to each. The resolved tenant is |
| 61 | +authoritative — there is no caller-supplied tenant override anywhere in the |
| 62 | +route surface. |
| 63 | + |
| 64 | +## The ports |
| 65 | + |
| 66 | +Five host seams. Four are types declared in `ports.ts`; `resolvePrincipal` is a |
| 67 | +`mountArtifacts` option, typed at the mount boundary because it takes the host's |
| 68 | +request context as `unknown`. Every optional seam has a **named, exported** |
| 69 | +default. |
| 70 | + |
| 71 | +| Seam | What it is for | Default | |
| 72 | +| --- | --- | --- | |
| 73 | +| `resolvePrincipal` | Who the request runs as. Reads the host session; returns `null` when signed out. | none — required | |
| 74 | +| `ContentStore` | Where an artifact's file bytes live. | none — required | |
| 75 | +| `AdminAuthz` (Seam A) | Authorization. Only archive/unarchive consults it. | `denyAllAdminAuthz` | |
| 76 | +| `Identity` (Seam B) | Owner display names, the agent→human ownership resolution, creator-kind principal sets, and cross-tenant membership. | `anonymousIdentity` | |
| 77 | +| `Provenance` (Seam C) | A **display-only** decorator over serialized rows. | `noProvenance` | |
| 78 | + |
| 79 | +Seam C's display-only status is a contract, not a convention: it may add fields |
| 80 | +to rows on their way out and must never affect *what* is returned or *who* may |
| 81 | +see it. That is why provenance is a port at all — joining a host's workflow |
| 82 | +tables to decorate a row would make this package depend on a schema it must not |
| 83 | +know, and the dep-guard fails the build on exactly that. |
| 84 | + |
| 85 | +Seam B's `ownerIsMemberOfTenant` gates cross-tenant reads and must fail closed; |
| 86 | +the shipped `anonymousIdentity` does. |
| 87 | + |
| 88 | +### ContentStore |
| 89 | + |
| 90 | +`ContentStore` is the substrate seam: |
| 91 | + |
| 92 | +- `put(tx, scope, blob)` persists bytes **inside the caller's transaction** and |
| 93 | + returns the two artifact-row fields a stored file determines (`content` and |
| 94 | + `source`). Taking the transaction rather than the handle is what makes "bytes, |
| 95 | + artifact row, and version 1" a single atomic write — a failure anywhere leaves |
| 96 | + no orphan bytes and no orphan artifact. |
| 97 | +- `get(db, artifact)` resolves **out-of-band** bytes, or `null` when the store |
| 98 | + keeps content in the row itself. It is tenant-scoped: a reference resolving to |
| 99 | + another tenant's bytes must return `null`. |
| 100 | + |
| 101 | +Two implementations ship and both pass the same suite: `InlineContentStore` |
| 102 | +(bytes in a tenant-owned `upload` row, referenced by `source.upload.id`) and |
| 103 | +`DataUrlContentStore` (bytes inline in `content` as a base64 `data:` URL, no |
| 104 | +side-table row, so its `get` returns `null`). A third backend — object storage, |
| 105 | +say — is a third implementation, not a rewrite. The single download path |
| 106 | +resolves the storage conventions in precedence order rather than branching on |
| 107 | +which store is installed. |
| 108 | + |
| 109 | +## Modules |
| 110 | + |
| 111 | +| File | Role | |
| 112 | +| --- | --- | |
| 113 | +| `mount.ts` | HTTP surface: parsing, validation, status codes. | |
| 114 | +| `artifacts.ts` | The core domain — create, revise, list, get, archive, serialize. | |
| 115 | +| `uploads.ts` | `createFileArtifact`, the MIME policies, and the size caps. | |
| 116 | +| `download.ts` | One download path over the three storage conventions. | |
| 117 | +| `content-store.ts` | The two shipped `ContentStore` implementations. | |
| 118 | +| `tools.ts` | Agent-facing tool definitions and windowed artifact reads. | |
| 119 | +| `web-site.ts` | The `web-site` kind's content encoding and validation. | |
| 120 | +| `mail-attachments.ts` | Artifact↔message associations. | |
| 121 | +| `ports.ts` | Four of the five host seam types, and the three fail-closed defaults. | |
| 122 | +| `schema.ts` / `migrations.ts` | The four tables, and the DDL that creates them. | |
| 123 | + |
| 124 | +## Data model |
| 125 | + |
| 126 | +Four physical tables — `artifact`, `artifact_version`, `upload`, |
| 127 | +`mail_attachment_ref` — plus this package's own migration ledger. |
| 128 | + |
| 129 | +**Hard control-plane foreign keys, by design.** `tenant_id` references |
| 130 | +`public.tenant(id)` (`ON DELETE CASCADE` — a deleted tenant takes its artifacts |
| 131 | +with it) and `principal_id` / `owner_principal_id` reference |
| 132 | +`public.principal(id)` (`ON DELETE SET NULL` — a removed principal detaches its |
| 133 | +artifacts rather than destroying them). This package is coupled to Interchange: |
| 134 | +it mounts on Interchange-shaped hosts only, and the host's own migrations must |
| 135 | +have run before `runArtifactMigrations`. The internal keys — |
| 136 | +`artifact.parent_id` (self-referential, cascading) and |
| 137 | +`artifact_version.artifact_id` — cascade as before. |
| 138 | + |
| 139 | +**`kind` is free-form text, not a pg enum,** validated at the application edge. |
| 140 | +New kinds cost no migration. What is *not* free-form is the import allowlist: |
| 141 | +`POST /api/artifacts` may only mint `link` or `document`, so an untrusted caller |
| 142 | +cannot stamp a file-shaped, downloadable kind onto a row whose content is a URL |
| 143 | +or a pasted body. |
| 144 | + |
| 145 | +**History is append-only.** Every create and every revision writes an |
| 146 | +`artifact_version` row — including version 1, written eagerly with the artifact — |
| 147 | +so a version-pinned read always resolves. The version bump has a double guard: |
| 148 | +`SELECT ... FOR UPDATE` serializes writers, and the `(artifact_id, version)` |
| 149 | +unique constraint makes a racing writer that somehow computed the same next |
| 150 | +version fail loudly rather than corrupt history. |
| 151 | + |
| 152 | +**Archival is a soft hide.** `archived_at` null means visible; a timestamp means |
| 153 | +hidden from discovery. Deep links to archived artifacts still load. |
| 154 | + |
| 155 | +**`parent_id` has no live writer.** It is kept deliberately as the declared |
| 156 | +nesting seam: one nullable column and one cascade now, versus a breaking schema |
| 157 | +change the moment a host nests. |
| 158 | + |
| 159 | +**`upload` is never a standalone resource.** There is no `POST /uploads`; every |
| 160 | +upload eagerly mints its artifact, and the row is reachable only through |
| 161 | +`source.upload.id`. `mail_attachment_ref` carries no bytes at all — the file |
| 162 | +already *is* an artifact, and the ref only records which artifacts rode with |
| 163 | +which message. |
| 164 | + |
| 165 | +The list index is `(tenant_id, updated_at, id)`. The `id` is the list's |
| 166 | +tie-break and must be *in* the index, or the keyset cursor's row-value |
| 167 | +comparison falls out of the index condition into a filter and drags a sort |
| 168 | +behind it. |
| 169 | + |
| 170 | +## Migrations |
| 171 | + |
| 172 | +`runArtifactMigrations(db)` is idempotent and safe to call unconditionally on |
| 173 | +every boot of every replica. |
| 174 | + |
| 175 | +- The whole run is one transaction whose first statements are |
| 176 | + `SET LOCAL client_min_messages = warning` and a **transaction-scoped** |
| 177 | + advisory lock. A transaction pins one pooled connection, so the lock, the |
| 178 | + ledger read and the DDL are the same session; the lock releases on commit or |
| 179 | + rollback, so there is no unlock call to lose on an error path. |
| 180 | + `CREATE TABLE IF NOT EXISTS` is not itself race-safe, so the lock — not the |
| 181 | + `IF NOT EXISTS` — is what makes concurrent cold starts safe. |
| 182 | +- Lowering `client_min_messages` is why a re-run prints **nothing**: every |
| 183 | + statement is `IF NOT EXISTS`, and on the second boot Postgres answers each with |
| 184 | + a NOTICE that postgres.js would otherwise dump to the console, making a clean |
| 185 | + re-boot look like a wall of errors. `SET LOCAL` scopes it to the transaction |
| 186 | + and stops at NOTICE — WARNING and above still reach the host. |
| 187 | +- Each migration applies inside a nested transaction (a savepoint) together with |
| 188 | + its ledger row, so a migration can never be recorded as applied with only some |
| 189 | + of its statements run. |
| 190 | +- The ledger is this package's own table, `artifacts.migrations`, |
| 191 | + never shared with a host's. Each row records a **checksum of the migration's |
| 192 | + rendered SQL**, so editing a shipped migration fails with |
| 193 | + `MigrationChecksumError` on the next boot instead of letting existing and |
| 194 | + fresh databases diverge silently. Ship a new migration instead. The column is |
| 195 | + `NOT NULL`, so the guarantee is unconditional: there is no unrecorded row for |
| 196 | + the runner to adopt and wave through. |
| 197 | + |
| 198 | +**The package owns its own Postgres schema.** Every table, index and the ledger |
| 199 | +live in `artifacts`, created by the runner and qualified in every |
| 200 | +DDL statement and every query — nothing resolves through `search_path`, so the |
| 201 | +package shares a database with the host's control plane without ever being able |
| 202 | +to collide with (or silently adopt) a host table of the same name. The coupling |
| 203 | +to the host is explicit instead: `tenant_id` and the principal columns are hard |
| 204 | +FKs into `public.tenant` / `public.principal` (see the data model). |
| 205 | + |
| 206 | +## Boundaries |
| 207 | + |
| 208 | +Owned by this package: the four tables and their migrations; the HTTP surface, |
| 209 | +its validation and its status codes; the version and archive semantics; the |
| 210 | +upload **gate** (`createFileArtifact` takes `policy` as a required argument and |
| 211 | +refuses anything outside it before the `ContentStore` is touched); and the |
| 212 | +download path with its `nosniff`/`attachment` behaviour. |
| 213 | + |
| 214 | +Supplied by the host: the Hono app and the database handle; who the caller is; |
| 215 | +whether they are an admin; the directory, if there is one; provenance |
| 216 | +decoration; and a `ContentStore`. |
| 217 | + |
| 218 | +## Known limits |
| 219 | + |
| 220 | +- **No file parsing, ever.** No PDF parser, no spreadsheet parser, no text |
| 221 | + extractor, and none is planned. An extractor is a heavyweight, fast-moving |
| 222 | + native dependency, and what the extracted text is *for* is the host's product. |
| 223 | + The contract the package offers a parsing host instead is **parse before you |
| 224 | + store**: `createFileArtifact` is the only way a file becomes an artifact, so a |
| 225 | + host that parses first and fails leaves nothing orphaned. |
| 226 | +- **Two of the three MIME allowlists are host-owned surfaces.** This package |
| 227 | + ships all three constants but only owns the gallery import route; spreadsheet |
| 228 | + ingest and attachment divert are the host's routes calling this package's gate. |
| 229 | +- **Upload caps are fixed constants,** not configuration: 10 MB per file, 50 |
| 230 | + files and 100 MB per request. |
| 231 | +- **`InlineContentStore` keeps bytes in Postgres.** That is a deliberate |
| 232 | + zero-dependency default, not a recommendation at scale; a large corpus wants a |
| 233 | + `ContentStore` over object storage. |
| 234 | +- **List paging caps at 100** rows (default 20). |
| 235 | +- **One 404 covers four causes** for a resolved caller — never minted, |
| 236 | + malformed, a `skill-draft`, or another tenant's. Distinguishing them would be |
| 237 | + an existence oracle. Expect no more detail than that from the API. |
0 commit comments