Skip to content

Commit 7a6b2ac

Browse files
committed
docs(golang): show the query methods, the database/sql query and the decrypt-and-verify step, and say se_query is gone
The README and the package documentation describe one method per operator, what each returns, why a matches query carries Verify and TargetLimit and an exact one neither, and that a MatchQuery holds its text because Go cannot wipe a string. The ADR-0007 amendment records se_operators, the removal of se_query, and the one reason se_term stays. Part of #1189
1 parent ea67ace commit 7a6b2ac

7 files changed

Lines changed: 160 additions & 19 deletions

File tree

‎languages/golang/cmd/stashgen/README.md‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ Your program calls those functions, and it never builds or names a plan.
4040
```go
4141
encrypted, err := users.Encrypt(ctx, cipher, people)
4242
opened, err := users.Decrypt(ctx, cipher, encrypted)
43-
query, err := users.Fields.Email.Query(ctx, cipher, "bob@example.com")
43+
query, err := users.Fields.Email.Eq(ctx, cipher, "bob@example.com")
4444
```
4545

4646
6. Store the encrypted type.
@@ -123,7 +123,12 @@ For `-type User`, the file `user_stash.go` holds:
123123
- `Encrypt` and `Decrypt`, which take a slice and return a slice, and send one ZeroKMS request for each 500 sealed values in it, plus one the first time a keyset is used.
124124
- `Fields`, with one entry for each sealed field.
125125
An opaque struct, or a struct with no sealed field outside an opaque one, gets no `Fields`: nothing in it is sealed on its own, so nothing can be queried.
126-
An entry encrypts one value, and it has a query method only for what the field declares: `Query` for `encrypt_into`, and `Equality`, `Match`, `Ore` or `Ope` for `index=`.
126+
An entry encrypts one value, and has one query method for each operator the engine answers on the field: `Eq`, `Lt`, `Lte`, `Gt`, `Gte`, `Between` or `Matches`.
127+
stashgen asks the embedded engine which operators each field answers, and which index answers each one, so it holds no copy of those rules: an equality index answers `Eq`, a match index on a string `Matches`, and an ORE or OPE index on an integer the four orderings and `Between`, through ORE when the field declares both.
128+
`encrypt_into=TextEq` answers `Eq`, which returns the type's query value, `eql.TextEqQuery`.
129+
`encrypt_into=TextMatch` answers `Matches`, which returns an `encrypt.EQLMatchQuery`: the `eql.TextMatchQuery` operand, with `Verify` and `TargetLimit`.
130+
The query of `Matches` carries `Verify` and `TargetLimit`, because its answer can hold rows that do not match; an exact operator's query carries neither.
131+
See the encrypt package's README for each method's result and the query with `database/sql`.
127132
- `String` and `LogValue` on `EncryptedUser`, which print the passthrough fields and hide the sealed ones.
128133
- A copy of the fields of `User`, so a change to them stops the build.
129134

‎languages/golang/encrypt/README.md‎

Lines changed: 91 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -60,11 +60,13 @@ The package reference is on [pkg.go.dev]; the generator's reference is
6060

6161
encrypted, err := users.Encrypt(ctx, cipher, people)
6262
opened, err := users.Decrypt(ctx, cipher, encrypted)
63-
query, err := users.Fields.Email.Query(ctx, cipher, "bob@example.com")
63+
term, err := users.Fields.Email.Eq(ctx, cipher, "bob@example.com")
6464
```
6565

6666
`Encrypt` and `Decrypt` send one ZeroKMS request for each 500 sealed
6767
values in the batch, plus one request the first time a keyset is used.
68+
A query derives its term locally and sends no request; see
69+
[Queries](#queries).
6870

6971
6. Store the encrypted type. Each `encrypt_into` field is one EQL v4 column:
7072
`eql.TextEq` implements `driver.Valuer` and `sql.Scanner`, so a database
@@ -211,7 +213,7 @@ the value is one of those. A match *query* of such text is refused with
211213
`TextMatch` field is stricter, as "EQL columns" below says.
212214
213215
Terms are byte-equal to the ones the Rust crate derives, so a term from
214-
`users.Fields.Email.Equality` compares against a stored term written from any
216+
`users.Fields.Email.Eq` compares against a stored term written from any
215217
language. `EqualityTerm.Equal` compares in constant time; `OreTerm.Compare`
216218
and `OpeTerm.Compare` order as the plaintexts; `MatchTerm.Positions` decodes
217219
the token positions.
@@ -224,8 +226,88 @@ A database can compare some terms by itself:
224226
- `Ore`: do not compare in SQL. Plain byte order is almost always right, and
225227
sometimes wrong, with no error. Compare in Go with `OreTerm.Compare`, or
226228
use an EQL type (`encrypt_into=`), which the database compares correctly.
227-
- `Match`: do not compare in SQL. Decode the positions in Go with
228-
`MatchTerm.Positions`, or use an EQL type.
229+
- `Match`: a row matches when its stored filter holds every position of the
230+
query's term. Decode both into positions in SQL and compare with `@>`, as
231+
[Queries](#queries) shows, or in Go with `MatchTerm.Positions`.
232+
233+
## Queries
234+
235+
Each sealed field's entry in `Fields` has one method for each operator the
236+
engine answers on the field: `Eq`, `Lt`, `Lte`, `Gt`, `Gte`, `Between` and
237+
`Matches`. A field has no method for an operator it does not answer, so such
238+
a query does not compile. The generator asks the engine which operators each
239+
field answers, and which index answers each one; it holds no copy of those
240+
rules.
241+
242+
| The field declares | Methods | Each returns |
243+
|---|---|---|
244+
| `index=equality` | `Eq` | `encrypt.EqualityTerm` |
245+
| `index=ore` or `index=ope`, on an integer | `Lt`, `Lte`, `Gt`, `Gte` | `encrypt.OreTerm`, or `encrypt.OpeTerm` when the field declares no `ore` |
246+
| the same | `Between(ctx, cipher, lo, hi)` | `encrypt.Range[encrypt.OreTerm]` or `encrypt.Range[encrypt.OpeTerm]`, the two bounds' terms |
247+
| `index=match`, on a string | `Matches` | `encrypt.MatchQuery[T]` |
248+
| `encrypt_into=TextEq` | `Eq` | `eql.TextEqQuery` |
249+
250+
A method takes the field's own Go type, so `Fields.Age.Lt` on a `uint32`
251+
field takes a `uint32`. An order index on text, bytes or a float stores its
252+
term but answers no ordering operator yet, so such a field has no `Lt`.
253+
254+
Every query type implements `driver.Valuer`, so it binds as a parameter in
255+
any library built on `database/sql` or pgx. A `Range` binds as two
256+
parameters, `Low` and `High`.
257+
258+
`Eq`, the orderings and `Between` are exact: the server's answer is the
259+
answer. `Matches` is probabilistic: the server answers with every row whose
260+
stored filter holds the query's bits, which can include rows that do not
261+
hold the query's text. So `MatchQuery` carries two more methods, and an
262+
exact query's type carries neither (on an `encrypt_into=TextMatch` field,
263+
`Matches` returns an `encrypt.EQLMatchQuery`, the `eql.TextMatchQuery`
264+
operand with the same two):
265+
266+
- `TargetLimit(ctx, want, stats)` is how many rows to fetch for `want` of
267+
them to verify. A field declared with tags states no expected number of
268+
tokens, so with no measured histogram of the stored filters in
269+
`encrypt.Stats` the limit is unbounded, and `Limit.FetchOr(page)` fetches
270+
a page.
271+
- `Verify(ctx, values)` says, for each decrypted value of a page, whether it
272+
really holds the query's text: one call into the engine for the page, with
273+
no key.
274+
275+
```go
276+
// Positions of a stored filter or a match term, for @>.
277+
const positions = `CREATE FUNCTION match_positions(term bytea) RETURNS int[]
278+
LANGUAGE sql IMMUTABLE STRICT AS $$
279+
SELECT coalesce(array_agg(get_byte(term, 2 * i) | (get_byte(term, 2 * i + 1) << 8)), '{}')
280+
FROM generate_series(0, length(term) / 2 - 1) AS i
281+
$$`
282+
283+
q, err := users.Fields.Email.Matches(ctx, cipher, "alice")
284+
limit, err := q.TargetLimit(ctx, 10, encrypt.Stats{Rows: rowCount})
285+
286+
rows, err := db.QueryContext(ctx, `
287+
SELECT id, email, email_eq, email_match, age, age_eq, age_ore FROM users
288+
WHERE match_positions(email_match) @> match_positions($1)
289+
ORDER BY id LIMIT $2`, q, limit.FetchOr(50))
290+
// scan each row into a users.Encrypted, then
291+
people, err := users.Decrypt(ctx, cipher, encrypted)
292+
emails := make([]string, len(people))
293+
for i, p := range people {
294+
emails[i] = p.Email
295+
}
296+
verified, err := q.Verify(ctx, emails)
297+
hits := people[:0]
298+
for i, p := range people {
299+
if verified[i] {
300+
hits = append(hits, p)
301+
}
302+
}
303+
```
304+
305+
`Verify` cuts the query's text into tokens again in the engine, with the
306+
field's own tokenizer, so a `MatchQuery` keeps that text for as long as the
307+
program holds the query. Go cannot wipe a string: the text stays in memory
308+
until the garbage collector reclaims it, as the string you passed in does.
309+
A `MatchQuery` never prints it; `fmt`, `slog` and `%#v` show the field and
310+
the size of the term only.
229311

230312
## Errors
231313

@@ -318,8 +400,11 @@ the catalog (ordering, dates, decimals, JSON) is listed in
318400
with that reason. The guest builds the value: the generated code
319401
imports `encrypt/eql`, which embeds the build of the engine that holds the
320402
EQL types and registers it on import, so the program runs that build and
321-
stores what it returns. `Fields.Email.Query` returns the `eql.TextEqQuery`
322-
the column's `eql_v4.query_text_eq` operand takes. `mise run
403+
stores what it returns. `Fields.Email.Eq` returns the `eql.TextEqQuery`
404+
the column's `eql_v4.query_text_eq` operand takes; `TextEq` answers `eq`
405+
alone, and a `TextMatch` field's `Matches` returns an
406+
`encrypt.EQLMatchQuery`: the `eql.TextMatchQuery` operand with `Verify` and
407+
`TargetLimit`. `mise run
323408
wasm:guest:build:eql` builds that guest beside the other. A cipher extended
324409
with a tenant part refuses a struct with an `encrypt_into` field: an EQL
325410
value is stored under a table and a column, and the extended label has no

‎languages/golang/encrypt/doc.go‎

Lines changed: 43 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,10 +40,11 @@
4040
// cipher := client.Keyset(encrypt.KeysetName("tenant-42"))
4141
// encrypted, err := users.Encrypt(ctx, cipher, people) // []users.EncryptedUser
4242
// opened, err := users.Decrypt(ctx, cipher, encrypted) // []users.User
43-
// query, err := users.Fields.Email.Query(ctx, cipher, "bob@example.com")
43+
// term, err := users.Fields.Email.Eq(ctx, cipher, "bob@example.com")
4444
//
4545
// Encrypt and Decrypt send one ZeroKMS request for each 500 sealed values in
46-
// the batch, plus one the first time a keyset is used.
46+
// the batch, plus one the first time a keyset is used. A query derives its
47+
// term with no request; see Queries below.
4748
//
4849
// 6. Store the encrypted type. Each encrypt_into field is one EQL v4 column
4950
// (an eql.TextEq goes in a public.eql_v4_text_eq column), implementing
@@ -124,6 +125,46 @@
124125
// refused with [ErrTerm], because an empty filter as a query would match every
125126
// row.
126127
//
128+
// # Queries
129+
//
130+
// A generated Fields entry has one method for each operator the engine
131+
// answers on the field, named for it: Eq, Lt, Lte, Gt, Gte, Between and
132+
// Matches. A field has no method for an operator it does not answer, so
133+
// such a query does not compile; the generator asks the engine which
134+
// operators each field answers, and holds no copy of the rules. Each method
135+
// takes the field's own Go type.
136+
//
137+
// Eq returns an [EqualityTerm] (or, for a field with encrypt_into, the EQL
138+
// type's query value), an ordering an [OreTerm] or [OpeTerm], and Between a
139+
// [Range] of two. Their answers are exact. Matches returns a [MatchQuery],
140+
// whose answer can hold rows that do not match: size the fetch with
141+
// [MatchQuery.TargetLimit], then decrypt the rows and keep those
142+
// [MatchQuery.Verify] reports true, one call into the engine for the page.
143+
// On a field with encrypt_into=TextMatch, Matches returns an
144+
// [EQLMatchQuery], the type's query value with the same two methods.
145+
// Every query type implements driver.Valuer, so it binds as a parameter
146+
// through database/sql or pgx:
147+
//
148+
// q, err := users.Fields.Email.Matches(ctx, cipher, "alice")
149+
// limit, err := q.TargetLimit(ctx, 10, encrypt.Stats{Rows: rowCount})
150+
// rows, err := db.QueryContext(ctx, `SELECT id, email, email_eq, email_match, age, age_eq, age_ore
151+
// FROM users WHERE match_positions(email_match) @> match_positions($1)
152+
// ORDER BY id LIMIT $2`, q, limit.FetchOr(50))
153+
// // scan into []users.EncryptedUser, then
154+
// people, err := users.Decrypt(ctx, cipher, encrypted)
155+
// emails := make([]string, len(people))
156+
// for i, p := range people {
157+
// emails[i] = p.Email
158+
// }
159+
// verified, err := q.Verify(ctx, emails) // one bool per row
160+
//
161+
// match_positions is a SQL function of your own that reads a match term's
162+
// little-endian uint16 positions into an int[]; the package README has one.
163+
// A MatchQuery keeps the text it was derived from, because Verify cuts its
164+
// tokens again in the engine. Go cannot wipe a string, so the text stays in
165+
// memory until the garbage collector reclaims it; a MatchQuery never prints
166+
// it.
167+
//
127168
// # Transport and auth
128169
//
129170
// The guest imports exactly two host functions: an HTTP send, served by any

‎languages/golang/encrypt/eql/eql.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
// the type's domain, and the query types that match them.
44
//
55
// A field with `stash:"email,encrypt_into=TextEq"` is an [TextEq] in the
6-
// generated encrypted type, and its Fields entry's Query returns a
6+
// generated encrypted type, and its Fields entry's Eq returns a
77
// [TextEqQuery]. The guest builds every value: generated code stores what
88
// the guest returns and never assembles an EQL value itself (ADR-0007,
99
// amended 2026-10-06). Each type implements driver.Valuer and sql.Scanner

‎languages/golang/encrypt/example/README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
A runnable tour of the Go SDK against real ZeroKMS: declare a struct with
44
`stash` tags, generate its encrypted type, seal a batch in one request, derive
5-
a query term, and open the batch again.
5+
a query term, open the batch again, and verify a matches query over it.
66

77
## Running it
88

@@ -28,5 +28,5 @@ until they are built.
2828
- `model.go`: the struct and its `go:generate` line.
2929
- `user_stash.go`: what `stashgen` wrote from the tags. Regenerate it with
3030
`go generate ./...`; CI fails when that changes a committed file.
31-
- `main.go`: `Encrypt`, `Fields.Email.Equality` and `Decrypt`, the only three
32-
calls a program makes.
31+
- `main.go`: `Encrypt`, `Fields.Email.Eq`, `Decrypt`, and `Fields.Email.Matches`
32+
with `Verify` on the decrypted rows.

‎languages/golang/encrypt/guest/src/ops.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -409,7 +409,7 @@ pub fn targets() -> Result<Vec<u8>, u32> {
409409
/// The operators one field of a codec-encoded plan answers, as a
410410
/// codec-encoded list: one object per operator, in the order the engine
411411
/// lists them, each written by
412-
/// [`dynamic::operator_to_value`](stack_encrypt::dynamic::operator_to_value):
412+
/// [`dynamic::operator_to_value`]:
413413
///
414414
/// ```text
415415
/// [ { "operator": "eq", "fidelity": "exact", "arity": "one", "index": "eq" }, ... ]

‎packages/stack-encrypt/docs/adr/0007-bindings-enter-through-a-plan-never-a-second-executor.md‎

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -292,10 +292,20 @@ from Go and the typed query chain derive the same bytes. A cross-language
292292
fixture holds them together (`tests/fixtures/query_operators.json`): the Rust
293293
side checks each recorded term against the typed chain, and the Go SDK's
294294
tests derive, verify and size the same queries through the guest.
295-
`se_query`, which derives a target field's twin with no operator, stays until
296-
the generated Go code names operators (#1189), and is removed then.
297-
`se_term`, a term by index under an explicit context, stays for the generated
298-
code of fields declared with indexes until that code names operators too.
295+
The catalog answers for an EQL type; a field declared with indexes has no
296+
type to look up, so `se_operators` answers for any field of a plan: the
297+
plan and the field's name in, the same operator objects out, computed by the
298+
one table `se_query_term` reads (`dynamic::field_operators`). The Go
299+
generator asks it for every sealed field, `encrypt_into` fields included,
300+
and writes one method per operator (#1189).
301+
302+
`se_query`, which derived a target field's twin with no operator, is
303+
removed: the generated Go code names operators (#1189), and `eq` on a target
304+
field is the twin. `se_term`, a term by index under an explicit context, no
305+
longer serves a query. It stays for one host need: a record call's term
306+
failure does not say which row and field it came from, and the Go SDK asks
307+
`se_term` again for each term to name them. When the record exports name the
308+
field themselves, it goes too.
299309

300310
A data plan's match index declares no `expected_tokens` prior, so
301311
`target_limit` on a field declared as data is unbounded without a measured

0 commit comments

Comments
 (0)