Skip to content

Commit 8bb6585

Browse files
committed
docs(golang): show the query methods, the database/sql query and the decrypt-and-verify step, and say se_query is gone
The README and the package documentation describe one method per operator, what each returns, why a matches query carries Verify and TargetLimit and an exact one neither, and that a MatchQuery holds its text because Go cannot wipe a string. The ADR-0007 amendment records se_operators, the removal of se_query, and the one reason se_term stays. Part of #1189
1 parent 20bd104 commit 8bb6585

7 files changed

Lines changed: 153 additions & 19 deletions

File tree

‎languages/golang/cmd/stashgen/README.md‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ Your program calls those functions, and it never builds or names a plan.
4040
```go
4141
encrypted, err := users.Encrypt(ctx, cipher, people)
4242
opened, err := users.Decrypt(ctx, cipher, encrypted)
43-
query, err := users.Fields.Email.Query(ctx, cipher, "bob@example.com")
43+
query, err := users.Fields.Email.Eq(ctx, cipher, "bob@example.com")
4444
```
4545

4646
6. Store the encrypted type.
@@ -123,7 +123,11 @@ For `-type User`, the file `user_stash.go` holds:
123123
- `Encrypt` and `Decrypt`, which take a slice and return a slice, and send one ZeroKMS request for each 500 sealed values in it, plus one the first time a keyset is used.
124124
- `Fields`, with one entry for each sealed field.
125125
An opaque struct, or a struct with no sealed field outside an opaque one, gets no `Fields`: nothing in it is sealed on its own, so nothing can be queried.
126-
An entry encrypts one value, and it has a query method only for what the field declares: `Query` for `encrypt_into`, and `Equality`, `Match`, `Ore` or `Ope` for `index=`.
126+
An entry encrypts one value, and has one query method for each operator the engine answers on the field: `Eq`, `Lt`, `Lte`, `Gt`, `Gte`, `Between` or `Matches`.
127+
stashgen asks the embedded engine which operators each field answers, and which index answers each one, so it holds no copy of those rules: an equality index answers `Eq`, a match index on a string `Matches`, and an ORE or OPE index on an integer the four orderings and `Between`, through ORE when the field declares both.
128+
`encrypt_into=TextEq` answers `Eq`, which returns the type's query value, `eql.TextEqQuery`.
129+
The query of `Matches` carries `Verify` and `TargetLimit`, because its answer can hold rows that do not match; an exact operator's query carries neither.
130+
See the encrypt package's README for each method's result and the query with `database/sql`.
127131
- `String` and `LogValue` on `EncryptedUser`, which print the passthrough fields and hide the sealed ones.
128132
- A copy of the fields of `User`, so a change to them stops the build.
129133

‎languages/golang/encrypt/README.md‎

Lines changed: 87 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -56,11 +56,13 @@ The package reference is on [pkg.go.dev]; the generator's reference is
5656

5757
encrypted, err := users.Encrypt(ctx, cipher, people)
5858
opened, err := users.Decrypt(ctx, cipher, encrypted)
59-
term, err := users.Fields.Email.Equality(ctx, cipher, "bob@example.com")
59+
term, err := users.Fields.Email.Eq(ctx, cipher, "bob@example.com")
6060
```
6161

6262
`Encrypt` and `Decrypt` send one ZeroKMS request for each 500 sealed
6363
values in the batch, plus one request the first time a keyset is used.
64+
A query derives its term locally and sends no request; see
65+
[Queries](#queries).
6466

6567
6. Store the encrypted type. Each sealed field is one or more byte columns:
6668
`e.Email.Ciphertext`, `e.Email.Equality`, `e.Email.Match`. `Ciphertext`
@@ -198,7 +200,7 @@ field and the index. So an optional or short value does not belong under
198200
`match`: give such a field `equality` alone, or make the value required.
199201
200202
Terms are byte-equal to the ones the Rust crate derives, so a term from
201-
`users.Fields.Email.Equality` compares against a stored term written from any
203+
`users.Fields.Email.Eq` compares against a stored term written from any
202204
language. `EqualityTerm.Equal` compares in constant time; `OreTerm.Compare`
203205
and `OpeTerm.Compare` order as the plaintexts; `MatchTerm.Positions` decodes
204206
the token positions.
@@ -211,8 +213,86 @@ A database can compare some terms by itself:
211213
- `Ore`: do not compare in SQL. Plain byte order is almost always right, and
212214
sometimes wrong, with no error. Compare in Go with `OreTerm.Compare`, or
213215
use an EQL type (`encrypt_into=`), which the database compares correctly.
214-
- `Match`: do not compare in SQL. Decode the positions in Go with
215-
`MatchTerm.Positions`, or use an EQL type.
216+
- `Match`: a row matches when its stored filter holds every position of the
217+
query's term. Decode both into positions in SQL and compare with `@>`, as
218+
[Queries](#queries) shows, or in Go with `MatchTerm.Positions`.
219+
220+
## Queries
221+
222+
Each sealed field's entry in `Fields` has one method for each operator the
223+
engine answers on the field: `Eq`, `Lt`, `Lte`, `Gt`, `Gte`, `Between` and
224+
`Matches`. A field has no method for an operator it does not answer, so such
225+
a query does not compile. The generator asks the engine which operators each
226+
field answers, and which index answers each one; it holds no copy of those
227+
rules.
228+
229+
| The field declares | Methods | Each returns |
230+
|---|---|---|
231+
| `index=equality` | `Eq` | `encrypt.EqualityTerm` |
232+
| `index=ore` or `index=ope`, on an integer | `Lt`, `Lte`, `Gt`, `Gte` | `encrypt.OreTerm`, or `encrypt.OpeTerm` when the field declares no `ore` |
233+
| the same | `Between(ctx, cipher, lo, hi)` | `encrypt.Range[encrypt.OreTerm]` or `encrypt.Range[encrypt.OpeTerm]`, the two bounds' terms |
234+
| `index=match`, on a string | `Matches` | `encrypt.MatchQuery[T]` |
235+
| `encrypt_into=TextEq` | `Eq` | `eql.TextEqQuery` |
236+
237+
A method takes the field's own Go type, so `Fields.Age.Lt` on a `uint32`
238+
field takes a `uint32`. An order index on text, bytes or a float stores its
239+
term but answers no ordering operator yet, so such a field has no `Lt`.
240+
241+
Every query type implements `driver.Valuer`, so it binds as a parameter in
242+
any library built on `database/sql` or pgx. A `Range` binds as two
243+
parameters, `Low` and `High`.
244+
245+
`Eq`, the orderings and `Between` are exact: the server's answer is the
246+
answer. `Matches` is probabilistic: the server answers with every row whose
247+
stored filter holds the query's bits, which can include rows that do not
248+
hold the query's text. So `MatchQuery` carries two more methods, and an
249+
exact query's type carries neither:
250+
251+
- `TargetLimit(ctx, want, stats)` is how many rows to fetch for `want` of
252+
them to verify. A field declared with tags states no expected number of
253+
tokens, so with no measured histogram of the stored filters in
254+
`encrypt.Stats` the limit is unbounded, and `Limit.FetchOr(page)` fetches
255+
a page.
256+
- `Verify(ctx, values)` says, for each decrypted value of a page, whether it
257+
really holds the query's text: one call into the engine for the page, with
258+
no key.
259+
260+
```go
261+
// Positions of a stored filter or a match term, for @>.
262+
const positions = `CREATE FUNCTION match_positions(term bytea) RETURNS int[]
263+
LANGUAGE sql IMMUTABLE STRICT AS $$
264+
SELECT coalesce(array_agg(get_byte(term, 2 * i) | (get_byte(term, 2 * i + 1) << 8)), '{}')
265+
FROM generate_series(0, length(term) / 2 - 1) AS i
266+
$$`
267+
268+
q, err := users.Fields.Email.Matches(ctx, cipher, "alice")
269+
limit, err := q.TargetLimit(ctx, 10, encrypt.Stats{Rows: rowCount})
270+
271+
rows, err := db.QueryContext(ctx, `
272+
SELECT id, email, email_eq, email_match, age, age_eq, age_ore FROM users
273+
WHERE match_positions(email_match) @> match_positions($1)
274+
ORDER BY id LIMIT $2`, q, limit.FetchOr(50))
275+
// scan each row into a users.Encrypted, then
276+
people, err := users.Decrypt(ctx, cipher, encrypted)
277+
emails := make([]string, len(people))
278+
for i, p := range people {
279+
emails[i] = p.Email
280+
}
281+
verified, err := q.Verify(ctx, emails)
282+
hits := people[:0]
283+
for i, p := range people {
284+
if verified[i] {
285+
hits = append(hits, p)
286+
}
287+
}
288+
```
289+
290+
`Verify` cuts the query's text into tokens again in the engine, with the
291+
field's own tokenizer, so a `MatchQuery` keeps that text for as long as the
292+
program holds the query. Go cannot wipe a string: the text stays in memory
293+
until the garbage collector reclaims it, as the string you passed in does.
294+
A `MatchQuery` never prints it; `fmt`, `slog` and `%#v` show the field and
295+
the size of the term only.
216296

217297
## Errors
218298

@@ -280,8 +360,9 @@ PostgreSQL holds in a `public.eql_v3_text_eq` column, as an `eql.TextEq`
280360
from package `encrypt/eql`. The guest builds the value: the generated code
281361
imports `encrypt/eql`, which embeds the build of the engine that holds the
282362
EQL types and registers it on import, so the program runs that build and
283-
stores what it returns. `Fields.Email.Query` returns the `eql.TextEqQuery`
284-
the column's `eql_v3.query_text_eq` operand takes. `mise run
363+
stores what it returns. `Fields.Email.Eq` returns the `eql.TextEqQuery`
364+
the column's `eql_v3.query_text_eq` operand takes; `TextEq` answers `eq`
365+
alone. `mise run
285366
wasm:guest:build:eql` builds that guest beside the other. A cipher extended
286367
with a tenant part refuses a struct with an `encrypt_into` field: an EQL
287368
value is stored under a table and a column, and the extended label has no

‎languages/golang/encrypt/doc.go‎

Lines changed: 41 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,10 +35,11 @@
3535
// cipher := client.Keyset(encrypt.KeysetName("tenant-42"))
3636
// encrypted, err := users.Encrypt(ctx, cipher, people) // []users.EncryptedUser
3737
// opened, err := users.Decrypt(ctx, cipher, encrypted) // []users.User
38-
// term, err := users.Fields.Email.Equality(ctx, cipher, "bob@example.com")
38+
// term, err := users.Fields.Email.Eq(ctx, cipher, "bob@example.com")
3939
//
4040
// Encrypt and Decrypt send one ZeroKMS request for each 500 sealed values in
41-
// the batch, plus one the first time a keyset is used.
41+
// the batch, plus one the first time a keyset is used. A query derives its
42+
// term with no request; see Queries below.
4243
//
4344
// 6. Store the encrypted type. Each sealed field is one or more columns of
4445
// bytes ([Ciphertext] and the term types), each implementing driver.Valuer
@@ -112,6 +113,44 @@
112113
// refused with [ErrTerm], because an empty filter as a query would match every
113114
// row.
114115
//
116+
// # Queries
117+
//
118+
// A generated Fields entry has one method for each operator the engine
119+
// answers on the field, named for it: Eq, Lt, Lte, Gt, Gte, Between and
120+
// Matches. A field has no method for an operator it does not answer, so
121+
// such a query does not compile; the generator asks the engine which
122+
// operators each field answers, and holds no copy of the rules. Each method
123+
// takes the field's own Go type.
124+
//
125+
// Eq returns an [EqualityTerm] (or, for a field with encrypt_into, the EQL
126+
// type's query value), an ordering an [OreTerm] or [OpeTerm], and Between a
127+
// [Range] of two. Their answers are exact. Matches returns a [MatchQuery],
128+
// whose answer can hold rows that do not match: size the fetch with
129+
// [MatchQuery.TargetLimit], then decrypt the rows and keep those
130+
// [MatchQuery.Verify] reports true, one call into the engine for the page.
131+
// Every query type implements driver.Valuer, so it binds as a parameter
132+
// through database/sql or pgx:
133+
//
134+
// q, err := users.Fields.Email.Matches(ctx, cipher, "alice")
135+
// limit, err := q.TargetLimit(ctx, 10, encrypt.Stats{Rows: rowCount})
136+
// rows, err := db.QueryContext(ctx, `SELECT id, email, email_eq, email_match, age, age_eq, age_ore
137+
// FROM users WHERE match_positions(email_match) @> match_positions($1)
138+
// ORDER BY id LIMIT $2`, q, limit.FetchOr(50))
139+
// // scan into []users.EncryptedUser, then
140+
// people, err := users.Decrypt(ctx, cipher, encrypted)
141+
// emails := make([]string, len(people))
142+
// for i, p := range people {
143+
// emails[i] = p.Email
144+
// }
145+
// verified, err := q.Verify(ctx, emails) // one bool per row
146+
//
147+
// match_positions is a SQL function of your own that reads a match term's
148+
// little-endian uint16 positions into an int[]; the package README has one.
149+
// A MatchQuery keeps the text it was derived from, because Verify cuts its
150+
// tokens again in the engine. Go cannot wipe a string, so the text stays in
151+
// memory until the garbage collector reclaims it; a MatchQuery never prints
152+
// it.
153+
//
115154
// # Transport and auth
116155
//
117156
// The guest imports exactly two host functions: an HTTP send, served by any

‎languages/golang/encrypt/eql/eql.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
// the type's domain, and the query types that match them.
44
//
55
// A field with `stash:"email,encrypt_into=TextEq"` is an [TextEq] in the
6-
// generated encrypted type, and its Fields entry's Query returns a
6+
// generated encrypted type, and its Fields entry's Eq returns a
77
// [TextEqQuery]. The guest builds every value: generated code stores what
88
// the guest returns and never assembles an EQL value itself (ADR-0007,
99
// amended 2026-10-06). Each type implements driver.Valuer and sql.Scanner

‎languages/golang/encrypt/example/README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
A runnable tour of the Go SDK against real ZeroKMS: declare a struct with
44
`stash` tags, generate its encrypted type, seal a batch in one request, derive
5-
a query term, and open the batch again.
5+
a query term, open the batch again, and verify a matches query over it.
66

77
## Running it
88

@@ -28,5 +28,5 @@ until they are built.
2828
- `model.go`: the struct and its `go:generate` line.
2929
- `user_stash.go`: what `stashgen` wrote from the tags. Regenerate it with
3030
`go generate ./...`; CI fails when that changes a committed file.
31-
- `main.go`: `Encrypt`, `Fields.Email.Equality` and `Decrypt`, the only three
32-
calls a program makes.
31+
- `main.go`: `Encrypt`, `Fields.Email.Eq`, `Decrypt`, and `Fields.Email.Matches`
32+
with `Verify` on the decrypted rows.

‎languages/golang/encrypt/guest/src/ops.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -408,7 +408,7 @@ pub fn targets() -> Result<Vec<u8>, u32> {
408408
/// The operators one field of a codec-encoded plan answers, as a
409409
/// codec-encoded list: one object per operator, in the order the engine
410410
/// lists them, each written by
411-
/// [`dynamic::operator_to_value`](stack_encrypt::dynamic::operator_to_value):
411+
/// [`dynamic::operator_to_value`]:
412412
///
413413
/// ```text
414414
/// [ { "operator": "eq", "fidelity": "exact", "arity": "one", "index": "eq" }, ... ]

‎packages/stack-encrypt/docs/adr/0007-bindings-enter-through-a-plan-never-a-second-executor.md‎

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -285,10 +285,20 @@ from Go and the typed query chain derive the same bytes. A cross-language
285285
fixture holds them together (`tests/fixtures/query_operators.json`): the Rust
286286
side checks each recorded term against the typed chain, and the Go SDK's
287287
tests derive, verify and size the same queries through the guest.
288-
`se_query`, which derives a target field's twin with no operator, stays until
289-
the generated Go code names operators (#1189), and is removed then.
290-
`se_term`, a term by index under an explicit context, stays for the generated
291-
code of fields declared with indexes until that code names operators too.
288+
The catalog answers for an EQL type; a field declared with indexes has no
289+
type to look up, so `se_operators` answers for any field of a plan: the
290+
plan and the field's name in, the same operator objects out, computed by the
291+
one table `se_query_term` reads (`dynamic::field_operators`). The Go
292+
generator asks it for every sealed field, `encrypt_into` fields included,
293+
and writes one method per operator (#1189).
294+
295+
`se_query`, which derived a target field's twin with no operator, is
296+
removed: the generated Go code names operators (#1189), and `eq` on a target
297+
field is the twin. `se_term`, a term by index under an explicit context, no
298+
longer serves a query. It stays for one host need: a record call's term
299+
failure does not say which row and field it came from, and the Go SDK asks
300+
`se_term` again for each term to name them. When the record exports name the
301+
field themselves, it goes too.
292302

293303
A data plan's match index declares no `expected_tokens` prior, so
294304
`target_limit` on a field declared as data is unbounded without a measured

0 commit comments

Comments
 (0)