Skip to content

Commit 7741dde

Browse files
committed
feat(eql-bindings, golang): produce the ordered date, timestamp and numeric types
With the engine ordering dates, timestamps and decimals, the nine EQL v4 types it refused become producible: DateOrd, DateOrdOpe, DateOrdOre and their timestamp and numeric twins. eql-codegen lists them in ENCRYPTION_DOMAINS and loses the pending-order reason; the generated bindings, the target table and Go's encrypt/eql table follow. eql-bindings' `v4::plaintext` now re-exports Stack Encrypt's `value` types, and its chrono and rust_decimal features turn on Stack Encrypt's. Its one remaining direct use of vitaminc-aead, the null tag in the by-name dispatch, goes through `stack_encrypt::dynamic::frame`, so the dependency is dropped. Tests: the OPE types join v4_order's ordering checks, the block ORE types join v4_ore_block with a PostgreSQL comparison over dates, timestamps (ties within a microsecond, flooring before the epoch) and decimals (1.5 ties 1.50), and the between smoke test now ranges over every ordered family. Tests that used DateOrdOre as the unproducible example use Json. Go: stashgen's static engine and notices admit order indexes on decimals, dates and timestamps; the ValueKinds model takes the order indexes the value-kinds fixture now declares, and its test checks Lt against the stored terms, folded queries included. Closes #1236 Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
1 parent 5618ba5 commit 7741dde

38 files changed

Lines changed: 800 additions & 1036 deletions

‎.changeset/eql-v4-date-timestamp-numeric.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,4 @@
22
'@cipherstash/eql': minor
33
---
44

5-
**Stack Encrypt produces the EQL v4 date, timestamp and numeric types `Date`, `DateEq`, `Timestamp`, `TimestampEq`, `Numeric` and `NumericEq`** (`eql-bindings`, `stack-encrypt` feature). Their Rust plaintexts are the crate's own `v4::plaintext::Date`, `Timestamp` and `Decimal`, which hold the integers the engine stores (days since 1970-01-01; Unix seconds and nanoseconds; a mantissa and a scale), are range checked when built and are wiped when dropped. The `chrono` and `rust_decimal` features convert to and from those crates' types. `=` on `NumericEq` normalises the scale (`1.5 = 1.50`) and on `TimestampEq` compares to the microsecond, truncating (never rounding) the nanoseconds below it; each value decrypts as it was stored. The ordered types of these families stay unproducible until the engine derives their order terms. The Go SDK's `encrypt/eql` table marks the six producible, from `encrypt.Date`, `time.Time` and `encrypt.Decimal`.
5+
**Stack Encrypt produces the EQL v4 date, timestamp and numeric types `Date`, `DateEq`, `Timestamp`, `TimestampEq`, `Numeric` and `NumericEq`** (`eql-bindings`, `stack-encrypt` feature). Their Rust plaintexts are `v4::plaintext::Date`, `Timestamp` and `Decimal`, which hold the integers the engine stores (days since 1970-01-01; Unix seconds and nanoseconds; a mantissa and a scale), are range checked when built and are wiped when dropped. The `chrono` and `rust_decimal` features convert to and from those crates' types. `=` on `NumericEq` normalises the scale (`1.5 = 1.50`) and on `TimestampEq` compares to the microsecond, truncating (never rounding) the nanoseconds below it; each value decrypts as it was stored. The Go SDK's `encrypt/eql` table marks the six producible, from `encrypt.Date`, `time.Time` and `encrypt.Decimal`.
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
'@cipherstash/eql': minor
3+
---
4+
5+
**Stack Encrypt produces the EQL v4 ordered date, timestamp and numeric types** (`eql-bindings`, `stack-encrypt` feature): `DateOrd`, `DateOrdOpe`, `DateOrdOre`, `TimestampOrd`, `TimestampOrdOpe`, `TimestampOrdOre`, `NumericOrd`, `NumericOrdOpe` and `NumericOrdOre`. A date orders by its day, a timestamp by the microsecond (truncated, as `TimestampEq`'s `=` is) and a decimal by value, so `1.5` and `1.50` order as equal. `v4::plaintext::Date`, `Timestamp`, `Decimal` and `RangeError` are now Stack Encrypt's own types, re-exported at the same path. The Go SDK's `encrypt/eql` table marks the nine producible.

‎languages/golang/CHANGELOG.md‎

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,14 @@ listed under **Breaking**. Releases are tagged `languages/golang/vX.Y.Z`.
1212

1313
### Added
1414

15+
- **Decimals, dates and timestamps take an order index (#1236).** An
16+
`encrypt.Decimal`, `encrypt.Date` or `time.Time` field takes `ore` and
17+
`ope`, and the generated code gives it `Lt`, `Lte`, `Gt`, `Gte` and
18+
`Between`. A decimal orders by value (`1.5` ties `1.50`), a date by its
19+
day, and a timestamp by the microsecond, truncated as its equality term
20+
is. The EQL types `DateOrd`, `DateOrdOpe`, `DateOrdOre`, `TimestampOrd`,
21+
`TimestampOrdOpe`, `TimestampOrdOre`, `NumericOrd`, `NumericOrdOpe` and
22+
`NumericOrdOre` are producible with `encrypt_into=`.
1523
- **An optional field: a sealed field declared over a pointer (#1298).**
1624
`DOB *encrypt.Date` with `stash:"dob,encrypt_into=Date"` generates (it
1725
was refused as a composite, #1293), and `Decrypt` gives `nil` back for a
@@ -85,8 +93,7 @@ listed under **Breaking**. Releases are tagged `languages/golang/vX.Y.Z`.
8593
`NumericEq` normalises the scale (`1.5 = 1.50`) and `TimestampEq` compares
8694
to the microsecond (truncated, never rounded), while each value opens as it was sealed. A
8795
`time.Time` seals its instant and opens in UTC. Their `Ord`, `OrdOpe` and
88-
`OrdOre` types stay unproducible until the engine derives order terms
89-
for these kinds (#1236).
96+
`OrdOre` types follow (see above).
9097
- **Every integer width, 128-bit integers, dates, timestamps and decimals
9198
as sealed fields** (#1144). Each integer type seals at its own width
9299
(`int8` as an int8, `uint16` as a uint16), so `record.Kind` and
@@ -96,8 +103,8 @@ listed under **Breaking**. Releases are tagged `languages/golang/vX.Y.Z`.
96103
`Decimal` are vitaminc's `vcvalue` types, with `encrypt.DateOf`,
97104
`ParseInt128`, `ParseUint128` and `ParseDecimal` (which refuses NaN and
98105
the infinities with `ErrNonFiniteDecimal`). Equality applies to all of
99-
them, ORE and OPE to every integer width but 8 bits. stashgen refuses an
100-
order index on a decimal, date or timestamp, naming #1236. The value-kinds fixture
106+
them, ORE and OPE to every integer width but 8 bits, and to decimals,
107+
dates and timestamps (see above). The value-kinds fixture
101108
test opens a record the Rust lowering sealed and derives the same terms.
102109
- **Any built-in Go integer field takes an integer EQL type** (ADR-0002):
103110
an `int16` or a `uint16` into `Integer`, an `int` into `Bigint`. The

‎languages/golang/cmd/stashgen/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -148,7 +148,7 @@ For `-type User`, the file `user_stash.go` holds:
148148
- `Fields`, with one entry for each sealed field.
149149
An opaque struct, or a struct with no sealed field outside an opaque one, gets no `Fields`: nothing in it is sealed on its own, so nothing can be queried.
150150
An entry encrypts one value, and has one query method for each operator the engine answers on the field: `Eq`, `Lt`, `Lte`, `Gt`, `Gte`, `Between` or `Matches`.
151-
stashgen asks the embedded engine which operators each field answers, and which index answers each one, so it holds no copy of those rules: an equality index answers `Eq`, a match index on a string `Matches`, and an ORE or OPE index on an integer, a float or a string the four orderings and `Between`, through ORE when the field declares both. A float orders as PostgreSQL's `float8` does (`-0.0` ties `0.0`, NaN sorts last), and a string by its folded form: case and accents folded, under Unicode 16.
151+
stashgen asks the embedded engine which operators each field answers, and which index answers each one, so it holds no copy of those rules: an equality index answers `Eq`, a match index on a string `Matches`, and an ORE or OPE index on an integer of 16 bits or more, a float, a string, an `encrypt.Decimal`, an `encrypt.Date` or a `time.Time` the four orderings and `Between`, through ORE when the field declares both. A float orders as PostgreSQL's `float8` does (`-0.0` ties `0.0`, NaN sorts last), and a string by its folded form: case and accents folded, under Unicode 16.
152152
`encrypt_into=TextEq` answers `Eq`, which returns the type's query value, `eql.TextEqQuery`.
153153
`encrypt_into=TextMatch` answers `Matches`, which returns an `encrypt.EQLMatchQuery`: the `eql.TextMatchQuery` operand, with `Verify` and `TargetLimit`.
154154
The query of `Matches` carries `Verify` and `TargetLimit`, because its answer can hold rows that do not match; an exact operator's query carries neither.
@@ -276,7 +276,7 @@ An `opaque` struct crosses as one JSON document and is one column, decoded back
276276
A float in an opaque struct must be finite: `encoding/json` refuses NaN and the infinities, so `Encrypt` fails for the batch with `encrypt.ErrEncoding`. A sealed float field outside an opaque struct carries them.
277277
A sealed field outside an opaque struct is one scalar, a type defined over one, or a pointer to either (see "Optional fields"); a struct, slice or map field seals only inside an opaque struct.
278278
The scalars are `string`, `[]byte`, `bool`, every integer type (each sealed at its own width, `int` and `uint` as 64-bit), `encrypt.Int128` and `encrypt.Uint128`, `float32`, `float64`, `encrypt.Decimal`, `encrypt.Date` and `time.Time` (a timestamp). The wire type `stashgen` writes for each is the field's own: `int8` is `gensupport.Int8`, `time.Time` is `gensupport.Timestamp`, and so on.
279-
An order index (`ore`, `ope`) on a decimal, date or timestamp is refused: those order terms are not supported yet (#1236).
279+
A decimal orders by value, so `1.5` ties `1.50`; a `time.Time` orders by the microsecond, so instants within one microsecond tie.
280280
The zero `encrypt.Date` is not a date: a date field must be set, and `Encrypt` refuses a zero one with `encrypt.ErrEncoding`, naming the field. An optional date is a `*encrypt.Date` field (see "Optional fields").
281281
A nil `[]byte` in a sealed field, or a nil slice of a type defined over `[]byte`, comes back from `Decrypt` as an empty, non-nil slice; inside an opaque struct, nil comes back as nil.
282282
A protobuf message with a `oneof` cannot be generated from a policy: protoc-gen-go puts its members in wrapper types, not in the message struct, and `Generate` refuses it.

‎languages/golang/cmd/stashgen/main_test.go‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -184,15 +184,15 @@ func TestRunAsksTheEmbeddedEngine(t *testing.T) {
184184
t.Fatalf("the generated file does not name the EQL type:\n%s", written)
185185
}
186186
// A type the engine cannot produce yet: refused by name, nothing written.
187-
unproducible := strings.Replace(userSource, "encrypt_into=TextEq", "encrypt_into=DateOrdOre", 1)
187+
unproducible := strings.Replace(userSource, "encrypt_into=TextEq", "encrypt_into=Json", 1)
188188
dir = writeModule(t, map[string]string{"model.go": unproducible})
189189
stderr.Reset()
190190
if code := run([]string{"-type", "User"}, dir, &stdout, &stderr, stashgen.GuestEngine); code != 1 {
191191
t.Fatalf("exit %d, want 1\n%s", code, stderr.String())
192192
}
193193
// Refused by the generator with the engine's reason, before the engine
194194
// is asked: se_targets lists the type as not producible.
195-
if !strings.Contains(stderr.String(), "cannot produce the EQL type DateOrdOre yet") {
195+
if !strings.Contains(stderr.String(), "cannot produce the EQL type Json yet") {
196196
t.Fatalf("stderr = %q", stderr.String())
197197
}
198198
if _, err := os.Stat(filepath.Join(dir, "user_stash.go")); !os.IsNotExist(err) {

‎languages/golang/encrypt/doc.go‎

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -123,10 +123,10 @@
123123
// or [Int128] and [Uint128]), a float32 or float64, a [Decimal], a [Date] or a
124124
// time.Time (a timestamp), or a type defined over one of the built-in types.
125125
// Each integer seals at its own width. Equality applies to every one but a
126-
// bool. ORE and OPE apply to integers of 16 bits or more, floats, text and
127-
// bytes. A bool or an 8-bit integer takes no order index, since over so few
128-
// values its order terms would show which rows hold which value. Decimals,
129-
// dates and timestamps cannot take an order index yet.
126+
// bool. ORE and OPE apply to integers of 16 bits or more, floats, text,
127+
// bytes, decimals, dates and timestamps. A bool or an 8-bit integer takes no
128+
// order index, since over so few values its order terms would show which
129+
// rows hold which value.
130130
//
131131
// Passthrough fields never cross the binding: the engine does nothing to a
132132
// passthrough value that a program could observe, and the FFI codec cannot
@@ -164,9 +164,10 @@
164164
// type's query value), an ordering an [OreTerm] or [OpeTerm], and Between a
165165
// [Range] of two. Their answers are exact. The orderings are answered on
166166
// integers, on floats, which order as PostgreSQL's float8 does (-0.0 ties
167-
// 0.0, NaN sorts last), and on text, which orders by its folded form (case
168-
// and accents folded, under Unicode 16), so Café and cafe tie and alice
169-
// orders before Zoe. Matches returns a [MatchQuery],
167+
// 0.0, NaN sorts last), on text, which orders by its folded form (case and
168+
// accents folded, under Unicode 16), so Café and cafe tie and alice orders
169+
// before Zoe, and on a [Decimal], which orders by value (1.5 ties 1.50), a
170+
// [Date], and a time.Time, which orders by the microsecond. Matches returns a [MatchQuery],
170171
// whose answer can hold rows that do not match: size the fetch with
171172
// [MatchQuery.TargetLimit], then decrypt the rows and keep those
172173
// [MatchQuery.Verify] reports true, one call into the engine for the page.

0 commit comments

Comments
 (0)