fix(lens): 3.5.1 review fixes

CHANGELOG.md

@@ -1,5 +1,37 @@
 # Changelog
 
+## 3.5.1 — review fixes: bridged fetches, to-many paths, JSON re-checks
+
+Fail-closed and doc findings from the 3.5.0 adversarial review.
+
+- **`toLensSelect` selects a bridged source's local reads.** It skipped every column a source across
+  a bridge reads, so `materializeSources` over its rows plus the far side threw `UsageError: lacks
+  'age'` for `any: [age > 35, crm:Profile.tier = gold]` when the lens hides `age`. It now selects
+  what the source reads on this side, hidden or not, and each crossed bridge's local `on` key, as
+  fetch-only columns: `projectRows(…, { keepClampColumns: true })` keeps them and a viewer's
+  projection drops them.
+- **A `label` or `groupBy` across a to-many relation (local or bridged) is refused by every
+  posture.** `validateNarrowing` reported it (`invalid_source`), but `toSourceQueries` and
+  `materializeSources` still ran and returned unlabeled or ungrouped options. One check
+  (`toOnePathError`) now serves validation and the source plans, so the postures throw the same
+  `LensRefusal` (`invalid_source`) and `validateNarrowing` is ok exactly when none refuses. The
+  plans' separate unresolvable-hop and to-many-clamp refusals fold into it.
+- **A re-check reading into a Json column selects the column whole.** `toSourceQueries` emitted a
+  nested `select` into the Json column (`meta: { select: { a: … } }`), which the typed Prisma
+  client rejects. It selects `meta: true`; flat SQL rows hold that column, so the SQL query is no
+  longer refused for it. A read through a relation still is, and the message says why per read
+  (through a relation, or not a column of the model).
+- **`SourcePrismaQuery.steps` is present whenever the where needs `executePrismaPlan`**: a count
+  step, as before, or a column reference (`{ __field }`) to resolve. A column-reference source used to
+  ship an unresolved where that Prisma rejects.
+- **A count operator across a bridge says so**: "'crm:Event' crosses a bridge: count operators
+  aren't supported across a bridge", not "is not a relation … Count operators require a relation
+  field".
+- README: both source-query snippets destructure `query.prisma` and run `steps` through
+  `executePrismaPlan` (the bridge snippet called `prisma[query.model].findMany(query.prisma)`,
+  which Prisma rejects); a test runs that shape against the rails harness. `orderRecords` notes
+  that strings sort in JavaScript code-unit order, which can differ from a database collation.
+
 ## 3.5.0 — grant → clamp
 
 A lens `where` only ever narrows: a row is hidden unless it holds. "Grant" read as permission

README.md

@@ -1162,7 +1162,7 @@ Composition across chained narrowings is pure intersection: relations are turned
 | `lensVisit(lens, relationPath, options?)` | One visit as `projectLens` (by path) gives it — its shown fields with values and options, sources, labels and axes — at a dotted relation path from the anchor (`''` for the anchor), resolved on demand: nothing is enumerated, so it is cheap on any schema. `null` when a relation on the path isn't shown there (off, omitted, or outside the model-default tree). A builder walks a lens with it instead of re-deriving the lens's rules. |
 | `walkLensPath(lens, path)` | Resolves one dotted path through the lens hop by hop: `{ outcome: 'resolved', hops, terminal, jsonSubPath }`, or `hidden` (a column it doesn't keep, or a relation it doesn't turn on there) / `missing` / `pastScalar` with the failing `index`. |
 | `readLensValue(lens, row, path, options?)` | One value off a row, as the lens shows it: `{ ok: true, value }`, or `{ ok: false, reason }` — `hidden` / `missing` / `pastScalar` (the walk `validateRuleInLens` gates a field with), `relation` (the path ends on rows, not a value) or `list` (it crosses a to-many). Each row on the way, the root included, is checked against its visit's clamps: one a clamp hides, or a missing one, reads `null`. Only own properties are read, into a Json column too. `options` is what each clamp is checked with (`now`, `bindings`). For values a template interpolates. |
-| `orderRecords(items, orderBy)` | Records sorted by an `OrderBy` exactly as a window orders them on every rail: each key in turn, own-property path reads, `dir` order, NULL or absent last in either direction, ties in input order. Returns a new array. |
+| `orderRecords(items, orderBy)` | Records sorted by an `OrderBy` exactly as a window orders them on every rail: each key in turn, own-property path reads, `dir` order, NULL or absent last in either direction, ties in input order. Returns a new array. Strings sort in JavaScript code-unit order, which can differ from a database's collation (see [Where the Rails Differ](#where-the-rails-differ)). |
 | `narrowRule(rule, narrowing)` | Composes the user rule with the lens's `where` clauses, injecting each at its anchor in the rule tree. Under an `all`, the clamp goes into the rule's window `filter`, which `check()` evaluates and `toPrisma` folds into the rule (`toSql` compiles no relation arrays). Other rules pass to `check` / `toPrisma` / `toSql`. |
 | `coerceRule(rule, lens)` | Stamps each field rule with its field's `coerceType` from the lens (`Int`, `Float`, `Decimal`, `BigInt`, `DateTime`, `Boolean`, `String`). Leaves date rules, aggregate comparisons, rules that already carry a `coerceType`, and anything below a Json column alone. |
 
@@ -1209,6 +1209,7 @@ to each field as `options`.
 
 ```ts
 import {
+  executePrismaPlan,
   materializeSourceQuery,
   materializeSources,
   projectLens,
@@ -1230,8 +1231,13 @@ const [query] = toSourceQueries(narrowing);
 // query.prisma => { model: 'User', distinct: ['region'], select: { region: true }, where: { AND: [...] } }
 // query.sql => { sql: 'SELECT DISTINCT "t0"."region" FROM "User" AS "t0" WHERE (...)', params: ['t-42', true] }
 // A query with `recheck` (a source across a bridge) returns candidates; see "Sources across a bridge".
-const { distinct, select, where } = query.prisma;
-const rows = await prisma.user.findMany({ distinct, select, where });
+// `steps` (a count step, a column reference) resolve through executePrismaPlan first.
+const { distinct, select, where, steps } = query.prisma; // query.model === 'User'
+const rows = await prisma.user.findMany({
+  distinct,
+  select,
+  where: steps ? await executePrismaPlan({ steps }, prisma) : where,
+});
 const values = materializeSourceQuery(query, rows); // { path, mapName, model, field, options: [{ value }] }
 
 // Or from rows already fetched under the lens (relations inline):
@@ -1243,7 +1249,7 @@ const projection = projectLens(narrowing, { sourceValues: [values] });
 
 | Function | Purpose |
 | --- | --- |
-| `toSourceQueries(lensOrNarrowing, options?)` | `SourceQuery[]`, one per sourced field: `{ path, mapName, model, field, label?, groupBy?, composedWhere, prisma, sql, recheck? }`. `options` is the clock (`now`, `timeZone`, `weekStart`) a relative date in the where compiles with — required for one, a plain usage error without it; bind a lens's binds with `bindLens` first. `prisma.steps` is present when the where needs `executePrismaPlan`. `sql.sql` is `null` with an `error` when SQL can't express the where. A source that reads across a bridge gets an over-fetching query and a `recheck`: its rows are candidates, not options (see "Sources across a bridge"). `distinct` is the value and a sibling label column; a grouped source or a dotted label drops it, so every label comes back and the least one is picked. |
+| `toSourceQueries(lensOrNarrowing, options?)` | `SourceQuery[]`, one per sourced field: `{ path, mapName, model, field, label?, groupBy?, composedWhere, prisma, sql, recheck? }`. `options` is the clock (`now`, `timeZone`, `weekStart`) a relative date in the where compiles with — required for one, a plain usage error without it; bind a lens's binds with `bindLens` first. `prisma.steps` is present when the where needs `executePrismaPlan` (a count step, or a column reference to resolve): query with the where it returns. `sql.sql` is `null` with an `error` when SQL can't express the where. A source that reads across a bridge gets an over-fetching query and a `recheck`: its rows are candidates, not options (see "Sources across a bridge"). `distinct` is the value and a sibling label column; a grouped source or a dotted label drops it, so every label comes back and the least one is picked. |
 | `materializeSourceQuery(query, rows, { rowShape?, lens?, now?, … })` | One query's fetched rows as `SourceValues`. `rowShape` is `'prisma'` (default: a dotted `label` and each `groupBy` axis come nested) or `'sql'` (they come flat as `__label` / `__group_i`). Options are deduplicated and sorted. A query with `recheck` needs `lens` and rows holding the far side: it re-checks each candidate (`check`, with the clock and bindings given) and reads a bridged label or axis from the far side; a missing far side is a `UsageError`. |
 | `materializeSources(lensOrNarrowing, rows, options?)` | `SourceValues[]` for every sourced field, from the rows the lens fetches — `toLensSelect`'s rows as fetched, or as `projectRows(…, { keepClampColumns: true })` keeps them. A viewer's projection drops what sources read: a row lacking any key a read walks — through each relation and list element to the column — that a source or a clamp on its path reads throws a `UsageError` (a fetch returns every key it selects, NULL as `null`). The path is walked down the rows — the tree is its link — each level's clamps met, and each row it reaches must meet, through `check()` with `options`, its visit's clamps, its source `where` narrowed as a rule is, the guards of the relations its label and axes cross and the values the lens allows — so it offers what `toSourceQueries` does. A scalar-list field gives one option per element; a value takes its least label. A `from: 'mapDefaults'` source throws (see below). |
 
@@ -1334,12 +1340,19 @@ see, and marks it with `recheck`:
   clamps above it are carried back across the bridge through the far model's bridge field, and
   come back as its `recheck` — so each candidate holds the near rows inline
   (`{ id, industry, 'prisma:FanUser': [{ email, … }] }`).
+- **A re-check reading into a Json column selects the column whole** (`meta: true`, never a nested
+  select into it); the re-check reads inside it in memory.
 - **SQL rows are flat**, so a bridged query whose re-check reads through a local relation has
   `sql.sql: null` (with `sql.error`); run the Prisma form.
 
 ```ts
 const [query] = toSourceQueries(lens, { now });
-const candidates = await prisma[query.model].findMany(query.prisma); // superset
+const { distinct, select, where, steps } = query.prisma; // query.model === 'User'
+const candidates = await prisma.user.findMany({
+  distinct,
+  select,
+  where: steps ? await executePrismaPlan({ steps }, prisma) : where,
+}); // a superset when `recheck` is present
 const rows = query.recheck === undefined ? candidates : await loadFarSide(candidates); // your join
 const values = materializeSourceQuery(query, rows, { lens, now });
 ```
@@ -1365,7 +1378,7 @@ const holds = forRecheck.filter((row) => check(narrowRule(rule, narrowing), row,
 
 | Function | Purpose |
 | --- | --- |
-| `toLensSelect(lensOrNarrowing, options?)` | `{ select }` for `findMany` at the base model. It selects each visit's visible columns and the relations turned on there (one that is off is not fetched; the model-default tree bounds it), and every column a `where` on the way reads. A to-many relation carries its visit's clamps compiled as its `where`, so related rows come pre-narrowed — unless a clamp reads that list: a clamp reads it whole, as the database does, so it is fetched whole and `projectRows` cuts it. A to-one relation takes no `where` in Prisma, so `projectRows` drops one its clamp hides. A relation that shows no column — one turned on with all its columns hidden, or one a clamp reads only for presence or a count — is fetched by its key alone — the join key, else `id` — even a hidden one, as a clamp's columns are; never another column. The fetch carries it for the re-check and a viewer's projection drops it; a model with no key is not fetched, and presence on it can't be re-checked from fetched rows. A root that shows no column is selected by its `id` likewise. Bridges are skipped. It also selects what each projected source reads — the value, its label and axes, and every column and relation its option query's condition reads — as it does a clamp's columns, so `materializeSources` over the fetched rows offers what the database does; a viewer's projection drops them. A to-many relation's clamp that needs a counting step (a count or an aggregate), or has a window toPrisma can't compile, is refused (a `LensRefusal`, which `validateNarrowing` reports) before anything compiles. `options` is the clock for compiling the clamps. The root's own clamps are the query's `where`: `toPrisma(rule, { lens })`. |
+| `toLensSelect(lensOrNarrowing, options?)` | `{ select }` for `findMany` at the base model. It selects each visit's visible columns and the relations turned on there (one that is off is not fetched; the model-default tree bounds it), and every column a `where` on the way reads. A to-many relation carries its visit's clamps compiled as its `where`, so related rows come pre-narrowed — unless a clamp reads that list: a clamp reads it whole, as the database does, so it is fetched whole and `projectRows` cuts it. A to-one relation takes no `where` in Prisma, so `projectRows` drops one its clamp hides. A relation that shows no column — one turned on with all its columns hidden, or one a clamp reads only for presence or a count — is fetched by its key alone — the join key, else `id` — even a hidden one, as a clamp's columns are; never another column. The fetch carries it for the re-check and a viewer's projection drops it; a model with no key is not fetched, and presence on it can't be re-checked from fetched rows. A root that shows no column is selected by its `id` likewise. Bridges are skipped. It also selects what each projected source reads — the value, its label and axes, and every column and relation its option query's condition reads — as it does a clamp's columns, so `materializeSources` over the fetched rows offers what the database does; a viewer's projection drops them. For a source across a bridge that is its local reads: what it reads on this side, hidden or not, and each crossed bridge's local `on` key, so the rows plus the far side loaded under its bridge field answer it. A to-many relation's clamp that needs a counting step (a count or an aggregate), or has a window toPrisma can't compile, is refused (a `LensRefusal`, which `validateNarrowing` reports) before anything compiles. `options` is the clock for compiling the clamps. The root's own clamps are the query's `where`: `toPrisma(rule, { lens })`. |
 | `projectRows(lensOrNarrowing, rows, options?)` | Rows cut to what the lens shows, recursively. Hidden columns, and relations that are off or omitted, are removed. A row a visit's `where` hides is dropped from the root or a list, and a to-one row becomes `null`. `keepClampColumns: true` keeps the columns those `where`s and the projected sources read, even hidden ones, and a hidden to-one row, or a hidden row of a list a clamp reads, as those columns alone, so `check(narrowRule(rule, lens), row)` re-tests the clamps as the database does, for any rule the lens admits; that output carries hidden values, so never return it to a viewer. The other options (`now`, `bindings`) are what each `where` is checked with. Plain JSON in and out. |
 
 ### Evaluating Across Bridges

package.json

@@ -1,6 +1,6 @@
 {
   "name": "@inixiative/json-rules",
-  "version": "3.5.0",
+  "version": "3.5.1",
   "description": "TypeScript-first JSON rules engine with intuitive syntax and detailed error messages",
   "main": "./dist/index.cjs",
   "module": "./dist/index.js",

src/lens/lensRows.ts

@@ -17,7 +17,7 @@ import {
 } from './policy.ts';
 import { compileOrRefuse, prismaRefusal } from './prismaRefusal.ts';
 import { readPaths } from './readPaths.ts';
-import { type SourcePlan, sourcePlansWith } from './sourceOptions.ts';
+import { localReads, type SourcePlan, sourcePlansWith } from './sourceOptions.ts';
 import type { Lens, LensNarrowing } from './types.ts';
 
 /** A Prisma `select` tree: a column, or a relation with its own `select` and, to-many, `where`. */
@@ -55,22 +55,29 @@ const mergeTrees = (into: PathTree, from: PathTree): PathTree => {
 const clampPaths = (effect: VisitEffect): string[] => effect.whereClauses.flatMap(readPaths);
 
 // What each visit's sources read, by relation path: the value, its label and axes, and their
-// condition at the visit — below it only: the fetched tree is the path above. The fetch and a
-// re-check keep it as they keep clamp columns — hidden or not, never for a viewer — so
-// materializeSources over fetched rows offers what the database does.
+// condition at the visit — below it only: the fetched tree is the path above. A bridged source's
+// reads are its local ones — a read across the bridge is that bridge's local `on` key — for the
+// re-check over the rows with the far side loaded. The fetch and a re-check keep them as they keep
+// clamp columns — hidden or not, never for a viewer — so materializeSources over fetched rows
+// offers what the database does.
 type SourceReads = ReadonlyMap<string, readonly string[]>;
-const sourceReads = (plans: readonly SourcePlan[]): SourceReads => {
+const sourceReads = (policy: Policy, plans: readonly SourcePlan[]): SourceReads => {
   const out = new Map<string, string[]>();
   for (const plan of plans) {
-    // A model source's rows aren't the fetched ones, and a bridged one's aren't fetched at all.
-    if (plan.from || plan.bridged !== undefined) continue;
+    // A model source's rows aren't the fetched ones.
+    if (plan.from) continue;
     const key = plan.path.split('.').slice(1).join('.');
-    out.set(key, [
-      ...(out.get(key) ?? []),
+    const reads = [
       plan.field,
       ...(plan.label === undefined ? [] : [plan.label]),
       ...(plan.groupBy ?? []),
       ...readPaths(plan.rowWhere),
+    ];
+    out.set(key, [
+      ...(out.get(key) ?? []),
+      ...(plan.bridged === undefined
+        ? reads
+        : localReads(policy.lens, plan.visit.mapName, plan.visit.model, reads)),
     ]);
   }
   return out;
@@ -219,7 +226,14 @@ export const toLensSelectWith = (
   options: LensSelectOptions = {},
   plans: readonly SourcePlan[] = sourcePlansWith(policy),
 ): { select: LensSelect } => {
-  const select = selectAt(policy, rootVisit(policy), 'declared', {}, options, sourceReads(plans));
+  const select = selectAt(
+    policy,
+    rootVisit(policy),
+    'declared',
+    {},
+    options,
+    sourceReads(policy, plans),
+  );
   // Prisma can't select nothing: a root that shows no column is fetched by its `id`, hidden or not
   // (a viewer's projection drops it). A root model with no `id` stays as it is.
   if (!Object.values(select).some((selected) => selected === true)) {
@@ -322,7 +336,7 @@ export const projectRows = (
   const root = rootVisit(policy);
   // A viewer's rows carry nothing a source reads.
   const reads = keepClampColumns
-    ? sourceReads(sourcePlansWith(policy))
+    ? sourceReads(policy, sourcePlansWith(policy))
     : new Map<string, string[]>();
   return rows.flatMap((row) => {
     const kept = cutRow(policy, root, row, {}, keepClampColumns, options, reads);