Skip to content

getBuildTargets reports a source it cannot compile instead of failing the plan - #3165

Closed
housejester wants to merge 4 commits into
malloydata:mainfrom
housejester:jde/build-targets-unbuildable
Closed

housejester wants to merge 4 commits into
malloydata:mainfrom
housejester:jde/build-targets-unbuildable

Conversation

@housejester

@housejester housejester commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

A persisted source that reads a given: with no default cannot be persisted: its table would have to be one table for every caller (#3041). Today that authoring error surfaces in two bad ways. getBuildTargets throws on it and takes the whole model's plan with it, so a builder cannot build any of the model's other tables. And both that throw, and a query over the source once a manifest is in play, carry a message that tells the reader to supply the given, which a builder cannot do and the query already did.

##! experimental { persistence givens }

given: TENANT :: string
source: orders is duckdb.table('orders')

#@ persist
source: daily_orders is orders -> { group_by: order_date; aggregate: n is count() }

#@ persist
source: my_orders is orders -> { where: tenant = $TENANT; group_by: order_date }

#@ persist
source: my_orders_at_read is orders -> { group_by: tenant, order_date } extend { where: tenant = $TENANT }

With this change:

  • getBuildTargets plans daily_orders and my_orders_at_read, and reports my_orders in a new unbuildable list instead of throwing.
  • The error, on the plan side and the query side alike, is runtime-persist-given-no-value: it names the source, says why it cannot be persisted, and points at the shape that can. An extend-block where: is not part of the persisted relation, so my_orders_at_read's table holds every tenant and each query reads its own slice.
const {connections, tagParseLog, unbuildable} =
  await runtime.getBuildTargets(model);
// unbuildable: {source: PersistSource, problem: LogMessage}[]

It also clears the way for the #3029 follow-up of deleting getBuildPlan. That never compiles SQL, so a builder on it can skip a source like my_orders and build the rest; on getBuildTargets it cannot.

Fixes #3041.

How

  • persistedSourceSQL in model/sql_compiled.ts compiles a persisted source's SQL for both the builder (PersistSource.getSQL) and the manifest lookup (persistedTableFor). Both compile without resolving givens (buildIdOptions), so a compiler-given-no-value raised there can only mean the source reads a given with no default; it becomes runtime-persist-given-no-value, at the given's location. The live inline of a persisted source still compiles with the query's givens and keeps the ordinary error.
  • getBuildTargets reports a source whose SQL raises a MalloyCompileError in unbuildable with that problem. Anything else, including a dialect that cannot express the SQL, still fails the call. A source built on an unbuildable one inlines its SQL, so it is reported too.
  • The same path covers a virtual source with no virtualMap entry (runtime-virtual-map-missing).
  • mkBuildTargets takes the list as an out-array, as resolvePersistWalk does tagParseLog. The sample builder prints what it skipped and builds the rest.

Open questions

  • Should this be a translator error at the #@ persist? Whether a persisted source reads a given with no default is visible at translate time. Proposed: keep it on the build and query paths, as persist: a source depending on a supplied given fails with an error that names neither persistence nor the given #3041 asks. A translator error would fail the whole model's compile because one source can't be persisted, and a builder would lose every other table in the model.
  • Translate at the boundary, or tell the compiler it is compiling a persisted relation? persistedSourceSQL catches compiler-given-no-value and raises the persist error. The alternative is a flag on the compile options, so the expression compiler raises it directly. That widens BuildIdOptions, which is deliberately narrow. Proposed: the boundary, since it's one function both sides already share.
  • unbuildable on BuildTargets, or per ConnectionBuild? Proposed: on BuildTargets, beside tagParseLog. Both report what's wrong with the model's persist sources, rather than what to build on a connection.

Behaviour change: a model that used to throw from getBuildTargets now plans with fewer targets. A builder that relied on the throw has to check unbuildable; doc/persist/api.md says so. A query that used to fail over such a source still fails, with the new message.

… the plan

A persist source whose SQL raises a MalloyCompileError (a given with no default, a virtual source with no map entry), and anything that inlines it, comes back in BuildTargets.unbuildable with the compiler's problem; the rest of the model is planned. Any other error still throws.

Signed-off-by: James Estes <james.estes@credibledata.com>
@housejester
housejester marked this pull request as ready for review October 20, 2026 15:50
James Estes added 3 commits October 20, 2026 14:21
…not be persisted

The builder and the manifest lookup both compile a persisted source's SQL without resolving givens, so a given with no default has no value there. Both now raise persist-given-no-value, naming the source and the extend-block where: that filters per caller, instead of asking for a value the builder cannot supply and the query already did. Fixes malloydata#3041.

Signed-off-by: James Estes <james.estes@credibledata.com>
…names in the present

Signed-off-by: James Estes <james.estes@credibledata.com>
… function's invariant

Signed-off-by: James Estes <james.estes@credibledata.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

persist: a source depending on a supplied given fails with an error that names neither persistence nor the given

1 participant