Skip to content

pkg/cueutil: write the config tree to disk once per pass - #2627

Draft
dancannon wants to merge 1 commit into
encoredev:mainfrom
dancannon:perf-cue
Draft

dancannon wants to merge 1 commit into
encoredev:mainfrom
dancannon:perf-cue

Conversation

@dancannon

@dancannon dancannon commented Oct 27, 2026 •

Copy link
Copy Markdown

cueutil.LoadFromFS copies the whole CUE tree to a temporary directory for every service: thousands of file writes on a large app.

  • Split it into WriteToDisk, which writes the tree once and returns a cleanup function, and LoadFromDir, which loads one service from it.
  • computeConfigs first generates every service's CUE, then writes the tree once and loads each service from it.
  • Add tests for loading several services from one tree, an incomplete config, and cleanup.

Why this should not change existing behaviour

  • The load itself is unchanged. LoadFromDir contains the same CUE loading, orphaned-file handling and validation as the old function, with the temporary directory passed in.
  • Each service still loads only the files under its own path. The old per-service tree held every service's files too, so the loader saw the same set of files per service. The only difference is that a service now sees the generated encore.gen.cue of every service in the tree, where before it saw those of the services generated before it. A config that imported another service's generated package would therefore have depended on service order, so none that works today can rely on that.
  • On a large app the resulting config JSON for every service is identical before and after.
  • LoadFromFS is internal to the encr.dev module and had no other callers.
  • Errors still carry the temporary directory path, are built before cleanup runs, and the directory is removed on every path, including write failure. A write failure is reported on the first service that uses config, and a load failure on the service that failed, as before.
  • Apps with no service config skip writing a tree altogether.

Testing

  • go test -race ./pkg/cueutil/... ./v2/v2builder/...
  • go vet, golangci-lint --new-from-rev=origin/main (0 issues) and a Windows build.

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

LoadFromFS copies the whole CUE tree to a temporary directory for every
service (thousands of file writes on a large app). Split it into
WriteToDisk and LoadFromDir, and write the tree once per service-config pass.

A service's config is evaluated with the generated encore.gen.cue of every
service present in the tree, not only those added before it. Each service
still loads only the files under its own path, and the resulting configs are
unchanged on a large app. Computing service configs is several times faster.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant