flake-compatish

Evaluate a flake without the flake evaluator. flake-compatish reads flake.nix and flake.lock in plain Nix and calls outputs itself.

Why

The flake evaluator copies your source to the Nix store before it evaluates anything. On a large tree, or when you work on a dependency such as nixpkgs, every change costs a copy.

flake-compatish lets you:

• Keep flake.nix for CI and for other people. The flake stays a normal flake.
• Skip the store copy on your own machine. Evaluation reads your working directory.
• Point inputs at local checkouts without an edit to flake.lock.
• Replace a whole flake anywhere in the graph, including copies of it that follows cannot reach.

Requirements

• builtins.fetchTree, which needs Nix 2.4 or later with flakes enabled.
• builtins.parseFlakeRef, but only when you give a flake reference as a string.

Quick start

Add flake-compatish as an input of your flake:

{
  inputs.flake-compatish.url = "github:Lillecarl/flake-compatish";
  # ...
}

Then write a default.nix next to flake.nix. It fetches flake-compatish at the revision your lockfile pins:

let
  lock = builtins.fromJSON (builtins.readFile ./flake.lock);
  flake-compatish = builtins.fetchTree lock.nodes.flake-compatish.locked;
in
import flake-compatish ./.

Build with the non-flake CLI:

nix build --file . outputs.packages.x86_64-linux.default
nix build --file . impure.packages.default   # current system selected for you

Two ways to call it

default.nix takes a path or an attrset. The path form is short for { source = ./.; overrides.self = ./.; }. Use the attrset form when you need any other argument.

A local source is read in place

A local tree is a path, a string that starts with /, or an attrset with such an outPath: an evaluated flake, a fetchTree result, or { outPath; sourceInfo; }. Any other string is a flake reference.

A local source is read where it is, in both forms. Nothing is copied to the store at evaluation; a build that uses self copies it then. A copy at evaluation would take every file, ignored ones too.

• Impure evaluation reads any local tree in place.
• Pure evaluation can read only the store. A tree in the store, such as a directory of the flake that calls flake-compatish, is read in place. A tree outside it is an error.
• A string with store context keeps that context, so a build uses that store path and does not copy it again. A path value in the store stays a path value. flake-compatish does not add context to it: in pure evaluation, a flake's source is a store path that is not in the store yet.
• sourceInfo comes from the attrset when it has one. Otherwise it is only outPath, lastModified = 0 and its date.

From inside another flake, this gives exactly what the flake evaluator gives, sourceInfo included:

{
  outputs = { self }: {
    sub = (import flake-compatish {
      source = {
        outPath = self.sourceInfo.outPath + "/sub";
        inherit (self) sourceInfo;
      };
    }).outputs;
  };
}

Arguments

argument	default	effect
source	required	The flake to evaluate. A local tree (see above), or a flake reference string such as "github:owner/repo".
overrides	{ }	Replace the source of an input. See Override an input's source.
overridesFile	source + "/overrides.nix"	The optional machine-local override file. See Local overrides.
systemOverrides	true	The machine's overrides for every repository. See System-wide overrides.
warnOverrides	true	Print an evaluation warning for each override that applies.
reroot	{ }	Replace an evaluated flake at any depth. See Replace a flake.
nixpkgsArgs	null	Configure one shared nixpkgs per system. See Shared package sets.

lib.nix takes one more, strict. See Strict names.

Output

{
  outputs   = { packages, ... };   # the root flake
  inputs    = { self, nixpkgs, ... };
  impure    = { packages, ... };   # outputs with builtins.currentSystem selected
  compatish = { ... };             # the lib.nix result, see below
}

• outputs is the root flake as another flake would see it: its outputs, plus outPath, sourceInfo, inputs, outputs and _type = "flake".
• inputs holds the root's resolved inputs, and self.
• impure takes each top-level attribute of outputs and selects .${builtins.currentSystem} where it exists. impure.packages.hello is outputs.packages.x86_64-linux.hello on an x86_64 Linux machine. It needs impure evaluation, which nix build --file gives by default.
• compatish is described in Overridable results.

Override an input's source

overrides replaces where an input's source comes from. The input keeps the inputs that flake.lock gives it.

let
  home = /. + builtins.getEnv "HOME";
in
import flake-compatish {
  source = ./.;
  overrides = {
    self = ./.;                                 # the root itself
    nixpkgs = home + "/Code/nixpkgs";           # a path: used in place, no copy
    home-manager = "github:nix-community/home-manager/master";  # a string: fetched
  };
}

• A local tree is used in place: a path, a string that starts with /, a store path such as pkgs.path or inputs.nixpkgs.outPath, or an evaluated flake. Evaluation reads it where it is, so an edit shows up on the next evaluation. The rules for source apply: a string with store context keeps it, and a flake brings its sourceInfo. A path value outside the store gets a store copy when a build uses it. Use path arithmetic for a subdirectory: input.outPath + "/sub" copies only sub, and only when a build needs it. "${input}/sub" copies the whole checkout.
• Any other string is a flake reference. flake-compatish parses it and fetches it.
• self names the root flake.
• Other names must be inputs of the root flake. flake-compatish ignores any other name and does not evaluate its value, so a typo goes unnoticed. Use lib.nix if you want an error instead.
• A local tree that does not exist falls back to the lockfile, with no warning. So one default.nix can name a checkout that only some machines have.

An override replaces the lockfile node with that name. Every flake in the graph that uses that node sees the override. A second copy of the same dependency has a suffixed node name, such as nixpkgs_2, and an override does not reach it. Use follows in flake.nix, a system-wide override, or reroot.

Paths in your home directory

Nix expands ~/Code/nixpkgs in impure evaluation only. In pure evaluation, Nix rejects the whole file while it parses it, even when nothing reads that value, and tryEval cannot catch the error.

• ~ is safe in the local overrides.nix, in the system-wide file, and in nix … --impure --expr. flake-compatish reads the two files in impure evaluation only.
• In a committed file, such as default.nix, use /. + builtins.getEnv "HOME" + "/Code/nixpkgs". In pure evaluation getEnv returns an empty string, so the path does not exist and the lockfile applies.

Local overrides

Machine-local checkouts belong in a file that git ignores. Put overrides.nix next to flake.nix:

{
  nixpkgs = ~/Code/nixpkgs;
}

And add it to .gitignore:

overrides.nix

flake-compatish looks for this file relative to source. overridesFile picks a different path.

• flake-compatish reads the file only during impure evaluation. It tests whether builtins.currentSystem exists, which pure evaluation removes. The flake evaluator therefore never applies it, even when a dirty tree copied it into the store.
• The same name rules apply as for overrides.
• A local value wins over everything else. See Precedence.

System-wide overrides

The local file belongs to one repository. A system-wide file belongs to the machine, and applies to every repository that uses flake-compatish.

It matches inputs by flake reference, not by name. Names belong to each project: one repository's nixpkgs can be nixos-unstable, and another's can be a 24.05 pin. A key names the dependency itself:

{
  "github:NixOS/nixpkgs" = ~/Code/nixpkgs;          # any branch
  "github:nix-community/home-manager/master" = ~/Code/home-manager;  # this branch only
}

Point the NIX_PATH entry flake-compatish-overrides at the file:

export NIX_PATH="flake-compatish-overrides=$HOME/.config/flake-compatish/overrides.nix:$NIX_PATH"

On NixOS or home-manager, nix.nixPath sets it declaratively. -I flake-compatish-overrides=… sets it for one command.

• A key matches a lockfile node when every part of the key equals that part of the node's original or locked reference. github:NixOS/nixpkgs matches every branch. github:NixOS/nixpkgs/nixos-unstable matches one. locked is what lets github:NixOS/nixpkgs match an input written as the registry name nixpkgs.
• Owner and repository ignore case, as GitHub does. A lockfile keeps the case the flake author wrote.
• The most specific key wins when several match: the one with the most parts.
• It reaches every depth. Matching runs on each lockfile node, so it also reaches nixpkgs_2 and the copies that overrides cannot.
• Values are the same as for overrides: a local tree used in place, or a flake reference string. A local tree that does not exist passes to the next source.
• Impure evaluation only. Pure evaluation has an empty NIX_PATH, so the flake evaluator never applies it.
• Strict names never apply. Most keys name a dependency that a given repository does not have.

For full control, the file can be a function instead of an attrset. It gets each node, and returns a value or null:

{ nodeName, original, locked }:
if original.type or "" == "github" && original.owner or "" == "NixOS" then
  ~/Code/nixpkgs
else
  null

systemOverrides changes where the overrides come from:

value	effect
true	read the NIX_PATH entry, if there is one
false	read nothing, for a repository that must opt out
a path, an attrset or a function	use it instead of the NIX_PATH entry

Precedence

flake-compatish takes the first source that gives a usable value for a node:

1. the local overrides.nix, by input name
2. the system-wide overrides, by flake reference
3. the overrides argument, by input name
4. the lockfile

Machine-local intent beats the defaults a project commits. A value that cannot be used, such as a path that does not exist here, passes to the next source.

Warnings

Each override that applies prints one line:

evaluation warning: flake-compatish: /home/me/project: overriding nixpkgs to /home/me/Code/nixpkgs

The first path is the root source. A system-wide override ends with (system-wide). Pass warnOverrides = false; to turn the warnings off.

Disable every override

Set FLAKE_COMPATISH_DISABLE_OVERRIDES to anything other than empty or 0:

FLAKE_COMPATISH_DISABLE_OVERRIDES=1 nix build --file . impure.packages.hello

flake-compatish then ignores overrides, the local file, the system-wide overrides, self and reroot, and evaluates from the lockfile alone. The root still stays in place: in impure evaluation it is never copied, override or not. Use it to check whether a local checkout causes a result, with no edit to any file. nixpkgsArgs still applies.

Replace a flake: reroot

overrides changes a source and keeps the lockfile's view of that input's own inputs. That is not enough two levels down. If your flake uses easykubenix, and easykubenix uses nanopynix, the lockfile can hold a second nanopynix node. No override reaches it.

reroot replaces the evaluated flake instead. You evaluate it yourself, with whatever inputs you want, and flake-compatish uses it wherever the graph has an input of that name:

let
  nanopynix = (import flake-compatish (/. + builtins.getEnv "HOME" + "/Code/nanopynix")).outputs;
in
import flake-compatish {
  source = ./.;
  reroot = { inherit nanopynix; };
}

• The key is the input name, not the node name. That is why it reaches nanopynix_2: the node has a suffix, and the input that names it does not.
• It applies at every depth, root included. No follows is needed.
• An input declared flake = false is not rerooted. It is a plain source, and the flake that declares it reads it as a path.
• flake-compatish passes the value on as it is. Give it what the consuming flakes read: usually outPath and the outputs. (import flake-compatish …).outputs has the full shape of an evaluated flake.

A function as the value

A reroot value can also be a function. It gets the flake that would have been used, and returns the replacement:

import flake-compatish {
  source = ./.;
  reroot.nixpkgs = prev: prev // {
    legacyPackages = builtins.mapAttrs (system: _:
      import prev.outPath { inherit system; config.allowUnfree = true; }
    ) prev.legacyPackages;
  };
}

flake-compatish calls the function once for each pair of input name and lockfile node, not once for each place the input appears. Two copies of nixpkgs give two calls; one copy shared by ten flakes gives one.

Shared package sets (experimental)

nixpkgsArgs gives every compatible flake in the graph the same configured nixpkgs, one per system:

import flake-compatish {
  source = ./.;
  nixpkgsArgs = system: {
    inherit system;
    config.allowUnfree = true;
  };
}

flake-compatish imports the root's nixpkgs input once for each system that input exposes. Then it replaces self.legacyPackages in each other flake that already has a legacyPackages output, for the systems that output already has. A compatible flake builds its packages from inputs.self.legacyPackages.${system}.

The root flake must have an input named nixpkgs, or evaluation aborts.

This is not the same as a reroot.nixpkgs function. reroot replaces the nixpkgs input that a flake receives. nixpkgsArgs replaces each flake's own self.legacyPackages.

What the lockfile can hold

• Lockfile versions 5, 6 and 7. Another version aborts with an error.
• A flake without flake.lock, which has no inputs except self.
• follows, at any depth.
• A flake in a subdirectory, ?dir=sub.
• Inputs with flake = false. They arrive as a source, not as a flake.
• A relative path: input, such as path:../sibling. It is in the tree of the flake that declares it, as with the flake evaluator: it shares that flake's sourceInfo, and an override of that flake moves it. Under a flake used in place, it is in place too.

Differences from the flake evaluator

With a flake reference as source, flake-compatish gives what builtins.getFlake gives: the same attributes, sourceInfo, outPath and outputs, at every depth. So does a local tree that carries its sourceInfo, in pure evaluation: see A local source is read in place.

Two tests compare them, on fixtures that nix flake lock locked. test/parity.nix runs impure, on flake references and on <nixpkgs>, which has no flake.lock. test/pure-parity.sh evaluates each fixture with the flake evaluator, and again from test/pure-parity, a flake with no inputs that imports it with flake-compatish. Both assert the differences below, so a change to one fails the tests.

• A local tree without sourceInfo. A path or a string gives sourceInfo with only outPath, lastModified = 0 and its date, because nothing is hashed. Its outputs are the flake evaluator's.
• A path value normalizes ... Relative inputs under a path value are path values too, and Nix normalizes their .. parts. The flake evaluator, and a string source, keep them: top/../leaf.
• A stale lockfile that lacks an input is an error, not a new lock. See below.
• Overrides, reroot and nixpkgsArgs change the result on purpose. They are not in the flake evaluator.

A stale lockfile

The flake evaluator locks a new input and drops a removed one by itself. flake-compatish cannot lock, so it stops with an error that says to run nix flake lock:

• flake.nix declares an input that flake.lock does not have. An argument of outputs counts as an input, as it does for the flake evaluator.
• flake.nix declares inputs, and there is no flake.lock.

The first test also applies to each dependency. An override can supply a newer checkout of a dependency with an input that the lockfile does not know.

Two cases give a warning and evaluate:

• flake.lock has a root input that flake.nix does not declare. flake-compatish does not pass it to outputs, as the flake evaluator drops it.
• The url of a root input no longer gives the reference in flake.lock. flake-compatish uses the locked reference.

Errors in an input

An error while flake-compatish fetches or evaluates an input names the chain of input names from the root:

… while fetching the flake input `tool → nixpkgs` (node `nixpkgs_2`)

The node name is shown when it differs from the last input name. A node that several chains reach shows the shortest one.

Overridable results: lib.nix

default.nix stays a plain flake-compat shim. lib.nix gives a result that you can override again after import. The idea comes from sprinkles.

let
  lock = builtins.fromJSON (builtins.readFile ./flake.lock);
  flake-compatish = builtins.fetchTree lock.nodes.flake-compatish.locked;
in
(import "${flake-compatish}/lib.nix").new ./.

new takes the same arguments as default.nix, plus strict. The result has inputs, outputs and impure, and also:

attribute	value
args	the arguments this result was evaluated with
sources	the root's inputs as fetched: after overrides, before reroot. outPath is the input's directory
override	evaluates again with changed arguments

You do not need lib.nix to get this. default.nix returns the same result as compatish, from the same evaluation, and computes it only when you read it:

nix build --impure --expr '
  ((import ./.).compatish.override { overrides.nixpkgs = ~/Code/nixpkgs; })
    .impure.packages.default'

override

An attrset merges into the arguments. overrides and reroot merge key by key, so each call adds to the one before:

nix build --impure --expr '
  (((import ./.).compatish
    .override { overrides.nixpkgs = ~/Code/nixpkgs; })
    .override { overrides.home-manager = ~/Code/home-manager; })
    .impure.packages.default'

A function gets the previous arguments and returns an attrset to merge with //. Use it to remove a key:

flake.override (prev: { overrides = removeAttrs prev.overrides [ "nixpkgs" ]; })

override needs the overrides. prefix because input names are free-form. An input can be named source or strict, so a flat override { nixpkgs = …; } could not tell an input from an argument.

Compose projects

When a project's default.nix uses flake-compatish, another project can take a configured copy of it at any depth:

let
  home = /. + builtins.getEnv "HOME";
in
import flake-compatish {
  source = ./.;
  reroot.mylib = _:
    ((import (home + "/Code/mylib")).compatish.override {
      overrides.nixpkgs = home + "/Code/nixpkgs";
    }).outputs;
}

Strict names

Through lib.nix or compatish, an unknown name aborts evaluation:

error: flake-compatish: no input to reroot named `simpel`; known: `nixpkgs`, `simple`, `with-dep`

• An overrides name must be self or an input of the root flake.
• A reroot name must be an input that some flake in the graph declares.
• The local overrides.nix is never checked. It belongs to the machine, not to the project.

Pass strict = false; to ignore unknown names, as default.nix does. default.nix stays lenient even when you read compatish: only reads through compatish see the check.

API reference

The documentation site is at https://lillecarl.github.io/flake-compatish/. It holds this README and an API reference.

Every function carries a nixdoc comment, and the reference is generated from them. Build it yourself:

nix build --file ./doc api    # the reference as Markdown
nix build --file ./doc site   # the HTML site

It covers default.nix, lib.nix, and the attributes that the internal nix/core.nix and nix/overridable.nix return. nixdoc does not extract let bindings, so the comments on internal helpers are for readers of the source.

Running tests

./test/integration.sh

It also runs test/no-copy.sh, which evaluates against an empty chroot store and fails if anything was copied to it.