Skip to content

Zig 0.17.0 support - #597

Open
ityonemo wants to merge 18 commits into
mainfrom
zig-0.17.0-support
Open

Zig 0.17.0 support#597
ityonemo wants to merge 18 commits into
mainfrom
zig-0.17.0-support

Conversation

@ityonemo

@ityonemo ityonemo commented Oct 4, 2026

Copy link
Copy Markdown
Collaborator

Updates zigler to Zig 0.17.0.

Reflection

0.17 reports struct, union and enum members as parallel arrays rather than an array of per-field records, which touches roughly forty sites in priv/beam. priv/beam/reflect.zig is a new module that rebuilds the old per-field view at comptime, so those sites keep reading field.name / field.type. It has to be its own build module -- a file may belong to exactly one module, and beam, the sema root and the nif all reach for it.

Alongside that: Type.Struct.decls is now decl_names, Type.Fn.params is now param_types/param_attrs, pointer attributes moved into a nested attrs struct and were renamed (is_const -> attrs.@"const"), and Type.ErrorSet is a struct holding an optional error_names. The "is_const" keys that sema emits are wire protocol and are deliberately unchanged.

Other 0.17 changes

  • std.builtin is deprecated in favour of std.lang; OptimizeMode is now Optimize and its tags lost the Release prefix. Zigler's own optimize: option already used :debug/:safe/:fast/:small, so this is not a user-facing break.
  • std.meta.Int was removed in favour of the new @Int builtin.
  • @bitSizeOf no longer accepts extern structs.
  • The ** array repetition operator and the errdefer capture are gone.
  • std.zig.Ast.parse takes a ParseOptions.

C translation

0.17 deprecates the built-in std.Build.Step.TranslateC, so header translation now goes through the ZSF translate-c package. It and its aro dependency are fetched as source-only git deps, so their commits are pinned in mix.lock, then staged beside the generated build.zig with aro's git URL rewritten to a path -- nif compilation still needs no network access. Verified by running the suite from a cold cache with networking disabled.

translate-c is a build-time tool, so it is built for the host; building it for the target broke cross compilation with "unable to spawn foreign binary translate-c.exe".

The vendored Windows headers stay. 0.17's translate-c handles the WinDynNifCallbacks pattern far better than the old implementation, but it renders the nested #define enif_alloc ERL_NIF_API_FUNC_MACRO(enif_alloc) literally, which zig then rejects as self-referential -- for all 174 macro-ised functions. priv/erl_nif_win/erl_nif_win.h now documents the mechanism and what to re-check on future translate-c releases.

Dependencies

zig_parser and zig_doc move to 0.8.0 (0.17 grammar); zig_get is a path dep until 0.17.0 is published to hex.

Incidental fixes

  • mix zig.get built its target directory as zig-{os}-{arch}-{version} while the tarball and Zig.Command both use {arch}-{os}, so an existing toolchain was never detected and ~54MB was re-downloaded on every run.
  • mix zig.get --file fetched the download manifest before reading the local tarball, making an offline install impossible.
  • The zig.get moduledoc advertised --from and --disable-verify; the task implements --file, with verification via the VERIFY environment variable.
  • mix docs was broken: run_sema_doc!/1 writes its own build.zig and did not know about the new reflect module. Verified against a 0.16 baseline that all 23 pages still generate with no documented symbols lost.

Verification

615 tests pass, including from a clean build and with networking disabled. mix docs generates all 23 pages. Not covered locally: Windows, macOS and FreeBSD, which are left to CI.

ityonemo and others added 18 commits October 4, 2026 11:59
@ityonemo @claude
updates to zig 0.17.0
693b8e0 
Zig 0.17 reshapes a lot of the compile-time reflection that the BEAM marshalling
code is built on, so the bulk of this is mechanical:

  * @typeinfo reports struct, union and enum members as parallel arrays instead of
    an array of per-field records.  priv/beam/reflect.zig is a new module that
    rebuilds the old per-field view at comptime, so the ~40 reflection sites keep
    reading field.name / field.type.  It has to be its own build module: a file may
    belong to exactly one module, and both beam and the sema root need it.
  * Type.Struct.decls -> decl_names, Type.Fn.params -> param_types/param_attrs
  * Type.Pointer attributes moved into a nested `attrs` struct and were renamed
    (is_const -> attrs.@"const", and so on).  The "is_const" JSON keys emitted by
    sema are wire protocol and deliberately unchanged.
  * Type.ErrorSet is a struct holding an optional error_names list
  * std.builtin is deprecated in favour of std.lang; OptimizeMode is now Optimize
    and its tags lost the "Release" prefix
  * std.meta.Int was removed in favour of the new @int builtin
  * @bitSizeOf no longer accepts extern structs
  * the ** array repetition operator is gone (@Splat), as is the errdefer capture
  * std.zig.Ast.parse takes ParseOptions

C header translation now goes through the ZSF translate-c package, since 0.17
deprecates the built-in std.Build.Step.TranslateC.  translate-c and its aro
dependency are fetched as source-only git deps so their commits are pinned in
mix.lock, then staged beside the generated build.zig with aro's git URL rewritten
to a path -- nif compilation still needs no network access.  translate-c is a
build-time tool, so it is built for the host even when cross compiling.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ityonemo
merges main into zig-0.17.0-support
7897b09
@ityonemo @claude
documents why the windows erl_nif headers are vendored
0f3d1ed 
Zig 0.17's translate-c handles the windows NIF callback table much better than the
old implementation, but not well enough to drop these copies: it renders the nested
`#define enif_alloc ERL_NIF_API_FUNC_MACRO(enif_alloc)` literally, which zig then
rejects as a declaration that depends on itself.  All 174 macro-ised functions are
affected -- zig reports only one per compile, so a single build makes it look like a
handful.

Records what was measured so the next person does not have to re-derive it, and what
to re-check when translate-c/aro improve.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ityonemo @claude
moves to zig_parser 0.8.0 / zig_doc 0.8.0
ef601dd 
zig_parser 0.8.0 is zig 0.17's own grammar.peg, which reshapes a good deal of the
AST.  Verified against it: zigler's suite passes unchanged.

zig_doc is on a path dependency for the moment because 0.8.0 is not yet published to
hex; it needs the bump only to relax its own `zig_parser ~> 0.7.0` pin.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ityonemo @claude
updates zig_get to 0.17.0
23a8312 
Bumps the package and the default toolchain version, and fixes three things found
while verifying an end-to-end install:

  * ensure_destination/1 built the target directory name as zig-{os}-{arch}-{version}
    while both the upstream tarball and Zig.Command use {arch}-{os}.  The name never
    matched, so an existing toolchain was never detected: `mix zig.get` re-downloaded
    ~54MB every run and --force had nothing to remove.
  * --file reached for https://ziglang.org/download/index.json before reading the
    local tarball, which made an offline install impossible.  There is nothing to
    look up when the file is already on disk -- the manifest hash describes the
    upstream download -- so get_meta/1 is skipped in that case.
  * the moduledoc advertised --from and --disable-verify; the task actually
    implements --file, and verification is controlled by the VERIFY environment
    variable.  Either documented flag would have crashed with no matching clause.

Also moves preferred_cli_env into `def cli`, which elixir 1.19 deprecated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ityonemo @claude
fixes doc generation, moves to published zig_doc 0.8.0
16f860a 
run_sema_doc!/1 writes its own build.zig, separate from the one rendered from
templates, and it was never taught about the new reflect module -- so `mix docs`
died with "no module named 'reflect' available within module 'root'" while the test
suite stayed green, since nothing but doc generation goes down that path.

Verified against a 0.16 baseline built from main: all 23 pages generate, and every
one of the documented symbols is still found (45 anchors on beam.html in both, none
missing anywhere).  The only differences are the version string, internal struct ids
and prose line wrapping.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ityonemo @claude
updates CI to current elixir/erlang, drops EOL versions
0ceeb5a 
OTP supports the two most recent majors, so the linux matrix drops OTP 27, 26 and
25 and now covers OTP 29.1 on elixir 1.20.4, and OTP 28.5 on 1.20.4 / 1.19.6 /
1.18.5.  macos already installs whatever brew ships; windows and freebsd move to
29.1 / 1.20.4, with freebsd's in-VM package and PATH following to erlang-runtime29.

The declared elixir requirement moves from ~> 1.15 to ~> 1.18 to match what is
actually tested.  The credo job deliberately stays on 1.19: credo 1.7.x cannot
tokenize the ~t() sigil under elixir 1.20.

Also fixes two caches that never worked:

  * linux cached a repo-relative `zig` directory, but mix zig.get installs into
    ZIG_ARCHIVE_PATH, so the toolchain was re-downloaded on every run.  Together
    with the zig.get directory-name fix, the cache is now actually hit.
  * macos computed the elixir version into stdout but never wrote $GITHUB_OUTPUT,
    then keyed on a nonexistent `stdout` output, so both of its cache keys were
    constant.

zig_get moves to the published 0.17.0 rather than the in-repo path dependency.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ityonemo @claude
fixes windows and freebsd CI
d31a5d8 
Three separate failures:

  * freebsd: mix shells out to git to check the lock status of the translate_c and
    arocc dependencies, but git is not installed in the VM image.  Before 0.17
    zigler had no git dependencies, so this is new.  The host runner still does the
    fetching; the VM only needs the binary present.

  * windows: std.debug.Pdb.getInlineeSourceLines now returns an iterator rather
    than a slice, and it yields the InlineeSourceLine directly instead of a wrapper
    carrying .info.

  * windows: enif_make_int64/enif_make_uint64 are aliases for enif_make_long and
    enif_make_ulong there, whose parameters are c_long and c_ulong, so casting to a
    hardcoded i64/u64 is wrong.  Let @intcast take the type from the function.

The last two were only reachable by cross compiling, which did not work either:
windows?/0 consulted the host OS and precompilation metadata but not the
cross-compilation target, so a TARGET_OS=windows build picked the host erl_nif.h
instead of the vendored windows headers.  That is pre-existing, and fixing it is
what made the two errors above visible locally.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ityonemo @claude
fixes macos error return traces
adf4beb 
zig 0.17 reports the source path relative to the build root on macos, so a
stacktrace carries e.g. "../../../../Users/runner/.../.Elixir.Foo.zig" rather than
the absolute path the manifest resolver matches on.  Neither the exact-path nor the
windows clause matched, so __resolve/2 fell through to its passthrough and returned
the generated zig line instead of mapping back to the .exs -- all four error return
trace tests failed with the zig line number.

Falls back to comparing basenames, which is stable however the path is spelled.
Unrelated files still pass through untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ityonemo @claude
stages translate-c once per build rather than per module
9eafbd0 
TranslateC.stage!/1 copied translate-c and aro into every module's staging
directory.  That is ~23MB across ~1300 files each time, which on windows was slow
enough that File.cp_r blew ExUnit's 60s timeout and failed four tests.

They are now staged once into a fingerprinted directory in the staging root, shared
by every module in the build.  The fingerprint comes from the dependency mtimes, so
`mix deps.update` lands in a fresh directory rather than half-overwriting the old
one, and the copy goes via a temporary name and an atomic rename so concurrent
compilations cannot observe a partial tree.

The reference has to stay relative: zig rejects absolute paths in build.zig.zon
("expected path relative to build root"), and since the shared tree is a sibling of
each staging directory it is reached as ../<fingerprint>/translate_c.

Re-verified: the suite passes, the windows cross compile is clean, and a cold build
with networking disabled still works.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ityonemo @claude
expands relative stacktrace paths
3d0ad76 
zig 0.17 reports source paths relative to the build root on macos, so a runtime
stacktrace carries "../../../../Users/.../transitive_error.zig".  The manifest
resolver's basename fallback covers the module's own generated file, but a nif that
errors inside a separate hand-written .zig file is reported under that file's own
path, which is then handed to the caller verbatim -- so the trace pointed at a
relative path where an absolute one was expected.

Expands any path that starts with ../ back to absolute, next to the existing
windows normalisation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ityonemo @claude
precompiles translate-c in CI before running tests
cf0aa58 
Compiling translate-c and aro costs about 70s and peaks near 1.1GB.  That happened
inside whichever test compiled a nif first, which on the slower runners blew
ExUnit's 60s timeout, and on freebsd an unbounded test fan-out met it with several
builds in flight and the VM was OOM-killed (exit 137).

`mix zig.warm` does that build up front, into the global zig cache that every
subsequent compilation shares.  Measured locally: the first nif compile drops from
102s to 33s behind it, and peak memory for a build that finds the cache warm is
~300MB rather than ~1.1GB.

Worth noting for anyone tempted to parallelise this: zig's global cache lock already
prevents duplicated effort.  Four concurrent cold builds take the same 72s as one,
with only the first paying the 1.1GB -- the rest wait and reuse the result.  The
problem was never duplication, just that the cost landed inside a timed test.

freebsd also caps the fan-out at 4, since each build still peaks around 300MB.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ityonemo @claude
fixes relative stacktrace paths and raises the test timeout
3118396 
The previous commit expanded "../"-prefixed stacktrace paths with Path.expand/1,
which resolves against the cwd.  That is not the build root, and on freebsd it
produced a doubled prefix:

  /home/runner/work/home/runner/work/zigler/zigler/test/transitive_error.zig

What zig actually emits is a run of "../" climbing out of the build root followed
by what was an absolute path, so the climb is stripped and the leading separator
restored instead.  Windows drive letters and genuinely relative paths are left
alone.

Separately, the per-test timeout moves from the 60s default to 300s.  Nearly every
test compiles zig, and the runners are an order of magnitude slower than a
development machine -- the suite takes ~20s locally and ~1600s on macos -- so the
handful of tests that compile a nif inside the test body were timing out there
while passing comfortably in under a second locally.  Overridable with
ZIGLER_TEST_TIMEOUT.

Windows drops to --max-cases 1: it OOM-killed the BEAM mid-run at 2, since each
concurrent zig build still peaks near 300MB with a warm cache.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ityonemo @claude
retries truncated sema output on windows
a2a98f5 
attempt_json/2 already retried when sema came back with no text at all -- a
long-standing windows flake.  It can also come back truncated, which sailed past
that check and blew up in Zig.Sema as a JSON.DecodeError partway through a full
test suite run.

The cause is not understood, and this does not claim to fix it.  Running the
failing module's sema executable 40 times by hand, and 40 more through
System.cmd/2, produced the complete 1348 byte document every time; it only goes
short during a full suite run, which builds hundreds of nifs back to back.  That
is a level of process churn no ordinary project generates, so a partial document
is now retried exactly like an empty one, and the retry budget goes from 5 to 10.
Sema is deterministic, so running it again gets the whole thing.

Worth recording for whoever picks this up: the "error: BrokenPipe" that shows up
alongside this is a symptom, not the cause -- it is printed after the elixir
stacktrace, when mix aborts and the still-writing sema process loses its reader.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ityonemo @claude
makes zig.warm actually warm the cache modules use
d62be67 
The warm task built from inside the staged cache directory, so its build.zig.zon
pointed at "../translate_c".  A module's generated zon points at
"../<fingerprint>/translate_c", because its staging directory is a sibling of the
staged tree rather than inside it.

Zig's cache key covers the source paths, so those two spell the same tree
differently and hash differently: warming produced a cache entry nothing ever hit,
and the first nif compiled rebuilt translate-c anyway.  On a windows VM with a warm
cache that rebuild still died with "LLVM ERROR: out of memory", which is what sent
me looking.

The warm task now builds from a sibling of the staged tree, at exactly the depth a
module's staging directory sits, and derives the relative path the same way
Zig.Builder does.  Verified: after warming, a module build performs zero
"compile exe translate-c" steps and takes 31s rather than the 102s it takes cold.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ityonemo @claude
updates the 0.17.0 release notes
031a99f 
The entry was written partway through the upgrade and had drifted:

  * the elixir requirement moved from ~> 1.15 to ~> 1.18, which is a breaking
    change for anyone on 1.15 through 1.17 and was not mentioned at all
  * zig_parser, zig_doc and zig_get all moved in lockstep with the zig version
  * `mix zig.warm` is new and worth knowing about for CI
  * the macos/freebsd stacktrace and windows cross-compilation fixes were missing
  * including priv/erl_nif_win in the hex package was credited here, but that was
    #596 on main, not part of this release

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ityonemo @claude
removes Zig.Analyzer
9a5aa3e 
Nothing calls it.  Both of its public functions -- info_for/2 and
translate_location/3 -- have no callers anywhere in lib/ or test/, and the module
is `@moduledoc false`, so it was never public API either.

translate_location/3 parsed the `// ref <file>:<line>` comments that zigler emits
ahead of each ~Z block, mapping a line in the generated zig back to the .exs it
came from.  Zig.Manifest does that now, through a resolver generated per module,
so this was a leftover from the earlier design.

Closes #589, which asked for info_for/2's results to be cached: there is nothing
to cache when nothing calls it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ityonemo @claude
notes the Zig.Analyzer removal in the release notes
e35a497 
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@ityonemo