Repository navigation
feat(checker): arrow tasks with captures, tasks without a destination, and a seeded Rng - #644
Merged
Merged
Conversation
…losing function's consts
The Rayon one-liners are now spellable with `nish/threads`: a new array
from a map, a new array from an index, and a body that reads the
function's local constants.
```ts
const ys = parallelMap(xs, (x) => x * k + b) // k, b: const locals
const sq = parallelFrom(n, (i) => i * i) // Array.from({ length: n }, ...)
parallelMapInto(xs, dst, (x) => table[x % 8] * k) // table: a const array, read-only
```
`parallelMap(src, f)` and `parallelFrom(n, f)` allocate their result on
the calling thread before the region and then run the existing map:
`parallelMap` is `parallelMapInto` into that array, and `parallelFrom`
the same region over the indices through a chunk loop of its own. Their
`f` is held to P1's rules unchanged, and a result that is not a number,
a `boolean` or an enum is refused at the call, before `std/threads.ts`
is instantiated for it.
An arrow passed to any of the four forms may now read the `const`
locals and the parameters of the function it is written in. Each name
it reads is appended to the arrow's parameters; every instance between
the call and the body carries the value as a parameter of its own, and
the region packs it into the context the runtime already passes. A
scalar is copied and a reference is read-only: a store through it is a
shared write and is refused as one, through an alias, a field, an
element, a helper or a method included. A `let` is refused with a
message telling the user to bind a `const`, as are `this`, a read from
outside a nested arrow, a lock, a guard, a channel, a scope and a
`Result`. In `parallelMapInto`, a capture that could reach `dst` is
refused, except a captured `const` array of its own beside a `dst` that
is another such `const`. No closure is made, and no other arrow
captures: the arrow is still a function of its own with a direct call.
Refs: docs/LANGUAGE.md#a-data-parallel-arrow-reads-its-functions-const-locals
Tests: tests/link/par_capture, tests/link/par_capture_large, tests/link/par_map_new, tests/link/par_from, tests/cases/reject_par_capture_*
Part of #575
…text loop keeps its length facts The loop that splits a region's arguments into its context fields read each element after a push, which ends the length facts, and kept two bounds checks the performance ratchet counts against src/.
…ete/data-forms # Conflicts: # tests/self/goldens/checked-self.txt # tests/self/goldens/checked.txt
…ete/data-forms # Conflicts: # docs/AI.md # tests/self/goldens/checked-self.txt
…function's locals Every rule about a local of the enclosing function now sees a capture as a read of that local where the call runs the body: a captured parameter keeps a link to the local it was made from (`Local.captureOf`), the walks that compare locals resolve through it (`sourceLocal`), and the walks that pass over an arrow read its captures at the arrow (`capturedReads`). That closes a scope destination read through `parallelFrom` (NL2394), a stale inline record reference read after a push or a pop (NL2290), and an array read after `secret(raw)` took it (NL2445). `dst` is resolved where the call is checked, in a generic caller's own tables, so two fresh `const`s are no longer refused there, and a parameter array beside a fresh `dst` is admitted: a fresh array is memory no older binding held.
…ete/data-forms # Conflicts: # tests/self/goldens/checked.txt
…ete/data-forms # Conflicts: # tests/run.js # tests/self/goldens/checked-self.txt
…tability warnings The portability rows that follow a local through its reads skipped an arrow argument's body, so a record copied into an array (NL8003) or a byte offset compared with a count (NL8001) went unreported when the read was a capture inside an arrow given to `parallelMap`, `parallelFrom`, `parallelMapInto` or `parallelReduce`. A captured read now counts as a read of the local where the call runs the arrow, as it already did for the scope, element-reference and `Secret` rules. A refused region call inside a scope also reports its arrow's captured destination read, and NL2494 says that `this` is refused for its spelling: `const self = this` is captured. Refs: docs/LANGUAGE.md#a-data-parallel-arrow-reads-its-functions-const-locals Tests: tests/link/port_rec_capture_copy, tests/link/port_str_capture_offset, tests/link/port_rec_capture_quiet, tests/cases/reject_par_capture_*
…ete/data-forms # Conflicts: # tests/self/goldens/checked.txt
…low an offset out of its answer NL8009 lined a function's written parameters up with its parameter types from the end, and an arrow handed to a data-parallel form has the names it captures appended after the written ones, so each was checked against another's type: a false warning beside a captured `i64`, a missed one beside a captured `i32`. The alignment now ends at the written parameters. NL8001 followed a captured byte offset into the arrow's `return` and read it as `main`'s exit code, and lost it at a concise body. A value an arrow answers with is now stored in the array its form writes, and `parallelReduce`'s answer is followed on from the call. Refs: docs/LANGUAGE.md#a-data-parallel-arrow-reads-its-functions-const-locals Tests: tests/link/port_num_capture_wide, tests/link/port_str_capture_result
…ete/data-forms # Conflicts: # tests/self/goldens/checked.txt
…ete/data-forms # Conflicts: # tests/self/goldens/checked-self.txt # tests/self/goldens/checked.txt
…ete/data-forms main took NL2492-NL2499 for typed JSON's options (#630), so the data-parallel capture refusals move to NL2500-NL2505, in the same order.
…ete/task-forms # Conflicts: # docs/LANGUAGE.md # docs/guide/using-nish/threads.md # src/arrays.ts # src/codes.ts # src/context.ts # src/emit-parallel.ts # src/expressions.ts # src/generics.ts # src/parallel.ts # tests/self/goldens/checked-self.txt # tests/self/goldens/checked.txt
…apture loop out of the ratchet
…ete/task-forms # Conflicts: # docs/guide/using-nish/threads.md # tests/self/goldens/checked-self.txt # tests/self/goldens/checked.txt
…sk made itself is a shared write when the caller's memory may be there
…ete/task-forms # Conflicts: # tests/link/caps_parallel/expected.caps.json # tests/link/panics_parallel_length/expected.panics # tests/self/goldens/checked.txt
…d a second declarator reject_thread_task_private_object_array_own now gets the task's argument into its own array by pushing a holder made around it, rather than by a direct store that the slot-write rule refused on its own; it compiles without reachesForeign and is refused with it. reject_thread_task_private_object_declarator reads the holder's field back in the statement's second declarator. Unused imports leave two cases.
…ete/task-forms # Conflicts: # src/codes.ts # tests/run.js # tests/self/goldens/checked-self.txt
This was referenced Oct 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Part of #575: stage
task-formsof "Threads to completion" (WP29 P2).s.spawn(() => work(a))is now the whole spelling of a task, and parallel code has random numbers that match under Node.Changes
liftArrowArgument/captureOuterinsrc/generics.ts). An arrow given tospawncaptures as feat(checker): parallelMap, parallelFrom and arrows that read the enclosing function's consts #609's data-parallel arrows do. Each name it reads is appended to its parameters, and every instance carries it as%cap.N. The method-call path now passes captures as well (Emitter.captureOperands,emitCallinsrc/emit-classes.ts). Each capture is copied into the task's payload after the written operands, and$runhands it back (emitSpawnTaskinsrc/emit-parallel.ts). NL2391 ("must be a top-level function named at the call, not an arrow") is retired: the lifted arrow is named after its function (main$arrow0), which is what a debugger shows for the thread.src/parallel.ts):handOnCaptures). Writing them before the join, by any name, helper ornew, is therefore refused, and a destination cannot be captured.RegionState.taskArgsnow holds the capture types too, with their names (taskArgRoots).taskMutexMessageForwalks each capture (NL2510).taskArgumentMessage,recordSendersandhandsNullableChannelread the task's argument throughtaskArgTypeOf, which is -1 for a thunk.Mutex. It may not capture a guard, a channel (its senders are counted from the argument), a scope, aResult, a value that may be absent or aSecret(NL2509,captureObstacle(…, task)). Alet(NL2506),this(NL2507) and an assignment (NL2508) each get the task's own wording.RefWalk.holdTaskInputs/readHeldinsrc/arrays.ts). A task reads its argument and its captures when the scope joins. So a name it was handed is read again at the end of the block that declares the scope, and apushbetween the spawn and the join that may move an element it refers to is refused. This also closes the same gap for a task's argument (reject_thread_task_arg_stale_join), which compiled before.ThreadScopegetsspawnWith(entry, arg)andspawnThunk(entry), and the checker picks one by argument count (spawnFormFor,src/members.ts). Their tasks (runWith,runThunk) are filed likerunTask's, and their$finishreturns without storing anything. An entry that answers a value is NL2511. Under Node every form reachesspawn, whose new first branch (slotWritten(at), never true natively) runs a task with no slot at once. That branch moves the IR of every program that spawns; nish-cmp reports 0 undeclared differences.RegionState.live()now means "a spawn on this scope has been passed" (started), so a scope whose tasks have no destinations still has a region.composeArgumentsinsrc/attributes.ts). An argument edge rooted at the function's own object (ROOT_OWN) marks no shared write when nothing the callee writes can be anything but that object:const r = new Rng(seed); r.nextI32(). When the path has steps (h.b,hs[0]) or the callee writes below its parameter's slots, and aDirtMarksays the caller's memory or memory of unknown origin may be there, the write is a shared one (reachesForeign, thenmarkPastWrite). Round 1 (H1): the first head skipped everyROOT_OWNedge, sohs.push(box); hs[0].bump()andh.r = shared; h.r.nextI32()set onlywritesReach, which the task and body rule never reads, and raced. Each of the reviewer's shapes and its siblings is now a negative, and each one compiled on a compiler built without the fix (reject_thread_task_private_object_element,_field,_ctor_alias,_declarator,_capture,_array_rng,_array_own,reject_par_private_object_field_rng,_array_rng,_for_of); the siblings (_map,_set,_nested,_forward,_forward_store) are refused by both. Round 2:_array_ownnow gets the argument into the array by a push ofnew HB(box)rather than by a direct store, whichnoteSlotWriterefused without this rule;_declaratorreads the holder's field back in the statement's second declarator (const h = new HB(box), b = h.b).reject_par_private_object_heldand_keptpin the refusal once the caller's array is inside the object. The--emit-checkeddumps move where a function's only shared write was into its own object (dump_shared_write_field,dump_checked,checked.txt).std/random.ts.Rngholds onei32of state. Each step adds the Weyl increment 0x9E3779B9 and answers MurmurHash3'sfmix32of the new state.nextF64builds 53 bits from two steps. The two wraps arewrappingAddandwrappingMulfromnish:unsafe, which Node answers with| 0andMath.imul, sotests/link/std_randomprints the same lines both ways. I used plain xorshift32 first, but seeds 1, 2, 3 gave visibly correlated first draws, because xorshift is linear in its seed. MurmurHash3 is public domain, and its constants are marked where they appear;THIRD_PARTY_NOTICES.mdlists it. Registered instdModuleNamesand inrelease.yml's three presence gates.docs/LANGUAGE.md("Scoped tasks": the three forms and the capture rules with their cases; "Nor is a write into an object's own slots, when the object is one this call made"; "Random numbers:Rnginnish/random"),docs/IR_COOKBOOK.md(thread-task-arrow,std-random),docs/guide/using-nish/threads.mdandstandard-library.md,docs/RUN_UNDER_NODE.md,docs/wp29-thread-surface.mdandstd/README.md.reject_thread_task_arrow,reject_par_capture_spawnandnl2391_thread_task_arrow, whose programs now compile. NL2391 is listed as retired intests/wordings/unreachable.txt.Negatives, one per route
All are
tests/cases/reject_thread_task_capture_*unless named otherwise:_parent_write_alias_helpernewwriting it:_newthishanded out of a constructor:_this_escapeusing:_declarator_dst_task_writenewrun inside the task:_task_newthishanded out of a constructor inside the task:_task_this_escape_mutex_lock, R8_mutex_deep(NL2510)_guard,_channel,_secret_or_return_stale_record,_stale_join, andreject_thread_task_arg_stale_joinfor the argument_secret_moved_let(NL2506),_this(NL2507),_assign(NL2508)_nested(NL2501)reject_thread_task_void_resultand_arrow(NL2511)reject_thread_task_private_object_element,_field,_ctor_alias,_declarator,_capture,_array_rng,_map,_set,_nested,_array_own,_forward,_forward_store;reject_par_private_object_field_rng,_array_rng,_for_of,_held,_kepttests/wordings/nl2506–nl2511pin each new message whole.The IR
The full modules are the committed goldens. Below is what each new program adds.
tests/link/thread_task_arrow(main.ll): the captures passed at each spawntests/link/thread_task_arrow(threads.ll): a thunk task and its payloadtests/link/thread_task_void(threads.ll):s.spawn(entry, arg), and a sender counted from its argumentEvery program that spawns (
threads.ll): the branch only Node takestests/link/std_random(random.ll)The
std_randomparallel body'snew Rng(i + 1)stays inside the element (no NL9012), and its tasks lock only to add a count.Testing
On this head, with main at
ce97fa2(#621, #639, #645, #647, #640) merged in, each command was run bare and its exit status read:npm run check,npm run lint,npm run lint:dead: exit 0WASI_SYSROOT=/usr npm test(wasi-libc andlibclang-rt-18-dev-wasm32installed), with LLVM 18 onPATHand noDEGRADED:line: 18,361 passed, 0 failed, 1 skipped, exit 0. The skip is the aarch64 static link (nocrt1.ohere), as on feat(checker): parallelMap, parallelFrom and arrows that read the enclosing function's consts #609node tests/run.js performance: the ratchet matchestests/perf-baseline.jsonexactly, with no new NL9007 or NL9011node docs/check-links.mjs: okruntime/*.cchangenode scripts/gen-diagnostic-codes.mjs --checkanddocs/cookbook/regen.sh --check: exit 0nish-cmpagainst the 0.18.0 seed (run bynpm testwithNISH_BOOTSTRAPset): 0 undeclared differencesnode tests/self/goldens.js --update,node tests/run.js <filter> --update-link-goldens,docs/cookbook/regen.shandnode tests/differential/goldens.js --update, never by hand; the diffs were read. After fix(checker): refuse an exported function that takes a C symbol, and spell private ones apart #640,checked-self.txtmoves byCheckContext's new field, and fix(checker): refuse an exported function that takes a C symbol, and spell private ones apart #640's two new thread link programs (thread_no_strict_hidden_task,thread_private_prefixed_names) takespawn'sslotWrittenbranch in theirthreads.ll. No new test program orstd/random.tsexports a C-reserved name (NL3037)thread_task_arrow,thread_task_voidandstd_random)src/'s own source (the rolling freeze)ROOT_OWNbranch incomposeArgumentsrestored to a barecontinuecompilesreject_thread_task_private_object_element,_field,_ctor_alias,_declarator,_capture,_array_rng,_array_own, andreject_par_private_object_field_rng,_array_rng,_for_of. This head refuses each one with its pinned.err("throughBox.bump" or "throughRng.nextI32"), so reverting the fix fails those cases. The siblings (_map,_set,_nested,_forward,_forward_store) are refused by both builds.Related issues
Part of #575.
The squash commit, which is the changelog entry: