Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 15 additions & 4 deletions docs/perf.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,10 +110,21 @@ Current status:
- `js_of_ocaml` is faster than upstream on nearly all cases; exceptions are
`add-all` at size 10000 (2105 ms vs 941.57 ms) and `q3` at sizes 1000 and
10000 (1.61 ms vs 1.31 ms at 10000).
- `js_of_ocaml` overflows the default Node.js stack at size 10000
(`RangeError: Maximum call stack size exceeded` during `add-all`). The
numbers above were measured with `node --stack-size=8000`; this is a known
js_of_ocaml recursion limitation, not a benchmark artifact.
- `js_of_ocaml` used to overflow the default Node.js stack at size 10000
(`RangeError: Maximum call stack size exceeded` during `add-all`), so the
size-10000 numbers above were measured with `node --stack-size=8000`. The
recursion was `List.remove_assoc` — non-tail in the OCaml stdlib — walking
the per-transaction tempid order list in `ensure_current_tx_tempid`
(`impl/transact.ml`), one entry per transacted entity (~10000 frames plus
`caml_compare` calls). A tail-recursive `remove_assoc` now ships in the
`Datascript_types.List` shadow (`type/datascript_types.ml`), next to the
existing `concat_map`/`concat`/`flatten` overrides, so native, Melange,
and js_of_ocaml all get the bounded-stack version. Verified: the full
size-10000 suite now completes under `node --stack-size=200`, and
`add-all` at size 10000 measured 764–923 ms (jsoo) vs 245 ms (native) on
the same machine (before the fix: 825 ms jsoo / 245 ms native, measured
with the default Node stack which happens to fit the ~10k-deep recursion
on Node 24).
- `storage-roundtrip` has no upstream equivalent (upstream bundle is in-memory
only), so it is reported without a comparison.
- `get-page-data` models Logseq's `logseq.api.db-based.tools/get-page-data`:
Expand Down
20 changes: 16 additions & 4 deletions type/datascript_types.ml
Original file line number Diff line number Diff line change
Expand Up @@ -534,10 +534,14 @@ type tx_report =
(* The Melange stdlib compiles List.concat_map/concat/flatten into recursive
JS calls whose depth grows with the input (concat_map once per element
mapped to [], concat/flatten once per sublist), which overflows the JS
call stack on data-sized lists such as query binding sets. These
tail-recursive versions keep the exact Stdlib.List semantics on every
platform. Declared here so the universal `open Datascript_types` (and
`open Datascript`) shadows them module-wide. *)
call stack on data-sized lists such as query binding sets. js_of_ocaml
hits the same wall through the OCaml stdlib: List.remove_assoc is not
tail-recursive there either, and walking a data-sized assoc list (e.g.
the tempid order list, one entry per transacted entity) overflows the JS
call stack where native barely notices. These tail-recursive versions
keep the exact Stdlib.List semantics on every platform. Declared here so
the universal `open Datascript_types` (and `open Datascript`) shadows
them module-wide. *)
module List = struct
include List

Expand All @@ -550,4 +554,12 @@ module List = struct

let concat l = concat_map (fun x -> x) l
let flatten = concat

let remove_assoc x l =
let rec aux rev_prefix = function
| [] -> List.rev rev_prefix
| ((a, _) as pair) :: rest ->
if a = x then List.rev_append rev_prefix rest else aux (pair :: rev_prefix) rest
in
aux [] l
end
Loading