Compatability issues for use with Doenet

[siefkenj]

math-expressions v3: what DoenetML needs to adopt it

For: maintainers of Doenet/math-expressions
From: the DoenetML side of the integration
Against: main @ c110a56 ("Rust improvements (#82)") — re-verified after PR #82 merged
Date: 2026-07-31 (revised)

We have integrated math-expressions-js-compat into DoenetML end to end and run it against real
test suites. This document is the resulting punch list: every change we need on the
math-expressions side, with the evidence that produced it and the DoenetML call sites affected.

Everything below is reproducible — see §6.

Re-evaluation note. This document was first written against PR #82's branch. It has been
re-run against merged main. Results:

• Every item below still reproduces, verbatim. git diff c870af6..c110a56 touches only
  active-plans/*, normalize/{full,mod,simplify,special_values}.rs, lib.rs and
  tests/full_simplify.rs — no file behind any request. math.test.ts scores identically
  (47 passed / 32 failed) on both revisions.
• One new blocking defect surfaced, R13 — exact rationals are projected to f64
  when crossing to JS, so simplify() turns 1/2 into 0.5. We had not probed exact-value
  preservation before; the more capable simplify on main made it visible. We consider this
  the highest-priority item in the document.

Headline: the port is in much better shape than the remaining list suggests. Parsing,
equality, differentiation, LaTeX/text rendering, the reviver round-trip and the AST bridge all
work against real DoenetML documents. Of the 32 remaining failures, one missing API family
accounted for all 79 failures before we patched around it, and the rest cluster into a handful
of concrete defects. None looks deep.

---

1. Summary table

#	Item	Severity	DoenetML call sites
R13	Exact rationals decimalize crossing to JS (1/2 → 0.5)	Blocking, highest	every simplify
R1	Browser/worker WASM loading in the compat package	Blocking	every realm
R2	from_ast rejects boolean and null leaves — contradicts the package's own Tree type	Blocking	boolean-valued math
R3	Render options (padToDecimals, padToDigits, showBlanks, …) silently dropped	Blocking	9 / 9 / 11
R4	get_component() not implemented	Blocking	63 in 24 files
R5	Context-level operation family absent	Blocking	see §R5
R6	Expression#f() not implemented although the bridge exists	Blocking	~21
R7	Normalization passes that silently no-op	Blocking	3 user-facing features
R8	Caller-owned handles are never freed; Sym interner is append-only	Blocking	long-lived worker
R9	panic = "abort" + WASM32 stack safety	Blocking	adversarial input
R10	substitute_component() not implemented	Minor	1
R11	MathML input (fromMml / mmlToAst)	To confirm	0 found
R12	Publish v3 to npm	Process	—

§5 lists the divergences we are not asking you to change — we will adapt DoenetML instead.

---

2. Blocking items

R13 — Exact rationals are projected to f64 on the way out to JS

This is the item we would most like fixed, and we think it is a small change.

expr/serde.rs::number_to_js projects Number::Rat to the nearest f64:

fn number_to_js(n: &Number) -> Value {
    match n {
        Number::Int(i) => json!(i),
        // Exact rationals (§3a) and Big numbers project to the nearest f64 —
        // what the JS trees actually hold — so the tree fixtures and the
        // differential harness stay meaningful.
        Number::Float(_) | Number::Rat(..) | Number::Big(_) => f64_to_js(n.to_f64()),
    }
}

Consequences, all observed on main:

rust (1/2).simplify().tree          →  0.5                    js →  ["/",1,2]
rust (2/4).simplify().tree          →  0.5                    js →  ["/",1,2]
rust (1/3).simplify().tree          →  0.3333333333333333     js →  ["/",1,3]
rust (1/2+1/3).simplify().tree      →  0.8333333333333334     js →  ["/",5,6]
rust cos(pi/3).simplify().tree      →  0.5                    js →  ["/",1,2]
rust (2/4).simplify().toLatex()     →  "0.5"                  js →  "\frac{1}{2}"

Every rational-producing entry point is affected — simplify, evaluate_numbers, expand,
reduce_rational, together. Note the last one especially: together() exists to produce a
single fraction and returns a decimal.

Why the stated rationale does not hold for us. The comment says f64 is "what the JS trees
actually hold". For rationals that is not the case — the legacy library represents 1/2 as
["/",1,2], as the side-by-side above shows. The projection was reasonable when the consumer was
the differential fixture harness; it is wrong for a product consumer.

Why it is blocking for DoenetML. Fractions are a first-class teaching subject. A <math>
with simplify that renders 0.5 where the author wrote 1/2 is a wrong answer in a fractions
lesson, and it is silent. It also defeats your own structural-comparison criteria — ReducedFraction,
ImproperFraction, MixedNumber, ExactValue (STRUCTURAL_COMPARISON_PLAN F1) cannot mean
anything if simplify has already decimalized the value. Those criteria are the single feature
we are most looking forward to adopting.

It is also lossy: 1/3 → 0.3333333333333333 cannot be recovered. Internally the engine is
fine — (1/3+1/3+1/3).simplify() is exactly 1, and equals still answers correctly — so the
exactness is being discarded purely at the JS boundary.

Request: emit ["/", num, den] for Number::Rat. try_from_js already reads that shape back
(["/", a, b] → Div), and it is what the parser and the legacy library both produce, so the
round-trip closes. If the differential fixtures depend on the current projection, we would suggest
the fixtures normalize at comparison time rather than the library lying to every caller. Big is
a separate question we have no stake in.

R1 — Give the compat package a browser/worker WASM path

lib/_wasm.ts is node-only: it reaches the nodejs-target build through createRequire, and
its own header says the browser path is future work. DoenetML runs in a Web Worker and on the
browser main thread, so today the package cannot be consumed as published.

The WASM side is already fine — build-wasm.sh takes web as a first-class target and the
playground ships it. Only the compat package's loader is unwired.

What we did as a workaround: aliased lib/_wasm at bundle time to our own loader, which
drives the web build. That works, but it means depending on an internal module path — fragile
across upstream reorganizations, and not something other consumers should have to discover.

What we would like: a supported way to hand the compat layer an already-initialized WASM
module, e.g.

import { setWasmModule } from "math-expressions/wasm";
import * as glue from "math-expressions/wasm/web";   // the --target web build

glue.initSync({ module: bytes });
setWasmModule(glue);

or equivalently a math-expressions/web entry point that does this internally and exposes
init(bytesOrUrl).

Two constraints from our environment, both non-negotiable on our side:

1. Instantiation from bytes, never fetch. fetch is blocked for blob/data URLs in the VS
   Code web-worker extension host (DoenetML issue #1375). We inline the WASM as base64 and pass
   an ArrayBuffer, the same as our existing Rust core does.
2. Synchronous init must remain possible. The legacy API is synchronous, and DoenetML calls
   it from ~150 files with no await. initSync from inlined bytes preserves that in a worker
   and under node. (Browsers refuse synchronous compilation of a module this size on the main
   thread, so an async path must also exist — but the sync path is what makes the drop-in a
   genuine drop-in.)

Sizes, for reference: the web WASM is 1.3 MB (before wasm-opt, which was not installed
in our container), and our whole bundled engine including the inlined WASM is 1.18 MB
gzipped — against 1.1 MB for the JavaScript library it replaces. Size is not a problem.

R2 — from_ast rejects boolean and null leaves

packages/math-expressions-rs/src/expr/serde.rs::try_from_js handles Number, String,
Object and Array, and falls through to Err(format!("unexpected value {other}")) for
everything else. But the package's own Tree type declares booleans legal —
lib/math-expressions.ts:14:

export type Tree = number | string | boolean | Tree[];

and the legacy typings agree (index.d.ts:18). Observed:

me.fromAst(true)              → Error: unexpected value true
me.fromAst(false)             → Error: unexpected value false
me.fromAst(null)              → Error: unexpected value null
me.fromAst(["+", "x", null])  → Error: unexpected value null
me.fromAst(["and", ["=","x",1], ["=","y",2]])  → OK

So the boolean operators round-trip but boolean literals do not, which is an odd seam —
["and", true, false] is unconstructible. This surfaced in our run as six otherwise-unrelated
failures (display rounding, units, LaTeX state variables) all reporting unexpected value null.

Request: handle Value::Bool in try_from_js. For null, either accept it as whatever the
legacy library treated it as, or — if it is genuinely not a tree — say so in the error and
tighten the Tree type, so the contract and the implementation agree. Silent disagreement
between a published type and the runtime is the actual defect here.

R3 — Render options are silently dropped

Expression#toLatex() / toString() / tex() accept no parameters; the legacy API took an
options object. JS_RUST_DIFF.md notes this, but the practical impact is larger than it reads,
because the dropped options are student-visible on every rendered <math>:

js   me.fromText("2.5").toLatex({ padToDecimals: 3 })  →  "2.500"
rust me.fromText("2.5").toLatex({ padToDecimals: 3 })  →  "2.5"

The options are ignored rather than rejected, so this fails silently — a number renders with the
wrong precision and nothing anywhere errors.

DoenetML source (excluding tests and build artifacts) uses padToDecimals in 9 places,
padToDigits in 9, showBlanks in 11, and explicitMultiplicationSymbols in 1. Note that
packages/doenetml-worker-rust/lib-js-wasm-binding/src/eval-math.ts is one of the consumers:
the DoenetML Rust core itself passes these options through its JS bridge, so this also blocks
the Stage 2 plan of depending on math-expressions-rs as a crate.

Request: accept the options — either to_latex_with_options(json) at the WASM boundary
mirroring the existing parse_*_with_options, or absorbed in the compat layer. At minimum,
throw on unsupported options rather than ignoring them.

R4 — get_component() is not implemented

Currently in compat's notImplemented list. DoenetML calls it 63 times across 24 source
files — vectors, matrices, lines, line segments, math lists, discrete simulation results — and
in EssentialValueWriter.ts on the inverse-definition path, so it is load-bearing for editing,
not only display. Three of our 32 failures are directly this.

R5 — The context-level operation family is missing

The legacy library exposed every Expression method a second time as a free function on the
context, taking the expression first: me.simplify(expr) alongside expr.simplify(). Compat's
Context stops at the factories (fromText/fromAst/…) and assumptions.

This is the single highest-impact item in the document. Our first Rust-engine run failed all 79
cases in math.test.ts, every one of them on me.round_numbers_to_precision_plus_decimals.
Regenerating the family from the prototype took the same spec to 47 passing without touching
anything else.

Request: generate the family in the compat layer. Roughly what we did locally, which mirrors
how the legacy library defined it:

for (const name of Object.getOwnPropertyNames(Expression.prototype)) {
  if (name === "constructor" || name in Context) continue;
  const d = Object.getOwnPropertyDescriptor(Expression.prototype, name);
  if (typeof d?.value !== "function") continue;      // skip accessors (`tree`)
  Context[name] = (expr, ...args) => Context.from(expr)[name](...args);
}

Existing context members must win, so the factories are not shadowed by same-named prototype
methods. We are happy to send this as a PR if useful.

R6 — Expression#f() is not implemented, though the machinery already exists

f() is in the notImplemented list, but math-expressions-rs-wasm's compileRustExpr does
exactly this job. DoenetML calls .f() in ~21 places to get a numeric evaluator for jsxgraph
plotting and root/extremum finding. Wiring it took four lines on our side:

Expression.prototype.f = function () {
  const compiled = compileRustExpr(math, this._w);
  return (bindings) => compiled.evaluate(bindings);
};

Request: wire it in the compat package — the "compile once in JS, evaluate per sample with no
boundary crossing" design is exactly right and should be the shipped default.

R7 — Passes that silently no-op are worse than passes that throw

Compat defines these as no-ops returning this: default_order, normalize_negative_numbers,
normalize_applied_functions, expand_relations, applyAllTransformations. The rationale
(folded into canonicalize) is reasonable, but three of them are user-facing DoenetML
features, and silently returning the input makes the feature vanish with no diagnostic:

Pass	DoenetML surface	Evidence
default_order()	simplify="normalizeorder" (mathexpressions.ts:51)	js: "3+x".default_order() → ["+","x",3]; rust → ["+",3,"x"]
evaluate_numbers({skip_ordering:true})	simplify="numberspreserveorder"	js: "1+x+2" → ["+",1,"x",2]; rust → ["+","x",3] (option ignored)
normalize_applied_functions()	answer grading (checkEquality.js:90)	no-op

The middle one is evaluate_numbers(_opts) ignoring its argument — same class of bug as R3.

This is also what produced our term-ordering failures, e.g. insu＿ vs sinu＿.

Request: either implement these (default_order and the skip_ordering option are the ones
we actually need), or make them throw so consumers discover the gap at the call site.
Answer grading silently changing behavior is the failure mode we most need to avoid.

R8 — Handle lifetime

The README is explicit that caller-owned handles are never freed, and ARCHITECTURE_REVIEW.md
notes the Sym interner is append-only with no cap. DoenetML's worker is long-lived and mints
expressions per state-variable evaluation and on every state JSON round-trip (our
serializedComponentsReviver calls fromAst on the way back in), so memory growth is unbounded
in a normal session.

We note the playground already frees deterministically via freeHandle with a
__wbg_ptr !== 0 guard, and that what corrupted the heap was FinalizationRegistry, not
explicit free().

Request, in our order of preference:

1. A value-first Expression — hold the plain Tree as canonical state and materialize a
   handle only for the duration of an operation. This also removes the cost of the uncached
   get tree() { return JSON.parse(this._w.tree_json()); }, which matters to us: fromAst and
   .tree are our two hottest calls (~675 and ~600 source sites), sitting in dependency-graph
   hot loops where the legacy library made them free property access.
2. A public free()/dispose() on the compat Expression, so hosts can own lifetimes.
3. A cap or eviction policy for the Sym interner — this one no consumer-side discipline can fix.

R9 — panic = "abort" and WASM32 stack safety

With panic = "abort", a reachable panic kills the worker and takes the student's session state
with it, where the JavaScript library raised a catchable exception. ARCHITECTURE_REVIEW.md
still lists reachable panic sites (Number::from_decimal_str), and STACK_SAFETY_PLAN items 21
and 23–26 are open — deep expressions can overflow the ~1 MB shadow stack, including on
Drop. Student input is adversarial by construction.

Request: the panic firewall (Result-ify the boundary) and stack-safety item 21 in
particular. Item 21 gets more important if R8 is solved by freeing aggressively.

---

3. Minor / to confirm

R10 — substitute_component()

One source call site (Math.js). Low
priority, but currently throws.

R11 — MathML input

WHATS_LEFT.md §A.1 lists mmlToAst as the one converter gap that is not marked "not needed
for Doenet". We found no source call sites for fromMml/mmlToAst in DoenetML. We are
fairly confident it is not needed, but flagging it so the claim is on record rather than assumed.

We also confirm from our side that GLSL, Guppy, MathML output and mathjsToAst have no
DoenetML consumers — your "not needed for Doenet" annotations for those are correct.

R12 — Publish v3

npm view math-expressions dist-tags still shows latest: 2.0.0-alpha94, and
packages/math-expressions-js-compat/package.json is still at 3.0.0-alpha1 on main. We are
consuming the repo as a git submodule for now, which is fine for development. A 3.0.0-alpha
prerelease tag would let us pin normally and would make the drop-in testable by anyone else.

---

4. What already works

Worth stating plainly, because the list above is all problems:

• Text and LaTeX parsing, equals, simplify, expand, derivative, variables,
  evaluate_to_constant, substitute, toLatex/toString (modulo R3), subscripts_to_strings,
  to_intervals, tuples_to_vectors, the assumptions surface, and match all behave against
  real DoenetML documents.
• me.reviver and toJSON preserve the {objectType: "math-expression", tree} shape, so
  DoenetML's state serialization round-trips unchanged — this was a risk we expected to have to
  work through and did not.
• me.math, me.converters.{textToAstObj,latexToAstObj} (with the options object), me.class,
  me.utils and isTree are all present and correctly shaped.
• The web WASM build, and the playground's engines.ts, were accurate and complete enough that
  we got a browser-target engine running without needing to ask a single question. The
  wasmApi.ts reflection trick over the generated .d.ts is a genuinely good idea and we intend
  to reuse it for conformance checking.

---

5. Divergences we are absorbing — no change requested

We are not asking you to preserve legacy behavior in these cases. Recording them so they are
not mistaken for bugs later:

• The more aggressive simplify (PR Rust improvements #82) is a feature. exp(ln x) → x, sin(2π) → 0,
  tan(π/4) → 1 (where the JS library returned 0.9999999999999999) and like-term collection are
  improvements over what the JavaScript library could do. It changes results in DoenetML's
  answer-checking and display paths — e.g. a test expecting the uncollected
  -2x^{2}+x^{2}+5x^{2}-2 now gets 4x^{2}-2 — and we will carefully update our tests and
  documented behavior to match, rather than asking for a less capable mode.

  This endorsement is about which simplifications happen, not about the form of the
  result. cos(π/4) → √2/2 is exactly right and shows the exact engine working. cos(π/3) → 0.5
  is the same feature spoiled on the way out (R13). We want the aggressive simplifier
  and exact output; they are independent.

• Exact-constant equality (equals(1/2, cos(π/3)) → true) — likewise an improvement for
  grading.

• Formatter differences that are pure presentation and not governed by an explicit option
  (the clean-slate printer's spacing and form choices). We will renormalize our assertions to
  compare parsed trees or use equals rather than exact strings, so our suite stops being
  coupled to one formatter.

The distinction we are drawing: a better default is welcome; an explicitly requested
transformation that silently does nothing (R7) or an explicitly passed option that is ignored
(R3) is not.

---

6. Reproducing our results

On the DoenetML side (branch new-math-expressions):

git submodule update --init --recursive   # vendor/math-expressions @ Doenet/main (c110a56)

# Build the seam against the Rust/WASM engine (requires a Rust toolchain,
# wasm32-unknown-unknown, and wasm-bindgen-cli matching the pinned wasm-bindgen =0.2.126)
DOENET_MATH_ENGINE=rust npm run build -w packages/math

# 10-case smoke suite — passes on both engines
npm run test -w packages/math

# The spec that produced the numbers in this document
npx vitest run --root packages/doenetml-worker-javascript src/test/tagSpecific/math.test.ts
#   JavaScript engine: 79 passed / 1 skipped
#   Rust engine:       47 passed / 32 failed   (identical on c870af6 and c110a56)

The 32 failures attribute as: 15 assertion divergences (term ordering from R7, formatter
differences we are absorbing), 6 × R2, 6 × R4, 5 × decimal-vs-exact rendering
(R13) and padding (R3).

DoenetML routes all ~150 math-expressions imports through one package, @doenet/math
(packages/math), which selects the engine at build time. The gap fills for R5 and R6 live in
packages/math/src/engine-rust.ts and the R1 workaround in packages/math/src/wasm-loader.ts —
all three are patches we would rather delete than maintain, and are offered upstream.

Full context: MATH_EXPRESSIONS_RUST_MIGRATION_PLAN.md in the DoenetML repo, which also covers
Stage 2 — depending on math-expressions-rs as a Cargo dependency of lib-doenetml-core and
deleting DoenetML's current Rust→JS math bridge (math_via_wasm.rs, which today calls the
JavaScript library out of Rust through a __forDoenetWorker global and passes ASTs as JSON
strings). R3 in particular blocks that work as well as this.