Make COST_PER_EC_OP_REPEAT measurable, and measure it - #233
Merged
Conversation
#226 asks for "an A/B on block 962,000, that constant the only difference". That A/B could not be run, and would have returned a confident null. ## Two reasons it could not be run 1. `COST_PER_EC_OP_REPEAT` was a hard `const`, unlike its four siblings, so varying it needed a rebuild per arm. Now read through `cost_const` like the rest; default unchanged. 2. `chunk-profile`'s cost-packed arm hardcoded `pack_chunks(&costs, nchunks, None)`. The memo term — and with it this constant — could never reach the partition it reports. Every value produced byte-identical output, which reads as "no effect" rather than "not exercised". Measured before the fix: 104,222 and 413,800 both gave straggler 1.2104, to the digit. It now honours `HAZYNC_PACK_KEYS=1`, matching the real packing path. ## What it measures Execute mode, block 962,000, 16 chunks, `HAZYNC_PACK_KEYS=1`, only this constant varying: value ratio straggler 104,222 0.249 1.7686 <- current 250,000 0.598 1.2192 275,000 0.658 1.1741 300,000 0.718 1.1389 <- best, and under the 1.163 line 390,000 0.933 1.1393 413,800 0.990 1.2038 baseline, memo pricing OFF (shipped default): 1.2104 ⛔ The current value is not merely unrefit. With memo pricing ON it is 1.7686 against a 1.2104 baseline — enabling `HAZYNC_PACK_KEYS` today would be a 46% REGRESSION, not an improvement. ## The cause is arithmetic, not physics 104,222 was 0.736 of the OLD `COST_PER_EC_OP` (141,612). That sibling was refit to 417,798 and this one was left absolute, so the RATIO silently became 0.249. Preserving the original 0.736 gives 417,798 x 0.736 = 307,500 — which is where the measured optimum sits. ⚠ I first derived ~413,800 from a profile: the liftx hint has collapsed `secp256k1_ge_set_xo_var` from 1,415,786,221 cycles to 27,771,835 (51x), so a repeat "should" save ~1% and price near parity. **That reasoning measured WORSE (1.2038) than the value it replaced.** The issue warned about exactly this, citing the Schnorr constant set by the same kind of inference. Recording it because the inference was plausible and wrong. Not changed here: the constant itself, and the `HAZYNC_PACK_KEYS` default. Both are decisions about what ships, and the numbers above are execute-mode cycles on one block, not GPU wall-clock. Refs #226. Claude-Session: https://claude.ai/code/session_01BGBba1FtGQjp2focJGWtjU
Second commit on this PR. The first made the constant measurable; this sets
it, and records why the other half of the obvious change is NOT being made.
## The constant
104,222 was 0.736 x the OLD `COST_PER_EC_OP` (141,612). That sibling was refit
to 417,798 and this one was left as an absolute, so the ratio silently became
0.249 — a value nobody chose. 350,000 is the measured optimum on block 962,000
and restores the ratio to the band the fit supports.
## ⛔ The gain does not generalise, so `HAZYNC_PACK_KEYS` stays OFF
Measured on a second block, and the direction REVERSES:
962,000 965,978
memo pricing OFF (ships) 1.2104 1.0812 <- best on 965,978
104,222 (current) 1.7686 1.9016
350,000 (962k optimum) 1.0806 1.1975 <- best on 962,000
On 962,000 enabling the memo path is 10.7% better; on 965,978 it is 10.8%
worse than simply not pricing repeats. A value tuned on one block does not
carry to the other, and neither does the decision to switch the path on.
So the "8 cards" reading from the 962,000 sweep is **withdrawn**. It was one
block, and the second one does not agree.
## Why refit at all, if the path is off
Because 104,222 makes the gate a landmine. Anyone setting `HAZYNC_PACK_KEYS=1`
today gets 1.7686 and 1.9016 — far worse than leaving it off, with nothing to
suggest the constant is why. At 350,000 the gate is merely not-a-win, which is
a much safer thing to leave lying around.
⚠ This is execute-mode cycles on two blocks, not GPU wall-clock, and the memo
model may simply be wrong rather than mis-parameterised: it prices a
per-chunk decompression memo whose cost the liftx hint has already collapsed
51x (`ge_set_xo_var` 1,415,786,221 -> 27,771,835). That would explain why no
value of the constant beats switching it off on 965,978.
Refs #226.
Claude-Session: https://claude.ai/code/session_01BGBba1FtGQjp2focJGWtjU
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #226. Makes the A/B the issue asks for possible, then runs it.
The A/B could not be run, and would have returned a confident null
Two reasons, both fixed here:
COST_PER_EC_OP_REPEATwas a hardconst— unlike its four siblings — so varying it needed a rebuild per arm. Now read viacost_const, default unchanged.chunk-profile's cost-packed arm hardcodedpack_chunks(&costs, nchunks, None). The memo term could never reach the partition it reports, so every value gave byte-identical output. Measured before the fix: 104,222 and 413,800 both produced straggler 1.2104 to the digit — a null that looks like "no effect" and is actually "not exercised". It now honoursHAZYNC_PACK_KEYS=1, matching the real packing path.The measurement
Execute mode, block 962,000, 16 chunks,
HAZYNC_PACK_KEYS=1, only this constant varying:baseline, memo pricing off (what ships today): 1.2104
⛔ The current value is not merely unrefit. With memo pricing on it gives 1.7686 against a 1.2104 baseline — enabling
HAZYNC_PACK_KEYStoday would be a 46% regression.✅ At ~300,000 the straggler is 1.1389, which is 5.9% better than the shipped baseline and under the 1.163 threshold the issue names for 8 cards. So the 3.4% the issue called "plausible but not promised" does appear to be there — but only with both the refit and
HAZYNC_PACK_KEYS=1.The cause is arithmetic, not physics
104,222 was 0.736 of the old
COST_PER_EC_OP(141,612). That sibling was refit to 417,798 and this one was left as an absolute, so the ratio silently became 0.249. Preserving 0.736 gives 417,798 × 0.736 = 307,500 — exactly where the measured optimum sits.⚠ My own inference measured worse than the answer
I first derived ~413,800 from a profile: the liftx hint has collapsed
secp256k1_ge_set_xo_varfrom 1,415,786,221 → 27,771,835 cycles (51x), so a repeat "should" save ~1% and price near parity. That measured 1.2038 — worse than the 0.736-era ratio it replaced. The issue warned about precisely this, citingCOST_PER_SCHNORR_OPset by the same kind of reasoning. Recording it because the inference was plausible and wrong.Deliberately not changed
The constant itself, and the
HAZYNC_PACK_KEYSdefault. Both are decisions about what ships, and these are execute-mode cycles on one block, not GPU wall-clock — though the default 1.2104 reproduces the documented 1.210 exactly, which is some evidence the pipeline is sound.