diff --git a/AGENTS.md b/AGENTS.md index cea395ea..38de33fe 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,7 +1,7 @@ # netherite (AGENTS.md) -Home: **Anvil-primary** - canonical at `anvil:~/dev/netherite`. Build and run -here. MacBook is control plane / Moonlight / image viewing only. +Home: **Anvil-primary** - canonical at `anvil:~/dev/netherite`. Build, run, +and validate here. MacBook is control plane / Moonlight / image viewing only. Git remote: **https://github.com/Infatoshi/netherite**. That is the only GitHub repo. `origin` on both machines points there. @@ -14,10 +14,20 @@ next to the code they govern. | OS | Role | |----|------| -| **Linux x86_64** | Full stack. Build CPU/CUDA, run Java oracle, train blaze, sweep. Canonical host: anvil (Ubuntu). Needs JDK 8 + NVIDIA CUDA for GPU paths. | +| **Linux x86_64** | Full stack. Build CPU/CUDA, run Java oracle, train blaze, sweep. Canonical host: anvil (Ubuntu, 9950X3D). Needs JDK 8 + NVIDIA CUDA for GPU paths. All Linux validation runs here. | | **macOS** | Build and verify the CPU and Metal game backends. Also serves as the control plane and image/video review host. Do not expect native `runClient` or CUDA here. | | **Windows** | Not a supported build/run host for this monorepo. Use WSL2 Linux if you must, or a remote Linux box. | +### Hosts (where work runs) + +| host | Role | +|------|------| +| **anvil** | Canonical clone. Linux validation: `make test`, magma CPU, tape replay, blaze M1/M2, `eval` (cpu/cuda/magma), oracle capture. CUDA `sm_120`. Shared GPUs: `nvidia-smi` first; long GPU jobs through `overnight-compute`. | +| **macOS** | Control plane. Metal build, `cpu==metal` kernel half, image/video review. Not the Linux validation host. | +| **gamer** | Not the validation host. Use only when a job needs the 3090 / `sm_86`. | + +Do not stage magma CPU gates, 13-seed eval, tape replay, or `make test` on gamer. Anvil is the faster CPU. + Prism / MultiMC / official launcher: optional jar source for assets. Fresh boxes do **not** need Prism credentials; `make -C java bootstrap-oracle` pulls MC 1.11.2 via ForgeGradle (you must own the game). @@ -65,9 +75,10 @@ sub-agents pointed at them without waiting for a human to pick items: 1. Pick from `magma/OPEN_DIVERGENCES.md` class A (top down) and `blaze/OPEN_DIVERGENCES.md` unported rows (dependency order). One lane per item. Skip items whose evidence (tape, golden) is not on this Mac. -2. Stage: `bash scripts/lane_stage.sh [--tape NAME]...` - (gamer for magma CPU work; anvil for oracle captures, CUDA, blaze M2; - Metal half of a twins lane builds on the Mac). +2. Stage: `bash scripts/lane_stage.sh anvil [--tape NAME]...` + (anvil for magma CPU, oracle captures, CUDA, blaze M1/M2, eval. + Metal half of a twins lane builds on the Mac. Do not stage + validation on gamer.) 3. Prompt = `docs/SUBAGENT.md` + a goal block: the item text, the gate command, the documented baseline numbers, the hard goal, and what is forbidden. Launch from the lane worktree, in the background. diff --git a/blaze/OPEN_DIVERGENCES.md b/blaze/OPEN_DIVERGENCES.md index f0dee68a..386e72d5 100644 --- a/blaze/OPEN_DIVERGENCES.md +++ b/blaze/OPEN_DIVERGENCES.md @@ -16,6 +16,7 @@ Definitions, so the list stays honest: - Gate runner: `blaze/env/port_matrix.py` over `blaze/env/port_matrix.yaml` (fail-closed; VERIFIED / BLOCKED / FAILED per row and tier). +Last documented: 2026-08-26 magma 13-seed transfer (`retrain_0821_best.bin` closed == cpu == cuda, 0/13 torches; replay MATCH 56/65. `ppo_ckpt_best.bin` closed t0:13; replay MATCH 45/65. Cam-px DIVERGE not blessed). No new M1/M2. Last verified: lane/blocklight 2026-08-25 (Sweep row 3: World.checkLightFor BLOCK flood. test_blocklight matches magma Manhattan 0..14. Fluids M2 raw/warp/scalar VERIFIED after moving CU_LIGHT_Q off the CUDA thread stack. listed --no-deps M1 27 VERIFIED; M2 27 VERIFIED; mining_slice M1+M2 BLOCKED rc=3. Root make test PASS. SNAP v11 unchanged. No BP_ light digest.) Last verified: lane/potions 2026-08-24 (Sweep row 13: SNAP v11 potion trailer after resume v10. Magma skip double fire decrement on shared hostile path. potions M1+M2 VERIFIED 120 raw/warp/scalar + resume N=90 M=30 dump version=11. listed --no-deps M1 27 VERIFIED; M2 27 VERIFIED; mining_slice BLOCKED: s14_t0_r48_no_liquid.bsnp is a v1 bake, 12 *_d*.bsnp present. Root make test PASS.) Last verified: lane/resumegate 2026-08-23 (Sweep rows 4 and 8: snapshot v10 resume trailers + mob sidecars. continuous-vs-resume BP_ gate. listed --no-deps M1 VERIFIED; M2 VERIFIED raw/warp/scalar except mining_slice BLOCKED missing *_d*.bsnp). @@ -122,15 +123,55 @@ Two consequences worth stating plainly: 48-point path cap), IntHashMap aliasing reproduced, all 8 detmob tapes must be blaze-exact. +## Spawn -> dragon (policy vs magma) + +A policy in blaze CUDA cannot complete magma `PRODUCT.md` steps 3-9 +(Nether portal, fortress/rods, eyes, stronghold, End, dragon, `won`) +while matching magma. Gate 2 accepts spawn->torch. `docs/GATES.md` +forbids dragon-fight RL. Isolated kernels (`nether_portal.h`, +`end_portal.h`, `nether_full.h`, `chunk_provider_nether.h`, +`ender_dragon.h`) are CPU==CUDA units; they are not `blaze_tick`. +Java-vs-magma leftovers on that same route stay in +`magma/OPEN_DIVERGENCES.md` (stronghold L, tick order M, detmob nether +A*, class A pixels). Do not file missing Nether/End in this env as a +magma bug. + +| PRODUCT step | Blaze env | Where | +|---|---|---| +| Overworld spawn, dig, craft, furnace, chests, fluids, TNT, overworld hostiles | yes, M1+M2 inside a `.bsnp` | Verified rows | +| Food / bed / buckets / flint / portal ignite | buckets tick but are not in `rl_crafts`; fire->portal follower not ported; `dimension` is always 0; id 51 edits unreachable | `blaze_core.h` (`cu_world_set_state` follower); sweep 2 | +| Linked Nether / End | no | unported `portals_dimensions`, `nether_route` | +| Fortress, blaze, ghast, pigman, magma cube, wither skeleton, silverfish | type tags exist (`entity_hostile_spine.h`); spawn table and live AI in the env are overworld only | `hostile_spawn.h` `hs_to_ew`; `blaze_core.h` header | +| Throw eyes of ender / fireballs | magma-only | projectiles close note | +| Stronghold hunt, end-portal frames, enter End | no dimension in the env. Magma vs Java stronghold is magma OPEN sweep 1 | magma `OPEN_DIVERGENCES.md` | +| Dragon, crystals, bed explode in DIM -1/1, exit portal, `won` | `ender_dragon.h` is a 200-tick fixed arena (no block destruction, no contact damage), not wired into `blaze_tick`. Bed explode in Nether/End is magma-only | unported `dragon_victory`; `player_bed.h` | + +Even on overworld snapshots, a trained policy does not see or act as +magma `player_ctl`: + +- Sweep 1: episode region is the snapshot (`cu_region_idx`, + `cu_recenter`); magma streams chunks. L. +- Sweep 2: no strafe, sneak, sprint, or slot-click. Camera planes are + log / leaves / coal / stone / dirt / table / occupancy / depth / + edge (`obs_pack.h` `pack_frame`). No mob, light, or health plane. +- Sweep 6: death is terminal. +- Detmob A* is magma-CPU (above). +- M1/M2 are BP_ digests inside that region, not a streamed survival + world. + +Closing the port-matrix DAG does not by itself make a dragon policy: +sweep 1 and 2 still block long-horizon transfer. An isolated kernel +PASS is not env support. + ## Sweep 2026-08-23 (codex full read, magma -> blaze; unverified by gate unless noted) Ranked by RL fidelity. Each row: magma site; blaze site or absent; what closing needs. Spot-checked by hand on 2026-08-23: rows 7 and 10 confirmed. -1. Episode region is fixed; no chunk streaming (`blaze/env/blaze_core.h: - 456-516, 4652-4659`). Magma streams chunks around the player. Every - row is VERIFIED only inside the snapshot region. L; blocks long-horizon - transfer. +1. Episode region is fixed; no chunk streaming (`cu_region_idx`, + `cu_recenter` in `blaze/env/blaze_core.h`). Magma streams chunks around + the player. Every row is VERIFIED only inside the snapshot region. L; + blocks long-horizon transfer. See "Spawn -> dragon" above. 2. Policy actions are privileged helpers (craft/interact/smelt in `blaze/rl/obs_pack.h:20-51`); no strafe, sneak, sprint, or slot-click as magma `player_ctl` sees them. `do_place` is reachable: RL `a[8]` is @@ -228,8 +269,14 @@ Observation exposes no mob, light, or health planes. - Native ppo spawn->torch t0 is 0.215 (chain4 curriculum, 510M ticks) vs the 0.4 target; stage4->torch is 8/8 seeds. GATES row 1. -- No native transfer/eval of `ppo_ckpt.bin` into magma (torch eval scripts - removed). GATES row 2. +- Native magma eval is wired (`eval --backend magma --transfer closed|replay`) + and the 32-tick BOLR harness is `make -C blaze/rl test-eval-magma`. + 13-seed tries=5 n=65 (gamer 2026-08-26): `retrain_0821_best.bin` magma + closed == cpu == cuda (torches 0/13, t0:6 logs3:7). Replay MATCH 56/65; + 9 DIVERGE all cam 1-5 px, not blessed. `ppo_ckpt_best.bin` magma + closed == cpu == cuda t0:13; replay MATCH 45/65. Camera stays compile-time 64x36 + (`oc_pixel`); `--rl-bin` is not a resolution flag. GATES row 2. Gate 2 + accept stays open (0 torches). - Python still owns replay/pixels/M2 verify; no binary tape; no root `make verify`. GATES row 13. - Blaze native Metal tick (M3) is sequenced after CUDA survival rows pass diff --git a/blaze/rl/Makefile b/blaze/rl/Makefile index 456c8c15..9089174d 100644 --- a/blaze/rl/Makefile +++ b/blaze/rl/Makefile @@ -110,6 +110,9 @@ TEST_SKY_BIN := $(OUT)/test_skylight_water TEST_BLK_BIN := $(OUT)/test_blocklight SMOKE_BIN := $(OUT)/ppo EVAL_BIN := $(OUT)/eval +EVAL_MAGMA_OBJ := $(OUT)/eval_magma.o +TEST_EVAL_MAGMA_BIN := $(OUT)/test_eval_magma +CKPT ?= out/blaze/rl/ppo_ckpt_best.bin SAN_TEST_BIN := $(SAN_OUT)/test_train_config SAN_CHAIN_BIN := $(SAN_OUT)/test_chain_reward SAN_SMOKE_BIN := $(SAN_OUT)/ppo @@ -122,7 +125,7 @@ CHAIN_OBJS := $(OUT)/chain_reward.o $(OUT)/chain_curr.o .PHONY: all test-config test-chain test-capture test-entity-spine \ test-falling test-weather test-chests test-furnaces test-projectiles test-explosions test-randtick test-biome-plane test-mobs test-mobs-ss test-mobs-end test-mobs-witch test-passives test-hazards test-ground-items test-potions test-placement test-beds test-skylight-water test-blocklight smoke smoke-cpu smoke-cuda \ smoke-metal smoke-reset test sanitize clean env-cpu env-cuda \ - env-metal eval + env-metal eval test-eval-magma eval-magma all: test-config smoke @@ -147,6 +150,9 @@ $(OUT)/chain_reward.o: chain_reward.c chain_reward.h | $(OUT) $(OUT)/chain_curr.o: chain_curr.c chain_curr.h chain_reward.h | $(OUT) $(CC) $(OWN_CFLAGS) $(CPPFLAGS) -c -o $@ chain_curr.c +$(EVAL_MAGMA_OBJ): eval_magma.c eval_magma.h | $(OUT) + $(CC) $(OWN_CFLAGS) $(CPPFLAGS) -c -o $@ eval_magma.c + # ---- Linux CUDA staging ---- ifneq ($(UNAME_S),Darwin) $(OUT)/env_cuda_stage.o: env_cuda_stage.c env_cuda_stage.h | $(OUT) @@ -400,10 +406,11 @@ $(SMOKE_BIN): ppo.c obs_pack.h rl_ckpt.h $(TRAIN_CFG_OBJ) $(CHAIN_OBJS) \ $(CHAIN_OBJS) $(METAL_OBS_OBJ) $(METAL_HOST_OBJ) $(NN_OBJS) \ $(LINK_LIBS) $(LDLIBS) -$(EVAL_BIN): eval.c obs_pack.h rl_ckpt.h $(CHAIN_OBJS) $(METAL_OBS_OBJ) \ - $(METAL_HOST_OBJ) $(NN_OBJS) $(ENV_CPU_SO) $(METAL_LIB) | $(OUT) - $(METAL_CC) $(OWN_CFLAGS) $(CPPFLAGS) -o $@ eval.c $(CHAIN_OBJS) \ - $(METAL_OBS_OBJ) $(METAL_HOST_OBJ) $(NN_OBJS) \ +$(EVAL_BIN): eval.c obs_pack.h rl_ckpt.h $(CHAIN_OBJS) $(EVAL_MAGMA_OBJ) \ + $(METAL_OBS_OBJ) $(METAL_HOST_OBJ) $(NN_OBJS) $(ENV_CPU_SO) \ + $(METAL_LIB) | $(OUT) + $(METAL_CC) $(OWN_CFLAGS) $(CPPFLAGS) -o $@ eval.c $(EVAL_MAGMA_OBJ) \ + $(CHAIN_OBJS) $(METAL_OBS_OBJ) $(METAL_HOST_OBJ) $(NN_OBJS) \ $(LINK_LIBS) $(LDLIBS) else $(SMOKE_BIN): ppo.c obs_pack.h rl_ckpt.h $(TRAIN_CFG_OBJ) $(CHAIN_OBJS) \ @@ -411,10 +418,10 @@ $(SMOKE_BIN): ppo.c obs_pack.h rl_ckpt.h $(TRAIN_CFG_OBJ) $(CHAIN_OBJS) \ $(CC) $(OWN_CFLAGS) $(CPPFLAGS) -o $@ ppo.c $(TRAIN_CFG_OBJ) $(CHAIN_OBJS) \ $(STAGE_OBJ) $(NN_OBJS) $(LINK_LIBS) -$(EVAL_BIN): eval.c obs_pack.h rl_ckpt.h $(CHAIN_OBJS) $(STAGE_OBJ) $(NN_OBJS) \ - $(ENV_CPU_SO) $(ENV_CUDA_SO) | $(OUT) - $(CC) $(OWN_CFLAGS) $(CPPFLAGS) -o $@ eval.c $(CHAIN_OBJS) $(STAGE_OBJ) \ - $(NN_OBJS) $(LINK_LIBS) +$(EVAL_BIN): eval.c obs_pack.h rl_ckpt.h $(CHAIN_OBJS) $(EVAL_MAGMA_OBJ) \ + $(STAGE_OBJ) $(NN_OBJS) $(ENV_CPU_SO) $(ENV_CUDA_SO) | $(OUT) + $(CC) $(OWN_CFLAGS) $(CPPFLAGS) -o $@ eval.c $(EVAL_MAGMA_OBJ) \ + $(CHAIN_OBJS) $(STAGE_OBJ) $(NN_OBJS) $(LINK_LIBS) endif test-config: $(TEST_BIN) @@ -556,6 +563,19 @@ smoke-reset: $(SMOKE_BIN) # and t0 snaps). CUDA/Metal slot via --backend; default is CPU. eval: $(EVAL_BIN) +# 32-tick Magma --rl-bin vs blaze_cpu BOLR. Not on `test`. Needs magma_game. +$(TEST_EVAL_MAGMA_BIN): test_eval_magma.c $(EVAL_MAGMA_OBJ) $(ENV_CPU_SO) | $(OUT) + $(CC) $(OWN_CFLAGS) $(CPPFLAGS) -o $@ test_eval_magma.c \ + $(EVAL_MAGMA_OBJ) $(LDLIBS) + +test-eval-magma: $(TEST_EVAL_MAGMA_BIN) $(ENV_CPU_SO) + cd $(REPO) && ./out/blaze/rl/test_eval_magma + +# 13-seed magma closed-loop. Override CKPT=. Not on `test`. +eval-magma: $(EVAL_BIN) $(ENV_CPU_SO) + cd $(REPO) && ./out/blaze/rl/eval --checkpoint $(CKPT) \ + --backend magma --stage 0 + test: test-config test-chain test-capture smoke # Sanitizer build of owned sources; env via dlopen of the same .so files. diff --git a/blaze/rl/eval.c b/blaze/rl/eval.c index 725169fc..b0e1802f 100644 --- a/blaze/rl/eval.c +++ b/blaze/rl/eval.c @@ -9,16 +9,26 @@ * ./out/blaze/rl/eval --checkpoint out/blaze/rl/overnight_gpu0_6m.bin * ./out/blaze/rl/eval --checkpoint PATH --stage 0 # default; t0 snaps * ./out/blaze/rl/eval --checkpoint PATH --stage all # t0 + stg1..4 ladder + * ./out/blaze/rl/eval --checkpoint PATH --backend magma --transfer closed + * ./out/blaze/rl/eval --checkpoint PATH --backend magma --transfer replay */ #define _DEFAULT_SOURCE #include "blaze_abi.h" #include "chain_curr.h" #include "chain_reward.h" +#include "eval_magma.h" #include "model.h" #include "nn.h" #include "obs_pack.h" #include "rl_ckpt.h" +_Static_assert(NN_CAM_W == OC_W && NN_CAM_H == OC_H, "nn camera != oc_pixel"); +_Static_assert((int)ENV_CAM_W == (int)NN_CAM_W && + (int)ENV_CAM_H == (int)NN_CAM_H, + "pack camera != nn"); +_Static_assert((int)ENV_ACT == (int)EM_ACT, "action width"); +_Static_assert((int)ENV_NPIX == (int)EM_NPIX, "pix count"); + #if defined(BLAZE_RL_HAVE_CUDA) && BLAZE_RL_HAVE_CUDA #include "env_cuda_stage.h" #endif @@ -49,7 +59,9 @@ enum { EVAL_MAX_TRIES = 16, EVAL_TORCH_ITEM = 50, EVAL_STAGE_ALL = -1, - EVAL_STAGE_MAX = 4 + EVAL_STAGE_MAX = 4, + EVAL_XFER_CLOSED = 0, + EVAL_XFER_REPLAY = 1 }; static const int kCanonSeeds[] = {2, 3, 10, 11, 14, 16, 20, @@ -89,6 +101,7 @@ typedef struct BlazeFns { BlazeStepFullFn step_full; BlazeObsCamInputsFn obs_cam_inputs; int (*set_success_item)(void *h, int item); + int (*emit)(void *h, int env, int want_cam, void *out); } BlazeFns; typedef struct EvalCfg { @@ -112,6 +125,8 @@ typedef struct EvalCfg { int op_trace; int no_ore_xy; int stage; /* 0..4, or EVAL_STAGE_ALL */ + int transfer; /* EVAL_XFER_CLOSED | EVAL_XFER_REPLAY */ + char magma_bin[EVAL_STR_MAX]; } EvalCfg; typedef struct StageSeedResult { @@ -174,7 +189,8 @@ static void *must_dlsym(void *lib, const char *name) { return p; } -static int blaze_fns_load(BlazeFns *f, const char *path, int want_cam_inputs) { +static int blaze_fns_load(BlazeFns *f, const char *path, int want_cam_inputs, + int want_emit) { void *lib; if (!f || !path) return -1; @@ -201,9 +217,11 @@ static int blaze_fns_load(BlazeFns *f, const char *path, int want_cam_inputs) { f->obs_cam_inputs = (BlazeObsCamInputsFn)must_dlsym(lib, "blaze_obs_cam_inputs"); } + if (want_emit) + f->emit = (int (*)(void *, int, int, void *))must_dlsym(lib, "blaze_emit"); if (!f->create || !f->destroy || !f->load_snapshots || !f->assign || !f->reset || !f->step_full || !f->set_success_item || - (want_cam_inputs && !f->obs_cam_inputs)) { + (want_cam_inputs && !f->obs_cam_inputs) || (want_emit && !f->emit)) { blaze_fns_close(f); return -1; } @@ -459,13 +477,16 @@ static void cfg_defaults(EvalCfg *c) { (void)str_copy_fit(c->metallib, sizeof(c->metallib), "auto"); c->warp_tick = 1; c->stage = 0; + c->transfer = EVAL_XFER_CLOSED; + (void)str_copy_fit(c->magma_bin, sizeof(c->magma_bin), "magma/magma_game"); } static int cfg_set(EvalCfg *c, const char *key, const char *val) { if (!c || !key || !val) return -1; if (!strcmp(key, "backend")) { - if (strcmp(val, "cpu") && strcmp(val, "cuda") && strcmp(val, "metal")) + if (strcmp(val, "cpu") && strcmp(val, "cuda") && strcmp(val, "metal") && + strcmp(val, "magma")) return -2; if (!str_copy_fit(c->backend, sizeof(c->backend), val)) return -2; @@ -537,6 +558,22 @@ static int cfg_set(EvalCfg *c, const char *key, const char *val) { return -2; return 0; } + if (!strcmp(key, "transfer")) { + if (!strcmp(val, "closed")) { + c->transfer = EVAL_XFER_CLOSED; + return 0; + } + if (!strcmp(val, "replay")) { + c->transfer = EVAL_XFER_REPLAY; + return 0; + } + return -2; + } + if (!strcmp(key, "magma_bin")) { + if (!str_copy_fit(c->magma_bin, sizeof(c->magma_bin), val)) + return -2; + return 0; + } return -1; } @@ -564,6 +601,9 @@ static void cfg_dump(const EvalCfg *c, FILE *out) { fprintf(out, " %-16s = all\n", "stage"); else fprintf(out, " %-16s = %d\n", "stage", c->stage); + fprintf(out, " %-16s = %s\n", "transfer", + c->transfer == EVAL_XFER_REPLAY ? "replay" : "closed"); + fprintf(out, " %-16s = %s\n", "magma_bin", c->magma_bin); } static void usage(void) { @@ -574,7 +614,9 @@ static void usage(void) { " --tries N best-of-N (default 5)\n" " --ep-ticks N ticks per episode (default 6000)\n" " --repeat N action repeat (default 4)\n" - " --backend cpu|cuda|metal\n" + " --backend cpu|cuda|metal|magma\n" + " --transfer closed|replay magma only (default closed)\n" + " --magma-bin PATH magma_game (default magma/magma_game)\n" " --device N\n" " --seed N Gumbel base seed (default 0)\n" " --stage 0|1|2|3|4|all snap ladder (default 0 = t0)\n" @@ -583,7 +625,9 @@ static void usage(void) { "Run from repo root. Loads schema-1 weights via nn_load.\n" "stage 0 loads s{seed}_t0.bsnp (missing file is fatal).\n" "stage K loads s{seed}_stg{K}.bsnp; missing prints SKIP, exit 0.\n" - "stage all runs 0..4 then a ladder table of new milestones.\n"); + "stage all runs 0..4 then a ladder table of new milestones.\n" + "magma closed: net reads Magma --rl-bin BOLR (64x36 oc_pixel).\n" + "magma replay: net reads blaze_cpu; actions replay on Magma.\n"); } static int parse_argv(EvalCfg *c, int argc, char **argv, int *dump) { @@ -632,6 +676,8 @@ static int parse_argv(EvalCfg *c, int argc, char **argv, int *dump) { key = "success_item"; if (!strcmp(key, "metal-max-cells")) key = "metal_max_cells"; + if (!strcmp(key, "magma-bin")) + key = "magma_bin"; } else { fprintf(stderr, "eval: unexpected argument '%s'\n", a); return -1; @@ -678,6 +724,10 @@ static void print_stage_report(const EvalCfg *cfg, int stage_k, uint64_t rng_see printf("backend %s rng_protocol nn_sample Gumbel rng_seed=%llu " "sample_step=decision ni=seed_index*%d+attempt\n", cfg->backend, (unsigned long long)rng_seed, cfg->tries); + if (strcmp(cfg->backend, "magma") == 0) + printf("transfer %s magma_bin %s camera %dx%d (oc_pixel, not window)\n", + cfg->transfer == EVAL_XFER_REPLAY ? "replay" : "closed", + cfg->magma_bin, NN_CAM_W, NN_CAM_H); n_ok = 0; n_ran = 0; n_skip = 0; @@ -756,6 +806,8 @@ static void print_stage_report(const EvalCfg *cfg, int stage_k, uint64_t rng_see static int eval_run_stage(const EvalCfg *cfg, int stage_k, StageSeedResult *results); +static int eval_run_stage_magma(const EvalCfg *cfg, int stage_k, + StageSeedResult *results); int main(int argc, char **argv) { EvalCfg cfg; @@ -790,6 +842,8 @@ int main(int argc, char **argv) { die("nseeds*tries too large"); if (access(cfg.checkpoint, R_OK) != 0) dief("checkpoint not readable: %s", cfg.checkpoint); + if (strcmp(cfg.backend, "magma") == 0 && access(cfg.magma_bin, X_OK) != 0) + dief("magma_game not executable: %s", cfg.magma_bin); if (cfg.stage == EVAL_STAGE_ALL) { StageSeedResult ladder[5][EVAL_MAX_SEEDS]; @@ -809,6 +863,459 @@ int main(int argc, char **argv) { return eval_run_stage(&cfg, cfg.stage, NULL); } +static void magma_close_n(EvalMagma **mag, int n) { + int i; + if (!mag) + return; + for (i = 0; i < n; ++i) { + eval_magma_close(mag[i]); + mag[i] = NULL; + } +} + +static void magma_lane_fill(const EvalMagmaObs *o, int i, unsigned short *cam, + unsigned char *depth, unsigned char *edge, + float *scal6, float *pose, int *status, + unsigned char *done_buf) { + eval_magma_fill_policy(o, cam + (size_t)i * ENV_NPIX, + depth + (size_t)i * ENV_NPIX, + edge + (size_t)i * ENV_NPIX, + pose + (size_t)i * ENV_POSE, + status + (size_t)i * ENV_STATUS, + scal6 + (size_t)i * ENV_SCAL); + if (!done_buf) + return; + if (o->dead) + done_buf[i] = 2; + else if (o->inv_counts[CR_IX_TORCH] >= 1) + done_buf[i] = 1; + else + done_buf[i] = 0; +} + +static int eval_run_stage_magma(const EvalCfg *cfg, int stage_k, + StageSeedResult *results) { + int n, nsnaps, ep_lim, i, dec, all_done, replay; + void *env = NULL; + Nn *nn = NULL; + NnCreate nd; + NnConfig nc; + BlazeCreateOpts opts; + BlazeFns fns; + char err[512]; + char path_store[EVAL_MAX_SEEDS][EVAL_STR_MAX]; + const char *paths[EVAL_MAX_SEEDS]; + int loaded_si[EVAL_MAX_SEEDS]; + uint8_t skip[EVAL_MAX_SEEDS]; + int seed_best[EVAL_MAX_SEEDS]; + int seed_base[EVAL_MAX_SEEDS]; + int *assign = NULL; + unsigned short *cam = NULL; + unsigned char *depth = NULL; + unsigned char *edge = NULL; + float *scal6 = NULL; + float *rew = NULL; + unsigned char *done_buf = NULL; + float *pose = NULL; + int *status = NULL; + double *act_rows = NULL; + uint8_t *planes = NULL; + float *scalars = NULL; + int32_t *acts = NULL; + float *logp = NULL; + float *values = NULL; + float *logits = NULL; + uint8_t *prior_frame = NULL; + uint8_t *have_prior = NULL; + uint8_t *frame_scratch = NULL; + int *ep_dec = NULL; + uint8_t *finished = NULL; + int *best9 = NULL; + int *reached = NULL; + int *base_stg = NULL; + EvalMagma **mag = NULL; + EvalMagmaObs blaze_obs; + int *div_step = NULL; + char (*div_why)[128] = NULL; + int rc_out = 1; + uint64_t rng_seed; + + memset(&fns, 0, sizeof(fns)); + memset(skip, 0, sizeof(skip)); + memset(seed_best, 0, sizeof(seed_best)); + memset(seed_base, 0, sizeof(seed_base)); + memset(loaded_si, 0, sizeof(loaded_si)); + blaze_create_opts_default(&opts); + replay = cfg->transfer == EVAL_XFER_REPLAY; + ep_lim = cfg->ep_ticks / cfg->action_repeat; + rng_seed = cfg->seed + (uint64_t)stage_k; + + nsnaps = 0; + for (i = 0; i < cfg->nseeds; ++i) { + if (snap_path(path_store[i], EVAL_STR_MAX, cfg->snaps_dir, cfg->seeds[i], + stage_k) != 0) + die("snaps path too long"); + if (access(path_store[i], R_OK) != 0) { + skip[i] = 1; + if (stage_k == 0) + dief("missing t0 snapshot: %s", path_store[i]); + continue; + } + paths[nsnaps] = path_store[i]; + loaded_si[nsnaps] = i; + nsnaps += 1; + } + + if (nsnaps == 0) { + if (results) { + for (i = 0; i < cfg->nseeds; ++i) { + results[i].skip = 1; + results[i].base = 0; + results[i].abs_end = 0; + results[i].torch = 0; + } + } + print_stage_report(cfg, stage_k, rng_seed, skip, seed_best, seed_base); + return 0; + } + + n = nsnaps * cfg->tries; + if (n <= 0) + die("nseeds and tries must be positive"); + if (replay && access(kEnvCpuSo, R_OK) != 0) + dief("missing blaze_cpu.so: %s", kEnvCpuSo); + + mag = (EvalMagma **)calloc((size_t)n, sizeof(*mag)); + div_step = (int *)calloc((size_t)n, sizeof(int)); + div_why = (char (*)[128])calloc((size_t)n, 128); + if (!mag || !div_step || !div_why) + die("alloc failed"); + for (i = 0; i < n; ++i) + div_step[i] = -1; + + for (i = 0; i < n; ++i) { + int loaded = i / cfg->tries; + int si = loaded_si[loaded]; + mag[i] = eval_magma_open(cfg->magma_bin, paths[loaded], cfg->seeds[si], err, + (int)sizeof(err)); + if (!mag[i]) { + fprintf(stderr, "eval: magma open seed %d: %s\n", cfg->seeds[si], err); + goto fail; + } + } + + if (replay) { + opts.ktime = cfg->ktime; + opts.stage_time = cfg->stage_time; + opts.legacy_recenter = cfg->legacy_recenter; + opts.warp_tick = cfg->warp_tick; + opts.op_trace = cfg->op_trace; + opts.no_ore_xy = cfg->no_ore_xy; + if (blaze_fns_load(&fns, kEnvCpuSo, 0, 1) != 0) { + fprintf(stderr, "eval: failed to load %s for magma replay\n", kEnvCpuSo); + goto fail; + } + env = fns.create(0, n, &opts); + if (!env) + die("blaze_create failed"); + if (fns.set_success_item(env, cfg->success_item) != 0) + die("blaze_set_success_item failed"); + if (fns.load_snapshots(env, paths, nsnaps, err, (int)sizeof(err)) < 0) { + fprintf(stderr, "eval: blaze_load_snapshots: %s\n", err); + goto fail; + } + } + + assign = (int *)calloc((size_t)n, sizeof(int)); + cam = (unsigned short *)calloc((size_t)n * ENV_NPIX, sizeof(*cam)); + depth = (unsigned char *)calloc((size_t)n * ENV_NPIX, 1); + edge = (unsigned char *)calloc((size_t)n * ENV_NPIX, 1); + scal6 = (float *)calloc((size_t)n * ENV_SCAL, sizeof(float)); + rew = (float *)calloc((size_t)n, sizeof(float)); + done_buf = (unsigned char *)calloc((size_t)n, 1); + pose = (float *)calloc((size_t)n * ENV_POSE, sizeof(float)); + status = (int *)calloc((size_t)n * ENV_STATUS, sizeof(int)); + act_rows = (double *)calloc((size_t)n * ENV_ACT, sizeof(double)); + planes = (uint8_t *)calloc((size_t)n * ENV_N_CH * ENV_NPIX, 1); + scalars = (float *)calloc((size_t)n * POL_SCAL, sizeof(float)); + acts = (int32_t *)calloc((size_t)n * POL_HEADS, sizeof(int32_t)); + logp = (float *)calloc((size_t)n, sizeof(float)); + values = (float *)calloc((size_t)n, sizeof(float)); + logits = (float *)calloc((size_t)n * NN_N_LOGITS, sizeof(float)); + prior_frame = (uint8_t *)calloc((size_t)n * ENV_N_PLANES * ENV_NPIX, 1); + have_prior = (uint8_t *)calloc((size_t)n, 1); + frame_scratch = (uint8_t *)calloc((size_t)n * ENV_N_PLANES * ENV_NPIX, 1); + ep_dec = (int *)calloc((size_t)n, sizeof(int)); + finished = (uint8_t *)calloc((size_t)n, 1); + best9 = (int *)calloc((size_t)n * 9, sizeof(int)); + reached = (int *)calloc((size_t)n, sizeof(int)); + base_stg = (int *)calloc((size_t)n, sizeof(int)); + if (!assign || !cam || !depth || !edge || !scal6 || !rew || !done_buf || + !pose || !status || !act_rows || !planes || !scalars || !acts || !logp || + !values || !logits || !prior_frame || !have_prior || !frame_scratch || + !ep_dec || !finished || !best9 || !reached || !base_stg) + die("alloc failed"); + + for (i = 0; i < n; ++i) + assign[i] = i / cfg->tries; + if (replay) { + if (fns.assign(env, assign) != 0 || fns.reset(env, NULL) != 0) + die("assign/reset failed"); + for (i = 0; i < n; ++i) { + char why[80]; + if (!mag[i] || div_step[i] >= 0) + continue; + if (fns.emit(env, i, 1, &blaze_obs) != 0) { + div_step[i] = 0; + snprintf(div_why[i], sizeof div_why[i], "blaze_emit t=0"); + continue; + } + if (eval_magma_cmp_gated(eval_magma_obs(mag[i]), &blaze_obs, why, + (int)sizeof(why)) != 0) { + div_step[i] = 0; + snprintf(div_why[i], sizeof div_why[i], "t=0 %s", why); + } + } + } + + nc = nn_config_default(); + nc.rng_seed = rng_seed; + nd.backend = NN_BACKEND_CPU; + nd.device = 0; + nd.max_n = n; + nd.config = nc; + nn = nn_create(&nd); + if (!nn) + dief("nn_create: %s", nn_last_error()); + if (rl_ckpt_load(nn, cfg->checkpoint) != 0) + dief("nn_load: %s", nn_last_error()); + + for (i = 0; i < n; ++i) { + int32_t *a = acts + (size_t)i * POL_HEADS; + memset(a, 0, POL_HEADS * sizeof(int32_t)); + a[0] = 1; + a[1] = 1; + a[2] = 1; + } + acts_to_rows(acts, n, act_rows); + if (replay) { + if (fns.step_full(env, act_rows, cfg->action_repeat, cam, depth, edge, + scal6, rew, done_buf, pose, status) != 0) + die("burn-in blaze step failed"); + } + for (i = 0; i < n; ++i) { + char why[80]; + if (!mag[i]) + continue; + if (eval_magma_step(mag[i], act_rows + (size_t)i * ENV_ACT, + cfg->action_repeat) != 0) { + if (div_step[i] < 0) { + div_step[i] = 1; + snprintf(div_why[i], sizeof div_why[i], "magma burn-in step"); + } + eval_magma_close(mag[i]); + mag[i] = NULL; + continue; + } + if (replay && div_step[i] < 0) { + if (fns.emit(env, i, 1, &blaze_obs) != 0) { + div_step[i] = 1; + snprintf(div_why[i], sizeof div_why[i], "blaze_emit burn-in"); + } else if (eval_magma_cmp_gated(eval_magma_obs(mag[i]), &blaze_obs, why, + (int)sizeof(why)) != 0) { + div_step[i] = 1; + snprintf(div_why[i], sizeof div_why[i], "burn-in %s", why); + } + } + if (!replay) + magma_lane_fill(eval_magma_obs(mag[i]), i, cam, depth, edge, scal6, pose, + status, done_buf); + } + + for (i = 0; i < n; ++i) { + int k; + for (k = 0; k < 9; ++k) + best9[(size_t)i * 9 + (size_t)k] = status[(size_t)i * ENV_STATUS + k]; + base_stg[i] = inv_stage(best9 + (size_t)i * 9); + have_prior[i] = 0; + ep_dec[i] = 0; + finished[i] = 0; + reached[i] = 0; + } + + for (dec = 0; dec < ep_lim; ++dec) { + pack_obs(cam, depth, edge, scal6, pose, status, ep_dec, ep_lim, have_prior, + prior_frame, n, planes, scalars, frame_scratch); + if (nn_forward(nn, planes, scalars, n, logits, values) != 0) + dief("nn_forward: %s", nn_last_error()); + if (nn_sample(nn, logits, n, NN_SAMPLE_GUMBEL, acts, logp, NULL) != 0) + dief("nn_sample: %s", nn_last_error()); + acts_to_rows(acts, n, act_rows); + if (replay) { + if (fns.step_full(env, act_rows, cfg->action_repeat, cam, depth, edge, + scal6, rew, done_buf, pose, status) != 0) + die("blaze_step_full failed"); + } + for (i = 0; i < n; ++i) { + char why[80]; + if (!mag[i]) + continue; + if (eval_magma_step(mag[i], act_rows + (size_t)i * ENV_ACT, + cfg->action_repeat) != 0) { + if (div_step[i] < 0) { + div_step[i] = dec + 2; + snprintf(div_why[i], sizeof div_why[i], "magma step"); + } + eval_magma_close(mag[i]); + mag[i] = NULL; + done_buf[i] = 2; + continue; + } + if (replay && div_step[i] < 0) { + if (fns.emit(env, i, 1, &blaze_obs) != 0) { + div_step[i] = dec + 2; + snprintf(div_why[i], sizeof div_why[i], "blaze_emit"); + } else if (eval_magma_cmp_gated(eval_magma_obs(mag[i]), &blaze_obs, why, + (int)sizeof(why)) != 0) { + div_step[i] = dec + 2; + snprintf(div_why[i], sizeof div_why[i], "%s", why); + } + } + if (!replay) + magma_lane_fill(eval_magma_obs(mag[i]), i, cam, depth, edge, scal6, + pose, status, done_buf); + } + + all_done = 1; + for (i = 0; i < n; ++i) { + int k; + uint8_t *frame; + uint8_t *prior; + int stg; + if (finished[i]) + continue; + for (k = 0; k < 9; ++k) { + int v = status[(size_t)i * ENV_STATUS + k]; + if (v > best9[(size_t)i * 9 + (size_t)k]) + best9[(size_t)i * 9 + (size_t)k] = v; + } + frame = frame_scratch + (size_t)i * ENV_N_PLANES * ENV_NPIX; + prior = prior_frame + (size_t)i * ENV_N_PLANES * ENV_NPIX; + memcpy(prior, frame, (size_t)ENV_N_PLANES * ENV_NPIX); + have_prior[i] = 1; + ep_dec[i] += 1; + if (best9[(size_t)i * 9 + CR_IX_TORCH] >= 1 || done_buf[i] == 1) { + reached[i] = CR_N_STAGES; + finished[i] = 1; + continue; + } + stg = cr_stage_of_best(best9 + (size_t)i * 9); + if (done_buf[i] > 0 || ep_dec[i] >= ep_lim) { + reached[i] = stg; + finished[i] = 1; + continue; + } + reached[i] = stg; + all_done = 0; + } + if (all_done) + break; + if ((dec + 1) % 50 == 0 || dec + 1 == ep_lim) { + int nfin = 0; + for (i = 0; i < n; ++i) + nfin += (int)finished[i]; + fprintf(stderr, "eval: decision %d/%d finished=%d/%d\n", dec + 1, ep_lim, + nfin, n); + } + } + for (i = 0; i < n; ++i) { + if (!finished[i]) + reached[i] = cr_stage_of_best(best9 + (size_t)i * 9); + } + + for (i = 0; i < cfg->nseeds; ++i) { + seed_best[i] = 0; + seed_base[i] = 0; + } + for (i = 0; i < n; ++i) { + int loaded = i / cfg->tries; + int si = loaded_si[loaded]; + if (reached[i] > seed_best[si]) + seed_best[si] = reached[i]; + if ((i % cfg->tries) == 0) + seed_base[si] = base_stg[i]; + } + + if (replay) { + int n_match = 0, n_ran = 0; + printf("replay magma --rl-bin vs blaze_cpu gated BOLR (cam/inv/pose/" + "hotbar/coal/dead)\n"); + for (i = 0; i < n; ++i) { + int loaded = i / cfg->tries; + int si = loaded_si[loaded]; + int attempt = i % cfg->tries; + n_ran += 1; + if (div_step[i] < 0) { + n_match += 1; + printf("seed %3d try %d: MATCH\n", cfg->seeds[si], attempt); + } else { + printf("seed %3d try %d: DIVERGE step=%d %s\n", cfg->seeds[si], + attempt, div_step[i], div_why[i]); + } + } + printf("MATCH %d/%d (step 0=t0, 1=burn-in, 2+=policy decision)\n", n_match, + n_ran); + fflush(stdout); + } + + if (results) { + for (i = 0; i < cfg->nseeds; ++i) { + results[i].skip = skip[i] ? 1 : 0; + results[i].base = seed_base[i]; + results[i].abs_end = seed_best[i]; + results[i].torch = seed_best[i] >= CR_N_STAGES; + } + } + print_stage_report(cfg, stage_k, rng_seed, skip, seed_best, seed_base); + rc_out = 0; + +fail: + magma_close_n(mag, n); + free(mag); + free(div_step); + free(div_why); + free(assign); + free(cam); + free(depth); + free(edge); + free(scal6); + free(rew); + free(done_buf); + free(pose); + free(status); + free(act_rows); + free(planes); + free(scalars); + free(acts); + free(logp); + free(values); + free(logits); + free(prior_frame); + free(have_prior); + free(frame_scratch); + free(ep_dec); + free(finished); + free(best9); + free(reached); + free(base_stg); + if (nn) + nn_destroy(nn); + if (env && fns.destroy) + fns.destroy(env); + blaze_fns_close(&fns); + return rc_out; +} + static int eval_run_stage(const EvalCfg *cfg, int stage_k, StageSeedResult *results) { int n, nsnaps, ep_lim, i, dec, all_done; @@ -881,6 +1388,9 @@ static int eval_run_stage(const EvalCfg *cfg, int stage_k, * among snapshots that exist. */ rng_seed = cfg->seed + (uint64_t)stage_k; + if (strcmp(cfg->backend, "magma") == 0) + return eval_run_stage_magma(cfg, stage_k, results); + if (strcmp(cfg->backend, "cpu") == 0) { is_cuda = 0; is_metal = 0; @@ -947,7 +1457,7 @@ static int eval_run_stage(const EvalCfg *cfg, int stage_k, if (n <= 0) die("nseeds and tries must be positive"); - if (blaze_fns_load(&fns, so_path, want_cam) != 0) { + if (blaze_fns_load(&fns, so_path, want_cam, 0) != 0) { fprintf(stderr, "eval: failed to load env library for backend=%s (path=%s); " "no fallback\n", diff --git a/blaze/rl/eval_magma.c b/blaze/rl/eval_magma.c new file mode 100644 index 00000000..46c5322a --- /dev/null +++ b/blaze/rl/eval_magma.c @@ -0,0 +1,382 @@ +#define _DEFAULT_SOURCE +#include "eval_magma.h" + +#include +#include +#include +#include +#include +#include +#include +#include + +#define EM_PI 3.14159265358979323846 +#define EM_EYE 1.62 + +struct EvalMagma { + pid_t pid; + FILE *in; /* parent writes JSON */ + FILE *out; /* parent reads BOLR */ + EvalMagmaObs obs; +}; + +static void err_set(char *err, int cap, const char *msg) { + if (!err || cap <= 0) + return; + snprintf(err, (size_t)cap, "%s", msg); +} + +static int read_exact(FILE *f, void *dst, size_t n) { + unsigned char *p = (unsigned char *)dst; + size_t got = 0; + while (got < n) { + size_t r = fread(p + got, 1, n - got, f); + if (r == 0) + return -1; + got += r; + } + return 0; +} + +static int read_bolr(FILE *f, EvalMagmaObs *o) { + unsigned char win[4]; + unsigned magic; + if (read_exact(f, win, 4) != 0) + return -1; + for (;;) { + memcpy(&magic, win, 4); + if (magic == EM_MAGIC) + break; + { + unsigned char b; + if (read_exact(f, &b, 1) != 0) + return -1; + win[0] = win[1]; + win[1] = win[2]; + win[2] = win[3]; + win[3] = b; + } + } + o->magic = magic; + if (read_exact(f, (char *)o + 4, sizeof(*o) - 4) != 0) + return -1; + if (o->magic != EM_MAGIC) + return -1; + return 0; +} + +static int write_act(FILE *f, const double *a) { + if (!f || !a) + return -1; + if (fprintf(f, + "{\"forward\":%.9g,\"strafe\":%.9g,\"dyaw\":%.9g,\"dpitch\":%.9g," + "\"jump\":%d,\"sneak\":%d,\"sprint\":%d,\"attack\":%d,\"use\":%d," + "\"hotbar\":%d,\"craft\":%d,\"interact\":%d,\"smelt\":%d," + "\"cam\":1}\n", + a[0], a[1], a[2], a[3], (int)a[4], (int)a[5], (int)a[6], + (int)a[7], (int)a[8], (int)a[9], (int)a[10], (int)a[11], + (int)a[12]) < 0) + return -1; + if (fflush(f) != 0) + return -1; + return 0; +} + +static double wrap180(double a) { + a = fmod(a + 180.0, 360.0); + if (a < 0.0) + a += 360.0; + return a - 180.0; +} + +static int nearest_coal(const EvalMagmaObs *o, double *ry, double *rp, + double *dist) { + double ex = o->x, ey = o->y + EM_EYE, ez = o->z; + double bd = 0.0; + int have = 0, i; + for (i = 0; i < EM_NCOAL; ++i) { + double dx, dy, dz, d; + if (o->coal[i][0] == 0 && o->coal[i][1] == 0 && o->coal[i][2] == 0) + break; + dx = o->coal[i][0] + 0.5 - ex; + dy = o->coal[i][1] + 0.5 - ey; + dz = o->coal[i][2] + 0.5 - ez; + d = sqrt(dx * dx + dy * dy + dz * dz); + if (!have || d < bd) { + double dd = d > 1e-9 ? d : 1e-9; + if (ry) + *ry = wrap180(atan2(-dx, dz) * (180.0 / EM_PI) - (double)o->yaw); + if (rp) + *rp = -asin(dy / dd) * (180.0 / EM_PI) - (double)o->pitch; + if (dist) + *dist = d; + bd = d; + have = 1; + } + } + return have; +} + +EvalMagma *eval_magma_open(const char *bin, const char *snap, int seed, + char *err, int err_cap) { + EvalMagma *m; + int pin[2] = {-1, -1}, pout[2] = {-1, -1}; + pid_t pid; + char seedbuf[32]; + + if (!bin || !bin[0] || !snap || !snap[0]) { + err_set(err, err_cap, "magma bin/snap empty"); + return NULL; + } + if (access(bin, X_OK) != 0) { + err_set(err, err_cap, "magma_game not executable"); + return NULL; + } + if (access(snap, R_OK) != 0) { + err_set(err, err_cap, "snapshot not readable"); + return NULL; + } + if (pipe(pin) != 0 || pipe(pout) != 0) { + err_set(err, err_cap, "pipe failed"); + return NULL; + } + signal(SIGPIPE, SIG_IGN); + pid = fork(); + if (pid < 0) { + err_set(err, err_cap, "fork failed"); + close(pin[0]); + close(pin[1]); + close(pout[0]); + close(pout[1]); + return NULL; + } + if (pid == 0) { + char *argv[16]; + int n = 0; + snprintf(seedbuf, sizeof(seedbuf), "%d", seed); + if (dup2(pin[0], 0) < 0 || dup2(pout[1], 1) < 0) + _exit(127); + close(pin[0]); + close(pin[1]); + close(pout[0]); + close(pout[1]); + argv[n++] = (char *)bin; + argv[n++] = "--rl-bin"; + argv[n++] = "--render"; + argv[n++] = "off"; + argv[n++] = "--pace"; + argv[n++] = "unlimited"; + argv[n++] = "--mobs"; + argv[n++] = "off"; + argv[n++] = "--snapshot-in"; + argv[n++] = (char *)snap; + argv[n++] = "--seed"; + argv[n++] = seedbuf; + argv[n] = NULL; + { + int dn = open("/dev/null", O_WRONLY); + if (dn >= 0) { + dup2(dn, 2); + close(dn); + } + } + execv(bin, argv); + _exit(127); + } + close(pin[0]); + close(pout[1]); + m = (EvalMagma *)calloc(1, sizeof(*m)); + if (!m) { + err_set(err, err_cap, "oom"); + close(pin[1]); + close(pout[0]); + kill(pid, SIGTERM); + waitpid(pid, NULL, 0); + return NULL; + } + m->pid = pid; + m->in = fdopen(pin[1], "w"); + m->out = fdopen(pout[0], "r"); + if (!m->in || !m->out) { + err_set(err, err_cap, "fdopen failed"); + eval_magma_close(m); + return NULL; + } + setvbuf(m->in, NULL, _IOLBF, 0); + setvbuf(m->out, NULL, _IONBF, 0); + if (read_bolr(m->out, &m->obs) != 0) { + int st = 0; + if (waitpid(pid, &st, WNOHANG) > 0) { + m->pid = 0; + if (err && err_cap > 0) + snprintf(err, (size_t)err_cap, "first BOLR: magma exited %d", + WIFEXITED(st) ? WEXITSTATUS(st) : -1); + } else + err_set(err, err_cap, "first BOLR read failed"); + eval_magma_close(m); + return NULL; + } + return m; +} + +void eval_magma_close(EvalMagma *m) { + int st; + if (!m) + return; + if (m->in) + fclose(m->in); + if (m->out) + fclose(m->out); + if (m->pid > 0) { + kill(m->pid, SIGTERM); + waitpid(m->pid, &st, 0); + } + free(m); +} + +int eval_magma_step(EvalMagma *m, const double *act13, int repeat) { + int i; + double a[EM_ACT]; + if (!m || !act13 || repeat < 1) + return -1; + memcpy(a, act13, sizeof a); + for (i = 0; i < repeat; ++i) { + if (i > 0) { + a[2] = 0.0; + a[3] = 0.0; + a[10] = -1.0; + a[11] = 0.0; + a[12] = 0.0; + } + if (write_act(m->in, a) != 0) + return -1; + if (read_bolr(m->out, &m->obs) != 0) + return -1; + } + return 0; +} + +const EvalMagmaObs *eval_magma_obs(const EvalMagma *m) { + return m ? &m->obs : NULL; +} + +void eval_magma_fill_policy(const EvalMagmaObs *o, unsigned short *cam, + unsigned char *dep, unsigned char *edg, float *pose, + int *status, float *scal6) { + int i, sel; + if (!o) + return; + if (cam) + memcpy(cam, o->cam, sizeof o->cam); + if (dep) + memcpy(dep, o->depth, sizeof o->depth); + if (edg) + memcpy(edg, o->edge, sizeof o->edge); + if (pose) { + pose[0] = (float)o->x; + pose[1] = (float)o->y; + pose[2] = (float)o->z; + pose[3] = o->yaw; + pose[4] = o->pitch; + } + if (status) { + memset(status, 0, 17 * sizeof(int)); + for (i = 0; i < EM_INV; ++i) + status[i] = o->inv_counts[i]; + sel = o->hotbar_sel; + if (sel < 0) + sel = 0; + if (sel > 8) + sel = 8; + status[9] = sel; + status[10] = o->hotbar_ids[sel]; + status[11] = o->container; + } + if (scal6) { + double ry = 0.0, rp = 0.0, dist = 0.0, pr; + int have = nearest_coal(o, &ry, &rp, &dist); + pr = (double)o->pitch * (EM_PI / 180.0); + if (!have) { + scal6[0] = 0.0f; + scal6[1] = 0.0f; + scal6[2] = 0.0f; + scal6[3] = 1.0f; + } else { + scal6[0] = (float)sin(ry * (EM_PI / 180.0)); + scal6[1] = (float)cos(ry * (EM_PI / 180.0)); + scal6[2] = (float)(rp / 90.0); + scal6[3] = (float)((dist < 24.0 ? dist : 24.0) / 24.0); + } + scal6[4] = (float)sin(pr); + scal6[5] = (float)cos(pr); + } +} + +int eval_magma_cmp_gated(const EvalMagmaObs *a, const EvalMagmaObs *b, + char *why, int why_cap) { + int i; + if (!a || !b) { + err_set(why, why_cap, "null obs"); + return -1; + } + if (a->dead != b->dead) { + snprintf(why, (size_t)why_cap, "dead %d vs %d", a->dead, b->dead); + return 1; + } + if (a->x != b->x || a->y != b->y || a->z != b->z) { + snprintf(why, (size_t)why_cap, + "pose xyz magma=(%.9g,%.9g,%.9g) blaze=(%.9g,%.9g,%.9g)", a->x, + a->y, a->z, b->x, b->y, b->z); + return 1; + } + if (a->yaw != b->yaw || a->pitch != b->pitch) { + snprintf(why, (size_t)why_cap, "yaw/pitch magma=%g/%g blaze=%g/%g", a->yaw, + a->pitch, b->yaw, b->pitch); + return 1; + } + if (a->container != b->container) { + snprintf(why, (size_t)why_cap, "container %d vs %d", a->container, + b->container); + return 1; + } + if (a->hotbar_sel != b->hotbar_sel) { + snprintf(why, (size_t)why_cap, "hotbar_sel %d vs %d", a->hotbar_sel, + b->hotbar_sel); + return 1; + } + for (i = 0; i < EM_INV; ++i) { + if (a->hotbar_ids[i] != b->hotbar_ids[i] || + a->hotbar_counts[i] != b->hotbar_counts[i]) { + snprintf(why, (size_t)why_cap, "hotbar[%d] id/count %d/%d vs %d/%d", i, + a->hotbar_ids[i], a->hotbar_counts[i], b->hotbar_ids[i], + b->hotbar_counts[i]); + return 1; + } + if (a->inv_counts[i] != b->inv_counts[i]) { + snprintf(why, (size_t)why_cap, "inv_counts[%d] %d vs %d", i, + a->inv_counts[i], b->inv_counts[i]); + return 1; + } + } + if (memcmp(a->coal, b->coal, sizeof a->coal) != 0) { + err_set(why, why_cap, "coal"); + return 1; + } + if (memcmp(a->cam, b->cam, sizeof a->cam) != 0) { + int n = 0; + for (i = 0; i < EM_NPIX; ++i) + if (a->cam[i] != b->cam[i]) + n++; + snprintf(why, (size_t)why_cap, "cam %d px", n); + return 1; + } + if (memcmp(a->depth, b->depth, sizeof a->depth) != 0) { + err_set(why, why_cap, "depth"); + return 1; + } + if (memcmp(a->edge, b->edge, sizeof a->edge) != 0) { + err_set(why, why_cap, "edge"); + return 1; + } + return 0; +} diff --git a/blaze/rl/eval_magma.h b/blaze/rl/eval_magma.h new file mode 100644 index 00000000..45e0c783 --- /dev/null +++ b/blaze/rl/eval_magma.h @@ -0,0 +1,78 @@ +/* Magma --rl-bin subprocess: one env, BOLR in, JSON actions out. + * Camera is oc_pixel (OC_W x OC_H). Not the window raster. Size is + * compile-time (obs_camera.h); not a Magma width/height or --rl-bin flag. */ +#pragma once + +#include "obs_camera.h" + +#include + +#ifdef __cplusplus +extern "C" { +#endif + +enum { + EM_NBLOCKS = 256, + EM_NLOGS = 64, + EM_NCOAL = 32, + EM_NPIX = OC_NPIX, + EM_ACT = 13, + EM_INV = 9 +}; + +#define EM_MAGIC 0x524c4f42u /* "BOLR" */ + +#pragma pack(push, 1) +typedef struct EvalMagmaObs { + unsigned magic; + long long tick; + double x, y, z; + float yaw, pitch; + int dead; + int hotbar_ids[EM_INV]; + int hotbar_counts[EM_INV]; + int hotbar_sel; + int container; + int inv_counts[EM_INV]; + int blocks[EM_NBLOCKS][4]; + int logs[EM_NLOGS][3]; + int coal[EM_NCOAL][3]; + unsigned short cam[EM_NPIX]; + unsigned char depth[EM_NPIX]; + unsigned char edge[EM_NPIX]; +} EvalMagmaObs; +#pragma pack(pop) + +_Static_assert(OC_W == 64 && OC_H == 36, "policy camera is frozen 64x36"); +_Static_assert(EM_NPIX == OC_NPIX, "EvalMagmaObs cam != oc_pixel"); +_Static_assert(sizeof(EvalMagmaObs) == 14628u, "BOLR layout"); + +typedef struct EvalMagma EvalMagma; + +/* Spawn magma_game --rl-bin --mobs off --snapshot-in SNAP. Reads the first + * BOLR. bin/snap are paths from the caller's cwd (repo root). */ +EvalMagma *eval_magma_open(const char *bin, const char *snap, int seed, + char *err, int err_cap); +void eval_magma_close(EvalMagma *m); + +/* Write act13 (blaze_step layout) as JSON, `repeat` times. Last BOLR kept. + * Tick 0 of the decision carries dyaw/dpitch/craft/interact/smelt. Later + * ticks zero look and the pre-tick primitives (blaze_step_full). */ +int eval_magma_step(EvalMagma *m, const double *act13, int repeat); + +const EvalMagmaObs *eval_magma_obs(const EvalMagma *m); + +/* Policy tensors from one BOLR. scal6 may be NULL. status is CU_STATUS_K-wide + * but only [0..11] are filled (dig/iron stay 0; pack_obs does not read them). */ +void eval_magma_fill_policy(const EvalMagmaObs *o, unsigned short *cam, + unsigned char *dep, unsigned char *edg, float *pose, + int *status, float *scal6); + +/* Gated M1 fields: pose, inv, hotbar, coal, cam, depth, edge, dead, + * container. blocks/logs/tick are excluded. Returns 0 if equal. */ +int eval_magma_cmp_gated(const EvalMagmaObs *a, const EvalMagmaObs *b, + char *why, int why_cap); + +#ifdef __cplusplus +} +#endif diff --git a/blaze/rl/test_eval_magma.c b/blaze/rl/test_eval_magma.c new file mode 100644 index 00000000..2dad5dc0 --- /dev/null +++ b/blaze/rl/test_eval_magma.c @@ -0,0 +1,165 @@ +/* 32-tick scripted Magma --rl-bin vs blaze_cpu.so BOLR gate. + * Run from repo root. No checkpoint. */ +#define _DEFAULT_SOURCE +#include "blaze_abi.h" +#include "eval_magma.h" + +#include +#include +#include +#include +#include + +static const char kSnap[] = "verify/fixtures/port/s10_t0_r64_no_liquid.bsnp"; +static const char kSo[] = "out/blaze/env/blaze_cpu.so"; +static const char kMagma[] = "magma/magma_game"; +enum { kTicks = 32, kSeed = 10 }; + +typedef void *(*CreateFn)(int, int, const BlazeCreateOpts *); +typedef void (*DestroyFn)(void *); +typedef int (*LoadFn)(void *, const char *const *, int, char *, int); +typedef int (*AssignFn)(void *, const int *); +typedef int (*ResetFn)(void *, const unsigned char *); +typedef int (*TickRawFn)(void *, int, const double *, int, void *); +typedef int (*EmitFn)(void *, int, int, void *); +typedef int (*ObsSizeFn)(void); +typedef int (*SetSuccessFn)(void *, int); + +static void *must_dlsym(void *lib, const char *name) { + void *s = dlsym(lib, name); + if (!s) + fprintf(stderr, "dlsym %s: %s\n", name, dlerror()); + return s; +} + +int main(void) { + void *lib = NULL; + void *env = NULL; + EvalMagma *mag = NULL; + CreateFn create; + DestroyFn destroy; + LoadFn load; + AssignFn assign; + ResetFn reset; + TickRawFn tick_raw; + EmitFn emit; + ObsSizeFn obs_size; + SetSuccessFn set_success; + BlazeCreateOpts opts; + const char *paths[1]; + int idx = 0; + char err[256]; + EvalMagmaObs blaze_obs; + double a17[17]; + double a13[EM_ACT]; + int t, rc = 1; + char why[128]; + + if (access(kMagma, X_OK) != 0) { + fprintf(stderr, "FAIL: missing %s (make -C magma game)\n", kMagma); + return 2; + } + if (access(kSnap, R_OK) != 0) { + fprintf(stderr, "FAIL: missing %s\n", kSnap); + return 2; + } + if (access(kSo, R_OK) != 0) { + fprintf(stderr, "FAIL: missing %s\n", kSo); + return 2; + } + + lib = dlopen(kSo, RTLD_NOW | RTLD_LOCAL); + if (!lib) { + fprintf(stderr, "FAIL: dlopen: %s\n", dlerror()); + return 1; + } + create = (CreateFn)must_dlsym(lib, "blaze_create"); + destroy = (DestroyFn)must_dlsym(lib, "blaze_destroy"); + load = (LoadFn)must_dlsym(lib, "blaze_load_snapshots"); + assign = (AssignFn)must_dlsym(lib, "blaze_assign"); + reset = (ResetFn)must_dlsym(lib, "blaze_reset"); + tick_raw = (TickRawFn)must_dlsym(lib, "blaze_tick_raw"); + emit = (EmitFn)must_dlsym(lib, "blaze_emit"); + obs_size = (ObsSizeFn)must_dlsym(lib, "blaze_obs_size"); + set_success = (SetSuccessFn)must_dlsym(lib, "blaze_set_success_item"); + if (!create || !destroy || !load || !assign || !reset || !tick_raw || !emit || + !obs_size || !set_success) + goto done; + if (obs_size() != (int)sizeof(EvalMagmaObs)) { + fprintf(stderr, "FAIL: BOLR size blaze %d magma-eval %d\n", obs_size(), + (int)sizeof(EvalMagmaObs)); + goto done; + } + + blaze_create_opts_default(&opts); + env = create(0, 1, &opts); + if (!env) { + fprintf(stderr, "FAIL: blaze_create\n"); + goto done; + } + if (set_success(env, 50) != 0) { + fprintf(stderr, "FAIL: blaze_set_success_item\n"); + goto done; + } + paths[0] = kSnap; + if (load(env, paths, 1, err, (int)sizeof(err)) < 0) { + fprintf(stderr, "FAIL: blaze_load_snapshots: %s\n", err); + goto done; + } + if (assign(env, &idx) != 0 || reset(env, NULL) != 0) { + fprintf(stderr, "FAIL: assign/reset\n"); + goto done; + } + + mag = eval_magma_open(kMagma, kSnap, kSeed, err, (int)sizeof(err)); + if (!mag) { + fprintf(stderr, "FAIL: eval_magma_open: %s\n", err); + goto done; + } + + memset(a17, 0, sizeof(a17)); + a17[9] = -1.0; + a17[10] = -1.0; + memset(a13, 0, sizeof(a13)); + a13[9] = -1.0; + a13[10] = -1.0; + if (emit(env, 0, 1, &blaze_obs) != 0) { + fprintf(stderr, "FAIL: blaze_emit t=0\n"); + goto done; + } + if (eval_magma_cmp_gated(eval_magma_obs(mag), &blaze_obs, why, (int)sizeof(why)) != + 0) { + fprintf(stderr, "FAIL: t=0 %s\n", why); + goto done; + } + + a13[0] = 1.0; + a17[0] = 1.0; + for (t = 0; t < kTicks; ++t) { + if (tick_raw(env, 0, a17, 1, &blaze_obs) != 0) { + fprintf(stderr, "FAIL: blaze tick %d\n", t); + goto done; + } + if (eval_magma_step(mag, a13, 1) != 0) { + fprintf(stderr, "FAIL: magma tick %d\n", t); + goto done; + } + if (eval_magma_cmp_gated(eval_magma_obs(mag), &blaze_obs, why, + (int)sizeof(why)) != 0) { + fprintf(stderr, "FAIL: t=%d %s\n", t, why); + goto done; + } + } + printf("PASS: magma --rl-bin vs blaze_cpu BOLR gated match, %d forward ticks " + "(s10 t0)\n", + kTicks); + rc = 0; + +done: + eval_magma_close(mag); + if (env && destroy) + destroy(env); + if (lib) + dlclose(lib); + return rc; +} diff --git a/docs/DEVLOG.md b/docs/DEVLOG.md index 8cfa801d..395d89e7 100644 --- a/docs/DEVLOG.md +++ b/docs/DEVLOG.md @@ -1,5 +1,52 @@ # DEVLOG (compressed) +## 2026-08-26 validation host is anvil + +Linux validation (magma CPU, `make test`, tape replay, blaze M1/M2, eval) +runs on anvil. Do not stage that work on gamer. Mac stays control plane +and Metal. AGENTS.md Hosts table, SUBAGENT.md, GATES row 2 host column. + +## 2026-08-26 magma 13-seed transfer ladder (gamer) + +Protocol: `--stage 0`, 5 tries x 6000 ticks, `rng_seed=0`, +`ni=seed_index*5+attempt`, Gumbel n=65. Camera 64x36 `oc_pixel`. +`nn_sample` Gumbel is batch-size dependent; n=13 tries=1 is not this table. + +`retrain_0821_best.bin` closed Magma (gamer 23:09 MDT, `closed_rc=0`) +matches cpu (Mac) and cuda (gamer) seed-by-seed: torches 0/13; +histogram t0:6, logs3:7. logs3 seeds: 3, 10, 14, 20, 32, 44, 46. +t0 seeds: 2, 11, 16, 27, 29, 33. + +`--transfer replay` same ckpt: MATCH 56/65. Nine DIVERGE, all `cam` 1-5 px +(not blessed): s3 t4 step 160 / 2; s10 t4 147 / 3; s16 t3 407 / 1; +s27 t0 137 / 1; s27 t1 38 / 1; s32 t1 1007 / 2; s44 t0 146 / 5; +s44 t3 361 / 4; s46 t0 422 / 2. Replay milestones equal cpu (policy +reads blaze_cpu). + +`ppo_ckpt_best.bin` cpu==cuda==magma-closed: t0:13, torches 0/13. +Magma replay MATCH 45/65 (cam-px DIVERGE, not blessed). Ladder +`ALL_DONE` 2026-08-26T05:54:10-06:00. + +Gate 2 accept stays OPEN. These nets place 0 torches on Magma. + +## 2026-08-25 magma --rl-bin native transfer eval + +`--rl-bin` is already the Blaze policy camera (`oc_pixel` 64x36). Magma +`width`/`height` stay the human framebuffer. No runtime `cam_w`/`cam_h`. + +Added `blaze/rl/eval_magma.c` (spawn `magma_game --rl-bin --mobs off`, +BOLR in, JSON act13 out; look/craft once per `action_repeat`). +`eval --backend magma --transfer closed|replay`. Harness +`make -C blaze/rl test-eval-magma` (32 forward ticks, s10 t0 port snap). +Not on `make test`. + +## 2026-08-25 spawn->dragon ownership in OPEN files + +Wrote the policy-vs-magma gap into `blaze/OPEN_DIVERGENCES.md` (section +Spawn -> dragon). Magma OPEN now lists Java-vs-magma leftovers on the +PRODUCT route and refuses GPU-sim holes as magma bugs. GATES remaining +#4 drops closed mobs / boats_elytra_xp rows. No new measurement. + ## 2026-08-25 BLOCK flood CUDA queue pool (lane/blocklight) Gamer RTX 3090 sm_86. Rebase `188e0ed` onto `origin/master` `d5acd77` diff --git a/docs/GATES.md b/docs/GATES.md index 2b308c68..055a2519 100644 --- a/docs/GATES.md +++ b/docs/GATES.md @@ -144,9 +144,9 @@ BLOCKED on the v1 `s14_t0_r48_no_liquid.bsnp` (no recapture on gamer). | # | Remaining | Gate | Class | Host | |---|-----------|------|-------|------| | 1 | Native `out/blaze/rl/ppo` reproduces spawn->torch (t0 ~0.4, transfer ~11/13). Wood-break t0 0.495 matched. Staged-curriculum chain4 (2026-08-22) reached t0 0.215 at 510M ticks, stage4->torch 8/8 seeds; spawn->torch t0 ~0.4 still open. | 2 | grindable | anvil gpu0 | -| 2 | Native transfer/eval of `ppo_ckpt.bin` into magma. Torch eval scripts are gone. | 2 | needs-design | anvil | +| 2 | Native transfer/eval into magma is wired (`make -C blaze/rl test-eval-magma`; `eval --backend magma --transfer closed` or `--transfer replay`). 13-seed tries=5 n=65 measured on gamer 2026-08-26: `retrain_0821_best.bin` magma closed == cpu == cuda seed-by-seed (torches 0/13, t0:6 logs3:7). Replay MATCH 56/65; 9 DIVERGE all cam 1-5 px, not blessed. `ppo_ckpt_best.bin` cpu==cuda==magma-closed t0:13. Camera stays 64x36 `oc_pixel`. Gate 2 accept still needs a net that places torches. Next measures on anvil. | 2 | grindable | anvil | | 3 | Magma 60 fps at 1080p. Last CUDA measure 35.93 fps (`--set bench=1`). Raster twins are a two-machine gate; do not edit one kernel overnight. | 3 | grindable | anvil gpu1 + Mac | -| 4 | Port-matrix rows after spawn-to-torch: mobs, portals_dimensions, nether_route, dragon_victory, boats_elytra_xp. fluids, random_ticks, random_ticks_bodies, entity_spine, falling_blocks, weather_optional, chests, projectiles, and explosions VERIFIED 2026-08-22/23. | 2 | grindable DAG | anvil cpu then gpu | +| 4 | Port-matrix after spawn-to-torch. VERIFIED through furnaces/hazards/mobs_*/boats/elytra/xp. Still `supported: false`: `portals_dimensions`, `nether_route`, `dragon_victory`. A policy cannot spawn->dragon matching magma until that DAG plus blaze sweep 1 (fixed region) and 2 (actions/obs). Dragon-fight RL stays out of scope. See `blaze/OPEN_DIVERGENCES.md` "Spawn -> dragon". | 2 | grindable DAG | anvil cpu then gpu | | 5 | Blaze Metal tick (M3). Sequence-blocked on row 4. | 2 | needs-design | Mac later | | 6 | Magma live tick on GPU. No gate accepts it. | none | keep-cpu | none | | 7 | Human spawn->End with zero first-divergence. | 1 | human | Moonlight | @@ -164,8 +164,9 @@ Oracle A/B are grindable (see `magma/OPEN_DIVERGENCES.md` 2026-08-21 captures). Harness holes: `NnUpdateStats` has `entropy_mean` but the chunk log omits -it; no KL/clipfrac; no native 13-seed eval; Metal `n == max_n`; schema-1 -ckpt has no Adam/curriculum; C replay has no PNG path. +it; no KL/clipfrac; Metal `n == max_n`; schema-1 +ckpt has no Adam/curriculum; C replay has no PNG path. Native 13-seed +eval is `out/blaze/rl/eval` (cpu/cuda/magma). Pixel and recorder forensics stay in `magma/OPEN_DIVERGENCES.md`. diff --git a/docs/SUBAGENT.md b/docs/SUBAGENT.md index 9909f342..86e3cd98 100644 --- a/docs/SUBAGENT.md +++ b/docs/SUBAGENT.md @@ -30,11 +30,13 @@ merges. You never merge. Follow this routine in order. - Python is `uv run --no-project --with python ...`. No pip. Pin `UV_CACHE_DIR=$HOME/.cache/uv TMPDIR=$HOME/dev/nw/.tmp` on anvil. -Hosts: `gamer` (idle, -j8 cap, never recapture oracle goldens there) and -`anvil` (shared: never touch tmux sessions or processes you did not create; -GPU only through `overnight-compute wait --agent --resource gpuN`, -check `nvidia-smi` first, release when the GPU run ends, analyze on CPU). -Oracle goldens come from anvil llvmpipe only. +Linux validation host is `anvil` (faster CPU). Magma CPU, tape replay, +`make test`, blaze M1/M2, and `eval` run there. Do not stage those on +`gamer`. Anvil is shared: never touch tmux sessions or processes you did +not create; GPU only through `overnight-compute wait --agent +--resource gpuN`, check `nvidia-smi` first, release when the GPU run +ends, analyze on CPU. Oracle goldens come from anvil llvmpipe only. +`gamer` is 3090 / `sm_86` only. ## 1. Java first, then C diff --git a/magma/OPEN_DIVERGENCES.md b/magma/OPEN_DIVERGENCES.md index 30dc6f7e..11116784 100644 --- a/magma/OPEN_DIVERGENCES.md +++ b/magma/OPEN_DIVERGENCES.md @@ -8,6 +8,32 @@ forensics in `CLOSED_DIVERGENCES.md` (a stub with the close date and one-line resolution stays here in place); read there before re-investigating anything that smells like a settled question. +## Spawn -> End (ownership) + +Magma has the PRODUCT route. `magma/game/test_route_e2e.sh` reaches `won` +on seed 0 (travel pose between generated landmarks; combat, drops, portal +use, and dragon death are live). Gate 1 still wants a clean human +spawn->End vs Java. + +This file is Java vs magma on that route. Remaining that hit it: + +- Sweep 1 stronghold placement (L). Silent: iron bars vs Java doors; + portal nearest-selection. Ungated. +- Sweep 8 tick order vs `WorldServer.tick` (M). Couples every M1 row. +- Class A 10 detmob nether A* window (T182511 t=745). Do not widen + `PNP_DY`. +- Class A 11 roster/AI, boat `UNDER_WATER`, aim-pin 1-tick break lag. +- Class A 2 portal overlay pixels. +- Class A 6 dragon death pixels (RESIDUAL; the fight ticks). +- Class A 8 nether arrival fire/lava anim + lightmap. +- Sim: crystal-explosion fire does not replay; dragon ring-buffer / + boss-bar. + +A GPU-sim policy matching magma to the dragon is +`blaze/OPEN_DIVERGENCES.md` section "Spawn -> dragon (policy vs magma)". +Missing Nether/End in blaze is not a magma bug. + +Last documented: 2026-08-25 spawn->End ownership (no new Java vs magma rows). Last verified: lane/potions 2026-08-24 (rows 10/11 CLOSED: potion tick + shield. SNAP v11 after resume v10. Magma skip double fire decrement on the shared hostile path. potions M1+M2 VERIFIED. Root make test PASS on diff --git a/scripts/lane_stage.sh b/scripts/lane_stage.sh index 4b92d197..e9e43411 100755 --- a/scripts/lane_stage.sh +++ b/scripts/lane_stage.sh @@ -4,12 +4,13 @@ # Mac side: worktree ~/dev/nw/LANE on branch lane/LANE from master, pushed, # with the same gitignored inputs agent_worktree.sh links (tapes, snaps, # java run/ and oracle-src, asset headers, goldens). -# Remote side (HOST = gamer | anvil): clone ~/nlanes/LANE from the host's -# ~/dev/netherite, origin -> GitHub, checkout lane/LANE, symlink java inputs, -# then rsync from the Mac: goldens, blaze snaps, the canonical tape, and every -# --tape NAME (jsonl + sidecars + _frames + _world). +# Remote side (HOST = anvil; gamer only for sm_86): clone ~/nlanes/LANE from +# the host's ~/dev/netherite, origin -> GitHub, checkout lane/LANE, symlink +# java inputs, then rsync from the Mac: goldens, blaze snaps, the canonical +# tape, and every --tape NAME (jsonl + sidecars + _frames + _world). +# Linux validation (magma CPU, make test, tapes, blaze M1/M2, eval) is anvil. # -# bash scripts/lane_stage.sh portaledge gamer +# bash scripts/lane_stage.sh portaledge anvil # bash scripts/lane_stage.sh raster anvil --tape scenario_slime_bounce_20260730T095754Z set -euo pipefail