Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 20 additions & 2 deletions coordinator/deploy/RUNBOOK.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,8 @@ the publisher is absent or stale. Setting it too low understates % complete and,
submissions above it as out of range.

`RANGE_SIZE=1000`. The unit binds `127.0.0.1` (behind the proxy); if the web box is a different
machine, set `COORD_BIND` to the private-network IP and firewall `:8899` to the web box only.
machine, firewall `:8899` to the web box only — see §7, and note that a private-network IP is NOT
available on this topology.

## 1b. Publish the node height (required for a correct chain tip)

Expand Down Expand Up @@ -193,7 +194,24 @@ data-durability gaps a public write endpoint exposes:
multi-block receipts 413 at the proxy). Part A of that file goes in the `http { }` context — remember
`sudo mkdir -p /var/cache/nginx/hazync && sudo chown www-data: /var/cache/nginx/hazync`.
- **Secure bind.** The unit binds `127.0.0.1` (behind the proxy). If the coordinator is a separate box,
set `COORD_BIND` to its **private-network IP** (not `0.0.0.0`) and firewall `:8899` to the web box.
firewall `:8899` to the web box.

⛔ **This previously said "set `COORD_BIND` to its private-network IP (not `0.0.0.0`)". That is not
possible on the live topology and the instruction was unfollowable** (hazync#218). The coordinator
has `lo`, one PUBLIC `eth0` (152.53.93.164) and `docker0`; the web box (83.136.255.218) proxies to
it **over the public internet**. There is no private IP to bind, and `COORD_BIND=127.0.0.1` would
take the board dark. `COORD_BIND=0.0.0.0` is correct here — the access control is the firewall,
not the bind address.

Measured on the coordinator 2026-09-08: `ufw` **active**, default deny incoming, `22` and `8333`
open, and `:8899` reachable only from the web box. The `INPUT` chain also carries hand-added rules
ahead of ufw's (`-P INPUT DROP`, three `--dport 8899` ACCEPTs, then a DROP), so **there are two
sources of truth for one port** — read `iptables -S INPUT` as well as `ufw status`, or you will
believe a rule that something else overrides.

⚠ One of those hand-added ACCEPTs is for a host that no longer talks to us: `94.237.17.228` moved
**0 packets in 90 s** while the web box moved 1,690. It is a stale allow for a recycled provider
address and should be removed.
The server now **refuses to bind a public interface** while verification/signatures are permissive
(`VERIFY_MODE=mock`, `COORD_ALLOW_MOCK`, missing sig lib, `COORD_ALLOW_UNSIGNED`) unless you set
`COORD_ALLOW_PUBLIC_INSECURE=1` — so a misconfigured redeploy fails loudly instead of crediting
Expand Down
11 changes: 10 additions & 1 deletion prover/host/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2602,9 +2602,18 @@ fn write_aggregate_env(
/// cannot become a valid assumption, and finding that out here names the file instead of failing
/// somewhere inside the prover.
fn read_chunk_receipts(nchunks: usize) -> Vec<risc0_zkvm::Receipt> {
// Directory the per-chunk receipts live in (hazync#229). `prove-chunk` writes each receipt to
// $HAZYNC_OUT, which is a full path, so the two halves of a run can disagree about where the
// receipts are: scripts/gpu-benchmark.sh has always exported HAZYNC_RECEIPTS here and nothing
// ever read it, so phase 3 resolved `chunk_0.bin` against its own CWD and died on a run whose
// proves had all succeeded.
//
// Defaults to `.`, which is exactly the previous behaviour — this adds a way to say where,
// it does not change where they are looked for when nobody says.
let dir = std::env::var("HAZYNC_RECEIPTS").unwrap_or_else(|_| ".".into());
let mut receipts: Vec<risc0_zkvm::Receipt> = Vec::with_capacity(nchunks);
for i in 0..nchunks {
let f = format!("chunk_{i}.bin");
let f = format!("{dir}/chunk_{i}.bin");
let r: risc0_zkvm::Receipt =
bincode::deserialize(&std::fs::read(&f).unwrap_or_else(|e| panic!("chunk receipt {f}: {e}"))).unwrap();
r.verify(METHOD_ID).unwrap_or_else(|e| panic!("chunk receipt {f} does not verify: {e}"));
Expand Down
Loading