H-7: challenger /24 subnets are self-reported, so the diversity floor is fabricable (was: capability proofs do not prove the capability)

[defenwycke]

This is a redesign, not a patch — it needs a design decision before code.

• Archive +5 is headers-only. Every response field is derivable from the public 80-byte header; tx_count is self-reported; the ArchiveTx path is dead. A pruned node, an SPV client or an on-demand proxy passes every honest challenger, no collusion required.
• GhostPay +4 is unanchored self-attestation. The challenger checks field shapes and nonce binding only; epoch_state_hash is never compared to anything; recipient re-derivation cannot return Fail by construction; the anchoring epoch_proof is None. A node with no L2 passes with a random hex string.
• Stratum +3 is a bare TCP connect. nc -l 3333 passes. The handshake-capable StratumVerifier exists but is unused in the challenge path.

Compounding this: challenger /24s are self-reported and never probed (H-7), and the payout-gating verification_ledger stores passed verbatim with no evidence the challenger probed anything (H-8). Griefing resistance rests on distinct-subnet majorities that are fabricable.

Direction: archive → demand tx-level data with merkle-branch verification (revive the dead ArchiveTx path); ghostpay → anchor epoch_state_hash in a fleet-visible epoch commitment; stratum → complete a real mining.subscribe in the probe; challengers → count toward diversity only after passing a reachability probe at the claimed address; ledger verdicts → carry a target-signed artifact.

Until then these three are honesty-graded, not proof-graded, and the docs should say so. Blocks multi-operator.

[defenwycke]

Correction: the Public Mining +3 challenge is worse than "a bare TCP connect"

Traced on e9ca0c446 while scoping the stratum fix. The issue attributes the weakness to
probe_stratum_port doing a bare TcpStream::connect. That function does do that — but it is not
in the live challenge path. Its only callers are its own unit tests
(client.rs:1429, 1439, 1448).

The live path is verify_stratum (client.rs:808), and it is an HTTP GET to the target asking the
target whether it serves stratum:

let url = self.build_url(node_address,
    &format!("/verify/stratum?protocol={}&unsigned=true", protocol_str))?;
...
Ok(resp) => result.stratum_verified = resp.success && resp.connected,   // client.rs:1042

So the verdict is resp.success && resp.connected — both self-reported by the node being
challenged. The challenger never touches the target's stratum port at all.

That means the bar is lower than the issue states:

	passes today?
nc -l 3333 (issue's example)	yes — but not even needed
no stratum port whatsoever, HTTP endpoint returning {"success":true,"connected":true}	yes

A bare TCP connect would at least establish that something is listening. The live check does not
establish even that. Note unsigned=true in the query string as well.

What this changes about the fix

The direction in the issue — "complete a real mining.subscribe in the probe" — is right, but the work
is larger than upgrading probe_stratum_port, because that function is not wired in. The challenger
has to stop asking and start connecting:

1. Call StratumVerifier::verify_sv1 (handlers.rs:232) against the target's advertised stratum port.
   It already sends {"id":1,"method":"mining.subscribe","params":["ghost-verify/1.0"]} and validates
   the response shape. It is written, tested, and unused.
2. Derive the verdict from the challenger's own handshake result, not from resp.success && resp.connected.
3. Keep the existing inconclusive-vs-fail discipline: host resolution failure or a challenger-side
   network error stays inconclusive (None), never a false fail. A successful TCP connect followed
   by no valid mining.subscribe reply is a genuine fail — that is precisely the nc -l 3333 case.
4. Height-gate it. This changes what qualifies a node for payout, so it cannot flip mid-fleet.

Layering note for whoever implements it

ghost-verification does not depend on ghost-pool, so it cannot read the gate constants where the
other nine live. The gate belongs in ghost-pool with the rest, surfaced to the client as a flag on
its config and set by the caller — not duplicated in ghost-verification, which would let the two
disagree about which rule is in force.

[defenwycke]

Retracting my own "correction" — the original description was right

Earlier in this issue I claimed that probe_stratum_port is not in the live path (only its own tests
call it) and that the live Public Mining check is an HTTP GET asking the target whether it serves
stratum, with the verdict resp.success && resp.connected, both self-reported.

Both claims were wrong.

• probe_stratum_port is live. It is called from crates/ghost-verification/src/task.rs:1642, by
  the challenger, against the target's stratum port — exactly as this issue described from the start.
• The HTTP path I pointed at is VerificationClient::verify_all_capabilities, which has no callers
  anywhere in the workspace. It is dead code, and the resp.success && resp.connected verdict in it
  never runs.

I grepped client.rs for callers, found only tests, and concluded from one file what needed the whole
crate. So the bar is what the issue said: a bare TCP connect, which nc -l 3333 passes — not the
weaker "a node with no stratum port at all passes". Apologies for the noise; the original triage was
better than my revision of it.

What has landed

STRATUM_HANDSHAKE_PROOF_HEIGHT = 962_000, armed.

Below the gate: the existing connect test. At and above it, the challenger completes a real
mining.subscribe via StratumVerifier::verify_sv1 — which was already written, tested, and unused —
and requires a well-formed JSON-RPC reply. That converts Public Mining's +3 from honesty-graded to
proof-graded.

Three-valued verdict, unchanged in spirit from the existing probe:

• Some(true) — connected AND a valid subscribe reply
• Some(false) — reachable but not speaking stratum, or refused
• None — target host undeterminable; a challenger-side condition, recorded as no verdict at all,
  because failing an honest node for our own resolution problem is worse than missing a dishonest one

Height-gated because it changes what qualifies a node for payout: a node still applying the connect
test while its peers demand a handshake computes a different qualified set and the node-reward split
diverges. 962_000 is ~300 blocks after the median-adoption gate so the two changes cannot be confused
for each other, and ~6 days beyond the tip at arming.

Two conservatisms worth noting: if the tip cannot be read the node stays on the legacy probe rather
than guessing the gate is active, and stratum_proof_gate_height defaults to u64::MAX so a build
that forgets to wire the gate keeps the old behaviour instead of unilaterally demanding proof.

Tests, each mutation-checked: a socket that accepts and stays silent fails; a socket that replies
with rubbish fails (this one exists because removing the protocol requirement left the silent-socket
test green — a silent peer returns Err from the timeout, so valid_protocol was never consulted); a
refused port fails; an unknown host is inconclusive; and the gate defaults dormant.

Still open on this issue: Archive +5, Ghost Pay +4, and challenger diversity (H-7/H-8). Public
Mining +3 is done, so 3 of the 12 unearned shares are now earned.

[defenwycke]

Archive (+5): traced, and the dead half is the ASKING, not the answering

Verified across the workspace rather than one file, after the Stratum description in this issue turned
out to be wrong in both directions.

The responder is built. ChallengeType::ArchiveTx exists at challenge.rs:36, is handled at
server.rs:2720 (handler.get_transaction(&txid) → tx_data), and is routed at routes.rs:1178. The
client already accepts txid: Option<&str> (client.rs:712).

The challenger never asks. task.rs:1290 calls
verify_archive(&peer.http_address, Some(&block_hash), None) — the None is the txid. So every archive
challenge is header-only, and the tx path is unreachable from the live code even though both ends of it
exist. That is what "the ArchiveTx path is dead" means, and it is a smaller gap than it reads.

What tx-level verification can actually establish

TxData (challenge.rs:90) carries txid, block_hash, tx_index, size, input_count,
output_count — metadata, not raw tx hex. There is no merkle branch field.

That is still verifiable, just not cryptographically: the challenger already has its own node, so it can
compare every field against its own view of that transaction. A pruned node cannot answer at all,
and neither can an SPV client, which is exactly the population that currently passes.

Being precise about the limit: this is not proof-of-storage. An on-demand proxy that fetches the
block when challenged still passes — it just has to do real work to do so. Raising the bar from "holds
80-byte headers" to "can serve arbitrary historical transaction detail" is the improvement this issue
asks for, and it should not be described as more than that.
→ [[gotcha_archive_challenge_header_only]]

Plan, in two increments

Increment 1 — ask for a transaction and verify the metadata. No protocol change; every field and
parameter already exists.

• Challenger already picks a random block via get_random_block_with_merkle(). Extend it to also pull
  that block's tx list (get_block(hash, 2)) and choose a random txid, recording the expected
  tx_index, size, input_count, output_count from its own node.
• Ask verify_archive(addr, Some(block_hash), Some(txid)).
• Fail the challenge if the returned TxData disagrees on any field, or if tx_data is absent.
• Keep the existing inconclusive discipline: RPC unavailable on the challenger side already skips
  without recording a verdict, and must continue to.
• Behind a new DORMANT gate, because it changes what qualifies a node for payout and a node applying it
  while its peers do not would compute a different qualified set.

Increment 2 — merkle-branch binding. Needs a merkle_branch field on TxData (a protocol change),
the responder computing the branch for that txid, and the challenger verifying it against the block
header's merkle root, which it already fetches. verify_merkle_proof already exists in
crates/ghost-reconciliation/src/batch.rs and should be reused rather than reimplemented. This is what
makes the response cryptographically bound to the block instead of merely agreeing with the
challenger's view.

Increment 1 is worth landing on its own: it excludes pruned and SPV nodes today, without a wire format
change.

[defenwycke]

Correcting my own design note: do NOT reuse verify_merkle_proof

I wrote earlier in this issue that merkle-branch verification should "reuse verify_merkle_proof
(crates/ghost-reconciliation/src/batch.rs) rather than reimplement it". That is wrong and would fail
on every block. Recording it before anyone follows it.

That function is a Ghost-INTERNAL merkle construction for batch consensus, not Bitcoin's:

	verify_merkle_proof	Bitcoin
hash	single SHA-256	double SHA-256
odd node at a level	carried forward unchanged	duplicated (the CVE-2012-2459 quirk)
root derivation	includes a domain tag and the leaf count	neither

It is deliberately domain-separated so its roots cannot be confused with anything else — which is
correct for its own purpose and precisely why it cannot check a Bitcoin block's merkle root.

The right path: gettxoutproof + bitcoin::MerkleBlock

Bitcoin Core already produces exactly the artifact needed, and the bitcoin crate (0.32.8, already a
ghost-verification dependency) already parses and verifies it with the real rules including the
duplication quirk. Hand-rolling this is how the byte-order trap gets hit a second time.

Shape:

1. RPC: add get_tx_out_proof(txids, block_hash) to crates/ghost-common/src/rpc.rs. Not currently
   exposed — checked.
2. Wire: txout_proof: Option<String> on TxData, #[serde(default)] so it is additive and an old
   responder still deserialises.
3. Responder: populate it from the node's own RPC when a txid is requested.
4. Challenger: deserialise into MerkleBlock, then require
   • its header's merkle root equals the root the challenger independently fetched for that block, and
   • extract_matches yields exactly the requested txid, at the index the challenger expects.

Byte order is the trap to watch, same as H-13: txids are internal order in the tree, and
Txid::to_string() prints REVERSED. Any comparison must be on to_byte_array(), and the test must drive
the real decision path rather than re-implementing the transformation — two H-13 tests passed while
production rejected every share for exactly that reason.

Scope note

This is a wire change, so it needs the same deploy-before-activate sequence as #606's vote size: the
field ships, the whole fleet carries it, and only then can the gate arm. ARCHIVE_TX_PROOF_HEIGHT is
already u64::MAX, so that ordering is satisfied by construction — the metadata check landed tonight
behind the same dormant gate, and merkle binding extends it rather than needing a second gate.

What it does and does not prove

With merkle binding, the response is cryptographically bound to the block instead of merely agreeing
with the challenger's view. It is still not proof-of-custody — a proxy that fetches the block on
demand passes. Operator direction is that Hazync/HazedProof recursive proofs are the eventual answer for
real custody; merkle binding is the interim step and should not be described as more.

[defenwycke]

Re-checked against c49ed365b: three of the five items in the body are stale

The body reads as a redesign. Measured against current main, most of it is built. Recording what is
actually true so this stops misdirecting planning — every claim below is from the tree or from a
live probe, not from the note.

Stratum +3 — BUILT and armed

StratumVerifier::verify_sv1 is used in the challenge path:

crates/ghost-verification/src/client.rs:650
    let verifier = crate::handlers::StratumVerifier::new().with_timeout(self.config.timeout);

with the doc directly above it: "This connects and speaks the protocol: StratumVerifier::verify_sv1
sends a real …". Armed at STRATUM_HANDSHAKE_PROOF_HEIGHT = 962_000, roughly 5–6 days out at the
current rate. The body's "the handshake-capable StratumVerifier exists but is unused in the
challenge path" no longer holds.

Archive +5 — BUILT, dormant

ChallengeType::ArchiveTx exists (challenge.rs:36), with archive_tx_gate_height and
with_archive_tx_gate in task.rs. ARCHIVE_TX_PROOF_HEIGHT is u64::MAX, so it is unarmed, not
dead. "The ArchiveTx path is dead" no longer holds; the open question is whether to arm it.

Worth noting when deciding: the fleet is all-pruned and reports archive_mode: false, with zero
archive rows in 24h, so arming changes nothing for these eight nodes. It matters for multi-operator.

GhostPay +4 — the body's claim does not hold against the live service

"A node with no L2 passes with a random hex string." Probed vm1's ghost-pay on :8800 (the service
the challenger actually targets — task.rs says outright "GhostPay runs on port 8800, not 8080
(ghost-pool)"):

/verify/ghostpay
  -> success:true  epoch_state_hash 0286beae…  nonce_bound_proof: null

/verify/ghostpay?challenge_epoch=1&challenge_nonce=deadbeefcafe1234
  -> success:true  epoch_state_hash be3f6899…  nonce_bound_proof f6428562…

/verify/ghostpay?challenge_epoch=999999999&challenge_nonce=deadbeefcafe1234
  -> success:false  "Epoch 999999999 not found (current epoch: 44)"

The third is the control, and it fails correctly: a fabricated epoch is rejected rather than
answered. Different epochs return genuinely different state, and the nonce binding is computed when
a nonce is supplied. bins/ghost-pay/src/main.rs:3620 backs this — it resolves the epoch via
get_l2_block_at_epoch(challenge_epoch) and errors when absent.

What IS still true: epoch_proof is null in every response. So the state hash is anchored to a
real epoch that the responder must actually have, but there is no separate cryptographic proof
object. Whether that gap matters is a design question, not a defect — see below.

⚠ Note for anyone re-reading the code: the #616 DIAGNOSTIC comment in
ghost-verification/src/server.rs and the surrounding epoch_proof_fn discussion are about
ghost-pool :8080, which is not the service the challenger asks. #616 was closed for exactly
that reason. I misread it that way first and it produced a wrong conclusion; the comment should
probably be pruned or re-labelled.

H-8 verdict trust — BUILT

target_signed_response: Option<String> on the verification message (message.rs:558), with
ResultReVerifier / ChainReVerifier re-deriving the verdict in
bins/ghost-pool/src/verification_reverify.rs. "Stores passed verbatim with no evidence" no
longer holds.

H-7 challenger /24s — the one still open, and the least certain

challenger_assignment.rs dedups the pool to one candidate per distinct /24 before ranking,
and its subnet is documented as "the /24 subnet it was observed on" — observed, not
self-reported — with the assignment gate live. But I found no explicit reachability probe at the
claimed address, which is what the body asks for and what the no-bond Sybil defence rests on.

This is the item I am least confident about and it deserves its own dig rather than a line here.

What is actually left

1. Arm ARCHIVE_TX_PROOF? Code built and dormant; inert on this fleet; matters for
   multi-operator. Needs a height and a decision.
2. Is epoch_proof: null a real gap, given the state hash is nonce-bound and per-epoch and a
   bogus epoch is rejected? Design call.
3. Does H-7 still need an address-directed reachability probe, given subnets are observed at
   assignment time?

That is three decisions, not a redesign. Suggest re-scoping this issue accordingly rather than
carrying the original framing into v1 planning.

[defenwycke]

H-7 confirmed open, with the full path

Following up the one item I left uncertain. The /24 used for challenger diversity is
self-reported, laundered through the database. Full chain:

1. Self-report. A peer sends public_address in its own health ping
   (ghost-consensus/src/message.rs:437).
2. Stored verbatim. health_handler.rs:843-852:

   if needs_update && !ping.public_address.is_empty() {
       let mut peer = crate::peer::Peer::new(envelope.sender, ping.public_address.clone());
       ...
       debug!(node_id = %short_id, address = %ping.public_address, "Updated peer address from health ping");
   }

   The only validation is non-empty. Nothing probes it.
3. Persisted. queries.rs:3181/3395 — INSERT INTO nodes (... public_address ...) /
   UPDATE nodes SET ... public_address = COALESCE(?2, public_address).
4. Becomes the subnet. voterset_assignment_pool (queries.rs:6806):

   SELECT node_id, public_address FROM nodes
   WHERE payout_address IS NOT NULL AND payout_address != ''

   then public_address.as_deref().and_then(ipv4_subnet_24).
5. Becomes diversity. challenger_assignment.rs:112-124 dedups to "one representative per
   subnet" and excludes candidates sharing the target's subnet.

So a node that advertises an address in a /24 it does not occupy is counted as a distinct
challenger. That is exactly the H-7 claim, and it is the no-bond Sybil crux: the griefing resistance
rests on distinct-subnet majorities that are fabricable at zero cost.

Note the docstring on AssignmentCandidate.subnet reads "the /24 subnet it was observed on".
That wording is misleading — the value is reported, not observed — and is probably why this looked
closed on a skim. Worth fixing the comment regardless of when the probe lands.

What the fix needs to decide

The direction in the body is "count toward diversity only after passing a reachability probe at the
claimed address". Three things need settling before code:

1. Who probes, and how often. Every node probing every peer's claimed address is O(n²) and is
   itself a griefing vector. A probe result that is then gossiped re-introduces the trust problem
   this is meant to remove.
2. What happens to an unprobed or unreachable /24. Excluding it from diversity is the safe
   default, but on this fleet that could collapse the pool to a handful of subnets and make the
   assignment draw degenerate. Needs measuring against the live node set before choosing.
3. Whether the probe proves occupancy at all. Connecting to a claimed address proves something
   answers there, not that the claimant controls it — a node can advertise any reachable third-party
   address. Binding the probe to the node's identity key (challenge/response signed by node_id
   over the connection) is what actually closes it, and that is a larger change than a TCP dial.

Point 3 is the one that decides whether this is a small fix or a real protocol addition, so it
should be settled first.