feat(testutil/validatormock): port Charon validatormock to Rust#418
Open
varex83 wants to merge 8 commits into
Open
feat(testutil/validatormock): port Charon validatormock to Rust#418varex83 wants to merge 8 commits into
varex83 wants to merge 8 commits into
Conversation
varex83agent
force-pushed
the
bohdan/validatormock-foundation
branch
from
9abe29e to
9f412fb
Compare
Mirrors `charon/testutil/validatormock` (Go) into Rust. This first phase lays the foundation that the propose/attest/synccomm/component ports build on: - `meta.rs` - SpecMeta + MetaSlot + MetaEpoch value types (port of meta.go). - `sign.rs` - `Sign` trait + `Signer` + `SignFunc = Arc<dyn Sign>` (Go's SignFunc + NewSigner) backed by `pluto-crypto` BlstImpl. - `validators.rs` - local `ActiveValidators` newtype + `active_validators(client)` helper. Avoids a `pluto-testutil -> pluto-app` dep, since `pluto-app` already dev-depends on this crate. - `capture.rs` - test-only `SubmissionCapture` wiremock helper. Equivalent of Go's `beaconMock.SubmitAttestationsFunc = ...` callback fields - mounts a high-priority `Mock` on `BeaconMock::server()` and records POST bodies. - `error.rs` - module-wide `Error` + `SignError` (thiserror). - `testdata/TestAttest_*.golden` - byte-for-byte copies of the Go fixtures used by the Phase-2 attest port. The Cargo cycle `pluto-eth2util <-> pluto-testutil` is broken by moving `pluto-testutil` from `[dependencies]` to `[dev-dependencies]` in `crates/eth2util/Cargo.toml`. The three call sites in `eth2util` (signing, eth2exp, enr tests) are all `#[cfg(test)]`, so the move is a no-op for production code. 11 unit tests pass; clippy + nightly fmt clean.
Ports `charon/testutil/validatormock/attest.go` to Rust. SlotAttester drives
Prepare -> Attest -> Aggregate for a single slot, gated by Arc<CloseOnce>
close-once flags (AtomicBool + Notify) that mirror Go's `chan struct{}` ready
signals; mutable state lives behind Arc<Mutex<_>> so all entry points stay
`&self`.
Because the Rust eth2api client encodes attestations using the Electra
SingleAttestation shape (incompatible with Go's VersionedAttestation JSON
captured in the goldens), the two submit endpoints
(`POST /eth/v2/beacon/pool/attestations` and
`POST /eth/v2/validator/aggregate_and_proofs`) are POSTed as raw JSON whose
structure matches `eth2spec.VersionedAttestation` /
`*SubmitAggregateAttestationsOpts`. All other beacon-node interactions use
the generated client.
TestAttest matches Go's golden output for DutyFactor {0, 1} on validator
set A via assert_json_eq, with submissions sorted by data.index to match
the Go test's deterministic ordering.
Also adds Eth2Exp and Submit variants to validatormock::Error, and a
ValidatorSet-driven POST `/eth/v1/beacon/states/head/validators` plus a
BeaconCommitteeSelections echo route in the tests because the beaconmock
defaults don't cover those DV-only paths.
…er reg) Ports `propose_block` and `register` from `charon/testutil/validatormock/propose.go`. Versioned dispatch matches the Go switch on `eth2spec.DataVersion`; blinded path is gated on the proposal's `execution_payload_blinded` flag (the Pluto generated client's analog of Go's `block.Blinded`). Phase0/Altair branches return `Error::UnsupportedVariant` until the Pluto typed surface for them is ready; Bellatrix .. Fulu (full + blinded) are implemented in full. `register` ports the *intended* behaviour of Go's `Register`: it switches on the input registration's `Version` (Go switches on the zero-value-initialized signed registration version, which always falls through to V1 — a Go quirk noted inline). Tests mirror `TestProposeBlock` and `TestProposeBlindedBlock`; variants whose random fixtures don't yet exist in `pluto-testutil::random` are `#[ignore]`d with a TODO pointing at the missing helper.
Ports `charon/testutil/validatormock/synccomm.go` to Rust. SyncCommMember
drives PrepareEpoch → PrepareSlot → Message → Aggregate; per-slot ready flags
use Arc<tokio::sync::OnceCell<()>> to mirror Go's lazy `chan struct{}` maps.
TestGetSubcommittees ports the Go internal test verbatim.
Ports `charon/testutil/validatormock/component.go` to Rust, completing the validatormock module. `Component` drives the duty-by-duty workflow across a sliding window of attesters (per slot) and sync-committee members (per epoch) for the configured pubkeys. - `clock.rs`: `Clock` trait + `SystemClock` (production) + `FakeClock` (tests). `FakeClock` is an explicit-advance clock; pending sleepers register a `oneshot::Sender` and fire when `advance` / `advance_to` moves past their deadline. Avoids `tokio::time::pause()`, which interacts poorly with `wiremock::MockServer` (Plan agent flagged this). - `component.rs`: scheduler built around `tokio::sync::mpsc` + `tokio::spawn` + `tokio_util::sync::CancellationToken`. `Component::builder()` returns a handle that owns the consumer task; `shutdown().await` cancels gracefully and `Drop` cancels for the panic path. `run_duty_via_inner` matches the Go `runDuty` switch on `pluto_core::types::DutyType`. - `duties_for_slot` + `duty_start_times` mirror Go's `dutyStartTimeFuncsByDuty` table, including the half-/third-slot offsets for attest/aggregate/sync messages. Tests cover the start-up swallow window (delay_start_slots = 2), the duty-start-time arithmetic, and the duty-collection algorithm. Driving the full Component through 2 epochs with FakeClock would require comprehensive post_state_validators mounting on BeaconMock; the slot_ticked test asserts the epoch-window machinery instead and shuts down cleanly.
… signal The previous `wait_ready(cell)` design parked consumers inside `OnceCell::get_or_init(noop_pending)`. The first waiter acquired the OnceCell's init semaphore permit and held it forever inside a never- resolving `std::future::pending` future. A producer's subsequent `cell.set(())` returned `SetError::InitializingError` because the permit was held, leaving the waiter parked indefinitely — a hard deadlock reachable whenever `aggregate`/`message`/`prepare_slot` ran ahead of its corresponding setter (legal under the scheduler's spawn-order non-determinism). Hoist `attest.rs`'s `CloseOnce` (AtomicBool + Notify) into a shared `validatormock::close_once` module and apply it to synccomm's `duties_ok`, `selections_ok`, and `block_root_ok` signals. The primitive notifies *every* registered waiter on close (`notify_waiters`) and the `wait` loop re-checks `closed` after registering interest, so the producer-before-consumer and consumer-before-producer schedules both resolve correctly. `set_*` helpers lose their `Result` return type — `CloseOnce::close` is idempotent (matching the deliberate Go divergence already documented on `attest.rs::SlotAttester::prepare`). Includes a regression test `waiter_ahead_of_producer_still_wakes` that exercises eight waiters parked before close.
…shutdown `run_scheduler` previously dropped each `tokio::spawn` JoinHandle, so duty tasks could outlive `Component::shutdown().await`. In tests this leaks in-flight HTTP requests against `wiremock::MockServer` across test boundaries, surfacing as `received unexpected request` warnings or order-dependent failures. Switch to `tokio::task::JoinSet` to track duty tasks; reap finished ones inside the main select, and drain the set before the scheduler returns. `Component::shutdown` awaits the scheduler task, so once the scheduler returns every duty has been joined. Also documents the `Drop` impl to make explicit that `shutdown().await` is required for clean drainage — `Drop` cannot `.await`.
…egistration
Go's `runDuty` has no `case` for `core.DutyBuilderRegistration`, so it
falls through to the `default:` arm and returns
`errors.New("unexpected duty")`
(charon/testutil/validatormock/component.go:305-306). The Rust port was
silently returning `Ok(())`, swallowing what Go surfaces as a logged
warning every epoch (the duty is scheduled by `duty_start_times` —
matching Go's `dutyStartTimeFuncsByDuty[BuilderRegistration]`).
Return `Error::Malformed("unexpected duty: DutyBuilderRegistration")`
so the scheduler's `warn!` line emits a duty-failed warning each epoch,
restoring functional equivalence with Go.
varex83agent
force-pushed
the
bohdan/validatormock-foundation
branch
from
9f412fb to
92e1eed
Compare
iamquang95
approved these changes
May 22, 2026
| // `cancel.cancelled()` in its outer select; bodies already running run | ||
| // to completion (mirroring Go's `Run`, which lets the duty finish | ||
| // before returning from the loop). | ||
| while duties.join_next().await.is_some() {} |
Collaborator
There was a problem hiding this comment.
if prepare() fails before closing readiness signals, later attest/aggregate/sync tasks wait forever; shutdown() cancels then drains JoinSet, so it can block forever
| atts: &[VersionedAttestationJson], | ||
| ) -> Result<()> { | ||
| const ENDPOINT: &str = "/eth/v2/beacon/pool/attestations"; | ||
| submit_json(eth2_cl, ENDPOINT, atts).await |
Collaborator
There was a problem hiding this comment.
In Charon this goes through the eth2 client SubmitAttestations, so it return the real submit error. This is only json from go golder file, which may not match the actual beacon API shape
| common: CommonOpts::default(), | ||
| signed_aggregate_and_proofs: aggs, | ||
| }; | ||
| submit_json(eth2_cl, ENDPOINT, &body).await |
Collaborator
There was a problem hiding this comment.
same as submit_attestations
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ports
charon/testutil/validatormock(Go) to Rust undercrates/testutil/src/validatormock/. Mirrors the 5 Go files file-per-concern with full functional parity:meta.rs—SpecMeta/MetaSlot/MetaEpochslot/epoch arithmeticsign.rs—Signtrait +Signer+SignFunc = Arc<dyn Sign>(Go'sSignFunc+NewSigner)validators.rs— localActiveValidatorsnewtype + helper (avoids apluto-testutil → pluto-appdependency cycle)capture.rs—SubmissionCapturewiremock helper, the test-side replacement for Go'sbeaconMock.SubmitAttestationsFunc = ...callback fieldsattest.rs—SlotAttesterwithPrepare/Attest/Aggregate; matches Go goldens byte-for-byte viaassert_json_diffpropose.rs—propose_block+register; Bellatrix→Fulu, blinded + full variantssynccomm.rs—SyncCommMember+TestGetSubcommitteescomponent.rs— scheduler (tokio::sync::mpsc+CancellationToken+ injectableClock)clock.rs—Clocktrait +SystemClock+ explicit-advanceFakeClockerror.rs— unifiedthiserror::Errortestdata/TestAttest_{0,1}_{attestations,aggregations}.golden— byte-for-byte fromcharon/testutil/validatormock/testdata/Based on
bohdan/beaconmockapi(#416). Will rebase ontomainonce #416 merges.Design notes
close(chan struct{})ready signals are modelled per-module:attest.rsusesAtomicBool+tokio::sync::Notify;synccomm.rsusesArc<tokio::sync::OnceCell<()>>(lazy per-slot map).tokio::spawn+mpsc+CancellationToken.clockwork.FakeClockmaps to an injectableClocktrait.tokio::time::pause()was rejected — it interacts poorly withwiremock::MockServer.pluto-eth2util ↔ pluto-testutilCargo cycle was fixed:pluto-testutilmoved from[dependencies]to[dev-dependencies]incrates/eth2util/Cargo.toml. The 3 call sites in eth2util (signing, eth2exp, enr tests) are all#[cfg(test)], so this is a no-op for production code.Workflow
Built across 4 phases on parallel git worktrees:
Test plan
cargo +nightly fmt --all --check— cleancargo clippy --workspace --all-targets --all-features -- -D warnings— cleancargo test --workspace --all-features— all green; 73 tests pass inpluto-testutil(6#[ignore]d propose tests waiting on missingRandomCapella/Deneb/Electra/Fulu*Proposalhelpers inpluto-testutil::random— follow-up)cargo deny check— cleantestdata/TestAttest_{0,1}_{attestations,aggregations}.goldenbyte-for-byte (structural viaassert_json_diff)Known follow-ups (out of scope)
RandomCapella/Deneb/Electra/Fulu{Versioned,Blinded}Proposalhelpers topluto-testutil::randomto un-ignore 6 propose tests.POST /eth/v1/beacon/states/{state_id}/validatorsmount toBeaconMockso callers don't have to inline it.registerdeviates from Charon'sswitch signedRegistration.Version(always zero-valued V1) and instead switches on the input registration'sVersion. Inline comment notes the Go quirk; please confirm intent.