Skip to content

test: simulation-mode acquisition acceptance suite#596

Open
hongquanli wants to merge 3 commits into
masterfrom
test/sim-acquisition-acceptance
Open

test: simulation-mode acquisition acceptance suite#596
hongquanli wants to merge 3 commits into
masterfrom
test/sim-acquisition-acceptance

Conversation

@hongquanli

Copy link
Copy Markdown
Contributor

Summary

End-to-end acceptance suite for simulated acquisitions — the safety net for the upcoming acquisition-engine refactor and protocol-v2 port. New software/tests/acceptance/ package (marker acceptance), driving MultiPointController in-process (no GUI, no QApplication — the controller uses plain threads and plain-callable callbacks) and asserting on observable artifacts only: files on disk and process state.

Scenario Test Pins
Full small acquisition test_full_small_acquisition 2×2 FOV × 2 ch × 3 z ⇒ exactly 24 images in 0/, 12 rows in per-timepoint coordinates.csv, 4 rows in root CSV, acquisition parameters.json contents, .done markers, no ERROR lines in acquisition.log
Format parity test_format_parity_ome_tiff_vs_zarr same geometry under OME_TIFF and ZARR_V3 ⇒ 16 planes each, identical plane shape (4168×4168) and dtype (uint16)
Abort mid-acquisition test_abort_mid_acquisition_then_reacquire clean stop after abort; a second acquisition succeeds in the same process; zero JobRunner subprocesses survive close() (the #582 orphan surface)
Backpressure regression test_zstack_completes_under_tight_byte_limit 12-image z-stack under a ~1.5-frame byte limit (~8× oversubscribed) completes instead of deadlocking; throttling verified engaged in logs
Multi-timepoint smoke test_multi_timepoint_smoke Nt=2 ⇒ 0/ and 1/ each with correct images, CSV rows, .done

Shared harness (harness.py + conftest.py) wraps the existing tests/control/test_stubs.py wiring; timeouts are deliberately generous for CI runners.

CI wiring

No workflow change needed: pytest's default collection picks up tests/acceptance/ under the existing python3 -m pytest step. Deselect locally with -m "not acceptance". Suite runtime: 5 passed in ~28 s locally (headless).

Behavior findings (pinned, not fixed — recorded per the session brief)

  1. acquisition parameters.json is written at start_new_experiment() time, not at run_acquisition() time — callers that configure NZ/Nt after creating the experiment get stale values in the JSON. Tests configure before creating; noted in the harness docstring.
  2. OME-TIFF output ignores the timepoint dir: files land at {experiment}/ome_tiff/{region}_{fov}.ome.tiff (one per FOV, series ZCYX), while individual-images mode writes under {experiment}/0/.
  3. Zarr v3 output also lands at the experiment root ({experiment}/zarr/{region}/fov_{n}.ome.zarr/0, 5D TCZYX), not under the per-timepoint path that ZarrWriterInfo.get_output_path's docstring suggests.
  4. The product's Zarr v3 stores are unreadable by the installed zarr-python 2.15 — tests read them back with tensorstore (the same library the product writes with), and the parity test importorskips it, matching test_zarr_writer.py.
  5. FILE_SAVING_OPTION is star-imported by multi_point_worker, so runtime format switching requires patching both the _def attribute and the worker's frozen binding — the conftest helper does this; a future cleanup could move the worker to the control._def.X runtime-access pattern.

Test plan

  • Full suite headless locally: python -m pytest tests/acceptance -q → 5 passed in 28.3 s
  • Abort test 3× consecutive green; backpressure 2× consecutive green
  • black --config software/pyproject.toml clean
  • CI on this PR runs the suite under Xvfb via the existing pytest step

Session-B note: this PR is the Sync-2 gate for the parallel-session plan (Session C's code phase starts when this is merged).

🤖 Generated with Claude Code

hongquanli and others added 3 commits July 12, 2026 16:31
End-to-end acceptance tests that drive MultiPointController directly
against simulated hardware (in-process, no GUI, no QApplication) and
assert on observable artifacts only: image files, coordinates.csv rows,
'acquisition parameters.json', .done markers, the per-acquisition log,
and subprocess state.

New tests/acceptance/ package (pytest marker: acceptance, registered in
its conftest; runs under the default pytest invocation, so CI picks it
up with no workflow change):

- test_full_acquisition.py — full small acquisition (2x2 FOV x 2ch x
  3z = 24 images + CSV/JSON/log assertions); OME_TIFF vs ZARR_V3 format
  parity (16 planes, identical plane shape/dtype; zarr read back via
  tensorstore, skipped where not installed); 2-timepoint smoke.
- test_abort.py — abort mid-acquisition: clean stop, a second
  acquisition succeeds in the same process, and no JobRunner subprocess
  survives close() (PR #582 regression surface).
- test_backpressure.py — z-stack under a ~1.5-frame byte limit
  completes instead of deadlocking (pins the byte-release-on-job-
  completion design; throttling verified engaged via logs).
- harness.py/conftest.py — shared AcquisitionHarness around
  tests/control/test_stubs.py wiring, artifact helpers, generous
  CI-safe timeouts, def-knob pinning (incl. patching the worker's
  star-imported FILE_SAVING_OPTION binding).

Suite runtime: 5 passed in ~28s locally.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review findings (Opus, all nits): fix harness docstring referencing a
nonexistent fixture; drop the redundant un-restored MERGE_CHANNELS module
write (acquisition_defaults owns it with restore); guard the backpressure
test's premise — assert on-disk acquisition bytes exceed 5x the byte
limit so a shrunken simulated frame fails loudly instead of passing
without exercising the deadlock path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…stant

The premise guard added in the previous commit fired in CI: the pristine
CI config crops the simulated camera to a quarter of the local frame
(~8.3 MiB vs ~33.1 MiB), so the hardcoded frame size overstated the
pressure and the guard correctly refused to pass. Snap one frame from
the simulated camera at test start and size the byte limit (1.5 frames)
from its actual nbytes — the 12-frame acquisition is then ~8x the limit
under any machine config, and the on-disk guard still verifies the
premise held.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@hongquanli

Copy link
Copy Markdown
Contributor Author

The review-added premise guard caught a real config divergence on its first CI run — CI's pristine configuration_Squid+.ini crops the simulated camera to ~8.3 MiB frames vs ~33.1 MiB locally, so the hardcoded frame size overstated the byte pressure (12 frames were only ~2× the limit in CI, not the intended ~8×). Fixed in fd156084: the test now snaps one frame at start and derives the 1.5-frame byte limit from its measured nbytes, making the ~8× oversubscription hold under any machine config; the on-disk guard still verifies the premise after the run.

Note: that CI run also ended with the flaky post-green exit segfault that #595 root-causes (3 of master's last 15 runs fail the same way) and fixes via SQUID_PYTEST_HARD_EXIT — until #595 lands, any PR can flake at interpreter exit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant