Skip to content

feat: acquisition method schema v2 — FOV patterns, per-well Z, tilted-plate pre-AF (GUI + API)#593

Draft
hongquanli wants to merge 19 commits into
feat/squid-core-servicefrom
feat/acquisition-schema-v2
Draft

feat: acquisition method schema v2 — FOV patterns, per-well Z, tilted-plate pre-AF (GUI + API)#593
hongquanli wants to merge 19 commits into
feat/squid-core-servicefrom
feat/acquisition-schema-v2

Conversation

@hongquanli

Copy link
Copy Markdown
Contributor

Summary

Acquisition-method schema v2: per-well FOV patterns, per-well Z control, and tilted-plate pre-autofocus positioning — with the GUI and the REST/method API both reading and writing the evolved schema. Realizes the "get YAML acquisition straight" goal from the core-service design docs.

Draft / stacked PR. Base is feat/squid-core-service (#578), not master — the diff shows only schema-v2's changes. Do not merge until #578 lands; GitHub will retarget this to master automatically when #578 merges and its branch is deleted. The first commit (port per-region laser-AF offset core hook from PR #562) is a forward-port of the #562 worker hook this builds on; it becomes a no-op on rebase once #562 or #578 carries it.

What's in it

FOV patterns (wellplate_scan.fov_pattern, per well):

  • coverage — area tiling (scan_size_mm + overlap_percent, with shape clipping). Today's behavior; the flat scan_size_mm/overlap_percent shorthand still means coverage.
  • centered_grid — an exact nx×ny montage centered on each well.
  • grid_subset — a chosen set of tiles: [[row,col],…] from that grid, identical in every well, with deterministic row-major indexing (i = row*nx + col).
  • randomn_fovs points sampled inside each well, reproducible-but-independent per well via sha256(seed:well) (never hash()).

Per-well Z — two distinct mechanisms (a real design point, kept separate on purpose):

  • well_z_offsets_um (TYPE-1, AF target offset): shifts where laser AF settles per well, µm from the AF reference plane. Requires laser_af; range-checked at preflight. Consumed via the per-region AF-offset hook (worker anchors at the reference, then applies the offset open-loop).
  • z_plan (TYPE-2, pre-AF stage positioning): for tilted plates, moves the stage to a fitted plane before AF so the laser spot lands within capture range. points: (3 [x,y,z]) bakes a known plane into the FOV coordinates; generate: true measures a 3-corner plane at run start (now works alongside reflection AF — the contrast-AF gate was loosened and the generated plane is baked into coordinates).

GUI + API both on v2:

  • GUI wellplate saves emit wells: + fov_pattern: for Select-Wells mode (breaking: drops regions[].center_mm there — manual/coordinate modes keep it). The reader loads both old and new forms and cleanly rejects (with a warning that now propagates as a load failure) patterns it can't yet render.
  • API/service-driven runs save a faithful v2 record too — the real fov_pattern is threaded to the acquisition-record writer, so a centered_grid/grid_subset/random method (and inline grid requests) round-trip as v2 instead of flattening to legacy regions.
  • Method-registry summaries expose pattern; quickstart-api.md + core-service-api.md document all three fields.

Backward compatibility

  • Old regions[].center_mm YAML still parses and loads (service and GUI reader) and still round-trips as regions on API legacy-method runs.
  • Non-GUI acquisition entry points are unaffected unless a method opts into the new fields.
  • Breaking change is scoped to what the GUI writes for Select-Wells saves (nothing YAML-related has shipped).

Testing

  • 98 schema-v2 tests (loader, four patterns, well_z_offsets preflight + resolution, z_plan points/generate, GUI writer/reader, API v2 round-trip incl. grid, registry) pass together.
  • Full suite green: 1636 passed, 0 real failures (run split to avoid an unrelated Qt-under-load hang in test_widgets.py, which passes 135/135 in isolation). black==25.12.0 + ruff F401/F841 clean on changed files.
  • Built via 10 spec-driven tasks, each independently implemented and reviewed, plus a final whole-branch review. Three merge-blockers were found and fixed during review: a YAML-load abort that reported success (could start an acquisition with stale well selection), z_plan colinearity surfacing as a 500 after preflight passed, and the loosened gen-map gate leaving a job stuck in ACQUIRING on a bounds/measurement failure.

Reviewer notes

  • Read order: acquisition_yaml_loader.py (schema + validation) → squid_service/service.py _configure_regions/_add_*_region (pattern dispatch), _resolve_well_z_offsets + preflight, _apply_z_plan_pointsmulti_point_controller.py (the one production-controller behavior change: the loosened elif self.gen_focus_map: gate + reflection-AF plane baking) → the writer threading (fov_pattern on AcquisitionParameters) → GUI widgets reader/writer.
  • Deferred to follow-up (from the whole-branch review, none blocking): grid_subset/random geometry faults raised at start rather than preflight (and not recorded to fault history); baked z not stage-Z-range-checked (relies on stage clamp); a picker UI for the new patterns in the GUI (API/method support does not wait for it); minor message/DRY polish.

🤖 Generated with Claude Code

hongquanli and others added 19 commits July 10, 2026 22:35
Non-GUI subset of PR #562 (per-region-laser-af-offset @ 9d33d54):
AcquisitionParameters.region_laser_af_offsets, the controller setter +
build_params threading + consume-and-clear, the worker's anchor-at-
reference + apply_relative_offset_um consumption, and the open-loop
offset method on LaserAutofocusController. Schema-v2's well_z_offsets_um
plumbs into this hook. If #562 merges first, this commit becomes a no-op
on rebase and should be dropped.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…l (schema v2)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…k with preflight validation

Resolve per-well laser-AF Z offsets (listed value, else "default", zeros omitted)
and push them into MultiPointController.set_region_laser_af_offsets on the yaml
start path after regions are configured. Add a preflight check that offsets
require effective laser AF (INVALID_PARAM) and that every offset (including
"default") is within the laser AF range (INVALID_PARAM_OUT_OF_RANGE).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…_offsets_um

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-laser-AF)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add direct MultiPointController coverage for run_acquisition's reflection-AF
plane-baking branch (gen_focus_map + do_reflection_af): the generated focus
plane is baked into every FOV z instead of enabling the contrast focus-map.

The 3-corner measurement and laser-AF closed-loop move are stubbed so the plane
is known and the run is deterministic (no reliance on real sim contrast/laser AF).
Asserts: every FOV becomes (x, y, plane_z(x)) within 1e-6, set_focus_map_use is
never called with True in the laser-AF path, and the run completes without abort.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ode (schema v2)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… pattern keys

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…or API-only patterns

WellplateMultiPointWidget._apply_yaml_settings now loads schema-v2 acquisitions:
- wells-by-name (range-expanded via squid_service.wells.parse_well_names) select
  the named wells and apply the coverage fov_pattern's scan_size_mm / overlap_percent
  / shape to the widget controls.
- Non-coverage patterns (centered_grid/grid_subset/random) are not representable in
  the GUI yet, so the load is aborted with a clear QMessageBox.warning directing the
  user to the API/method registry. The signal-unblock finally: still runs.
- Legacy wellplate_regions path is unchanged (now the elif branch).

Adds GUI tests (in the CI-skipped test_HighContentScreeningGui.py, run locally with
QT_API=pyqt5) covering both the coverage load and the non-coverage rejection.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…lse success

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Task-0 port of the per-region laser-AF hook (c230268) changed
perform_autofocus to read self.region_laser_af_offsets but did not carry
#562's matching stub fix (10b2475), so two _af_stub-based tests
regressed vs the pre-hook base. Real worker sets this in __init__.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ed at run start)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rly returns (B2)

B1: _validate_z_plan now rejects XY-colinear point sets with a clear ValueError,
so a colinear z_plan fails cleanly as INVALID_PARAM at preflight (via check_yaml)
instead of letting interpolate_plane raise a raw ValueError inside
start_acquisition._apply_z_plan_points -> misleading HARDWARE_FAULT 500 after
preflight already passed.

B2: run_acquisition's gen_focus_map branch had two early returns
(`if not bounds` and `except ValueError`) that skipped acquisition-finished
signaling. After the Task 6 gate loosening, reflection-AF runs reach them, so
finish never fired: the service job stuck in ACQUIRING forever and the GUI stayed
disabled. Each early return now emits self.callbacks.signal_acquisition_finished()
(same as the validate_acquisition_settings early return); the service classifies
the finish-without-start as a validation failure and completes the job as FAILURE,
returning the instrument to INITIALIZED and re-enabling the GUI.

Tests: colinear z_plan rejected at loader + service preflight (INVALID_PARAM, no
500); non-colinear plane still parses; no-bounds gen_focus_map early return still
fires the finished signal (no hang).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…n threaded to the writer)

Thread the real fov_pattern from the service to the acquisition-record writer,
mirroring the Task-5 region_laser_af_offsets threading: an AcquisitionParameters
field, a controller attribute + set_fov_pattern setter, consume-and-clear at run
start, and build_params passthrough. The writer's v2 wellplate branch now emits
params.fov_pattern when present, else the existing coverage-from-scan-size default
(GUI Select-Wells save stays byte-unchanged). The service sets xy_mode
deterministically on every path (wells -> Select Wells + real pattern, legacy
regions -> Manual + None, grid -> Select Wells + centered_grid) so the writer's
v2-vs-legacy discriminator is never inherited stale, and consume-and-clear
guarantees no pattern leak between runs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@hongquanli

Copy link
Copy Markdown
Contributor Author

Review guide

Stacked / draft. Base is feat/squid-core-service (#578), so the diff above is only schema-v2. Don't merge until #578 lands (GitHub retargets this to master automatically then). First commit forward-ports the #562 worker hook and no-ops away on rebase once #562/#578 carries it. 19 commits, +1.5k, CI green on Linux.

How it was built: 10 spec-driven tasks, each independently implemented + reviewed, plus a final whole-branch review — which found and fixed 3 merge-blockers (a YAML-load abort that reported success and could start an acquisition with stale wells; z_plan colinearity surfacing as a 500 after preflight passed; the loosened gen-map gate leaving a job stuck ACQUIRING). Line-level correctness was reviewed exhaustively; spend your time on the judgment calls below.

Reading order (each layer builds on the previous):

  1. control/acquisition_yaml_loader.py — the v2 schema + all validation (fov_pattern 4 types, well_z_offsets_um, z_plan). Sets the vocabulary.
  2. squid_service/service.py_configure_regions/_add_*_region (pattern dispatch), _resolve_well_z_offsets + preflight, _apply_z_plan_points.
  3. control/core/multi_point_controller.pythe one production-controller behavior change: the loosened elif self.gen_focus_map: gate + the reflection-AF plane-baking branch. Also the fov_pattern/offset threading to the writer.
  4. GUI control/widgets.py reader/writer + multi_point_controller._save_acquisition_yaml.

Where human judgment matters most:

  • The two Z mechanisms are deliberately distinctwell_z_offsets_um (TYPE-1: shifts where laser AF settles, µm from the reference plane) vs z_plan (TYPE-2: moves the stage to a fitted plane before AF so a tilted plate lands in capture range). Confirm this split matches how you focus at the bench, and that the composition order (z_plan coarse-position → laser AF anchors at reference + offset) is right.
  • Tilted-plate generate: true measures a 3-corner contrast-AF plane at run start even on a laser-AF run — worth a domain eye.
  • Breaking change, scoped: GUI Select-Wells saves now emit wells:+fov_pattern: and drop regions[].center_mm there only (manual/coordinate modes unchanged; old files still load). Nothing YAML-related has shipped, so this was sanctioned.
  • Determinism: random uses sha256(seed:well) (reproducible-but-independent per well), grid_subset forces row-major indexing.

Hardware validation (sim can't prove these): a real tilted-plate acquisition (z_plan) and a real per-well-offset acquisition (well_z_offsets_um) on an instrument — the laser-AF-in-capture-range and offset-anchoring behavior are exactly the untested seam.

Deferred to follow-up (none blocking): grid_subset/random geometry faults raised at start not preflight; baked z not stage-Z-range-checked (relies on stage clamp); a GUI picker for the new patterns (API/method support doesn't wait for it).

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant