fix: improved brush shape for rotated images

[Devu-trenser]

Context

The brush tools wrote the wrong voxels on a rotated or oblique viewport. A
circular brush and a spherical brush each drew a distorted shape. A rectangular
brush drew an unstable shape, and the rectangle changed when the user changed
the view orientation.

Fixes: #2651

FlyWheel.io contributed the first version of this pull
request.

The cause

Each brush fill did two steps. First the fill made an axis-aligned bounding box
in IJK index space. Then the fill tested every voxel in that box with a
predicate.

An oblique brush breaks both steps:

• The box is large along all three axes, but the brush covers only a thin sheet
  inside the box. The fill therefore tests O(N³) voxels to write O(N²)
  voxels.
• The predicate must reject every voxel that the box holds and the sheet does
  not contain. The predicate does this with a depth tolerance. A small
  tolerance leaves holes in the sheet. A large tolerance writes voxels in the
  neighbouring slices. No single value is correct for every orientation.

The second point is the defect. The first point is only a cost.

Changes

main carries the shared oblique-capable voxel iterator of #2893. This pull
request moves the brush fills onto that iterator.

Each brush is now a VoxelSlabShape on the view plane, and
csUtils.voxelSlab.iterateVoxelsInShape gives the voxels of that shape:

Brush	Shape	Depth along the normal
Circle and ellipse	createEllipseShape, flat	The view slab thickness
Sphere	createCircleShape with depthRadius	Its own radius
Rectangle	createRectangleShape, flat	One voxel

The iterator gives exact integer runs of voxels along one voxel axis, and the
slab of Rule M bounds those runs along the normal. No fill holds a depth
tolerance any more, because the slab bound is exact at every orientation.

The area annotation tools measure with the same iterator, the same shapes and
the same rule. A brush therefore writes the voxels that an annotation of the
same shape reports.

New file in core: createUnionShape

The shared design had no way to describe a stroke. A stroke writes the union of
one disc for each sample of the pointer, and one shape cannot describe that
union. createUnionShape merges the runs of its members into one disjoint
sequence in ascending order. Two results follow:

• The iterator visits a voxel that two discs share one time only. A fill that
  writes a voxel two times records two undo entries for that voxel.
• The cost is the member count for each row. A per-voxel predicate over the
  same members costs the member count for every voxel of the bounding box.

New file in tools: brushVoxelSlab.ts

This file holds the three shape builders, and the function that gives the
voxels to the regionFill composition. regionFill keeps the bounding box walk
as a fallback path, for a degenerate brush and for a strategy that this pull
request does not move.

The plane of the rectangle

The rectangle fill now takes its plane from the four corners that the user
drew, and no longer takes the plane from the camera. orderRectangleCorners
gives the corners a stable winding order, so the two edges and the diagonal are
always the same pairs of corners. This is what makes the rectangle stable when
the view orientation changes. RectangleScissorsTool passes viewPlaneNormal
and viewUp to the strategy, which the fill needs for a degenerate rectangle.

Shared index bounds

sampleAreaAnnotationVoxels exports its index bounds helper as
getShapeIndexBounds. The annotation code and the brush code now bound their
iteration with one implementation.

The files

The diff is 17 files, 2357 insertions and 186 deletions.

File	Change
core/.../voxelSlab/shapes/createUnionShape.ts	New. The union of shapes
core/test/voxelSlabUnionShape.jest.js	New. Tests for the union
tools/.../strategies/utils/brushVoxelSlab.ts	New. The brush shapes and fill
tools/.../strategies/__tests__/brushVoxelSlab.spec.ts	New. Tests for the fills
tools/.../strategies/__tests__/fillRectangle.spec.ts	New. Tests for the corner order
docs/.../segmentation/planar-fill-iteration.md	New. The fill contract
tools/.../strategies/fillCircle.ts	Builds the circle fill
tools/.../strategies/fillSphere.ts	Builds the sphere fill
tools/.../strategies/fillRectangle.ts	Builds the rectangle fill
tools/.../strategies/compositions/regionFill.ts	Reads the fill, or falls back
tools/.../strategies/BrushStrategy.ts	The brushVoxelSlabFill field
tools/.../segmentation/RectangleScissorsTool.ts	Passes the view axes
tools/src/utilities/sampleAreaAnnotationVoxels.ts	Exports getShapeIndexBounds
core/.../voxelSlab/shapes/index.ts	Exports the union
docs/sidebars.js	Adds the new page
docs/.../migration-guides/5x/1-migration-notes.md	Adds a migration note
tools/examples/labelmapSegmentationTools/index.ts	The controls for the manual test

A note on the commit history of this branch

The branch first added a second oblique voxel iterator of its own, in
csUtils.obliqueIntegerIterator, csUtils.iterateOverPlane,
csUtils.getInPlaneSpacingAndXYDirections and
strategies/utils/obliqueIntegerFill.ts. #2893 then merged the shared iterator
into main. The two solve the same problem, so a later commit on this branch
removes the second iterator and keeps the shared one. None of those four
modules reached a release, and none of them appears in the diff of this pull
request. A reviewer who reads the commits one at a time will see them, and a
reviewer who reads only the diff will not.

The same commit returns two files to the version in main, so neither file
appears in the diff:

• getSubPixelSpacingAndXYDirections uses
  getEffectiveSpacingAlongDirection, which measures how far to step to cross
  one voxel. The branch had sent that path through
  getSpacingInNormalDirection, which answers a different question. The
  repository keeps one utility for one question.
• The fallback bounds of fillSphere follow the stroke again. The branch had
  replaced those bounds with a box around one centre, and that box clips a drag.

Results

• A circular brush, a spherical brush and a rectangular brush each write the
  correct shape at every view orientation.
• A rectangular brush is stable, because its plane comes from the corners and
  not from the camera.
• A thin view writes one oblique layer of voxels. A full-thickness view writes
  every layer through the slab. An area computation over a full-thickness fill
  must divide by the depth in voxels.
• A fill writes one layer of voxel centres. The old bounding box walk wrote
  voxels in the neighbouring slices, and this fill does not.
• One layer of voxels is still thicker than one oblique slice, because a voxel
  is a box and not a plane. The paint therefore shows on more than one oblique
  slice. The Testing section gives the measurements.

Testing

Automated tests, which I ran

• packages/core: 42 test suites and 728 tests. All tests pass.
• packages/tools: 55 test suites and 778 tests. All tests pass.
• tsc --noEmit reports no error for packages/core and for packages/tools.
• oxlint reports 0 errors. prettier --check reports no change for every
  file that this pull request touches.

packages/core/test/voxelSlabUnionShape.jest.js tests createUnionShape
against the brute-force reference implementation of Rule M. The test covers
oblique angles from 1 degree to 89 degrees, and it asserts that no voxel
appears two times.

packages/tools/src/tools/segmentation/strategies/__tests__/brushVoxelSlab.spec.ts
asserts that:

• a thin view writes one layer, and a thick slab writes every layer;
• a spherical brush writes a complete sphere, and writes no voxel outside that
  sphere, at every angle from 0 degrees to 90 degrees;
• every voxel that a fill writes passes the depth test of Rule M against the
  plane that the user drew. This test covers the exact defect of the old
  bounding box walk.

Manual test with a cornerstonejs example

This pull request updates the labelmapSegmentationTools example, so that a
reviewer can make a plane oblique and then paint on that plane. The example now
holds a crosshairs tool, a slider for an exact oblique angle, and slice
navigation on the mouse wheel.

Run the example:

pnpm install
pnpm example labelmapSegmentationTools

The command opens http://localhost:3000/, and shows an axial viewport, a
sagittal viewport and a coronal viewport of one CT volume.

The controls:

Control	Action
Left click	Paint with the tool of the first dropdown
Shift + left click	Erase with the circular brush
Middle click drag	Move the crosshairs, or drag a reference line
Middle click drag on a circle handle	Rotate the planes to an oblique angle
Mouse wheel	Navigate the slices of the viewport below the pointer
Ctrl + left click	Pan
Right click	Zoom
Crosshairs button	Turn the crosshairs off, and give the middle button to the pan tool
Axial Oblique Angle slider	Tilt the axial plane by an exact angle
Reset Cameras button	Return every viewport to its first orientation

The steps:

1. Move the Axial Oblique Angle slider to 45 degrees. The axial viewport now
   shows an oblique plane, and the axial reference line in the sagittal
   viewport is at 45 degrees.
2. Select CircularBrush in the first dropdown. Paint one stroke in the axial
   viewport.
3. Look at the sagittal viewport and at the coronal viewport. The stroke must
   make one continuous oblique line, and that line must hold no hole. Before
   this pull request the line held holes, or the paint reached the
   neighbouring slices.
4. Select SphereBrush, and paint at the same angle. The paint must show a
   circle in all three viewports.
5. Select RectangleScissorsTool, and drag a rectangle in the axial viewport.
   Then rotate the planes with a circle handle of the crosshairs. The paint
   must keep the shape of the rectangle that you drew.
6. Press Reset Cameras. Then rotate the planes with a circle handle of the
   crosshairs in the sagittal viewport, and do step 2 and step 3 again. The
   result must be the same for an angle that the crosshairs give.

What one layer of voxels looks like on an oblique plane

A reviewer who moves the mouse wheel one slice in the oblique viewport still
sees the paint. This is correct, and the reason is geometry, not the fill.

The example loads a CT volume with a voxel spacing of 0.977 mm by 0.977 mm by
3.27 mm. At an oblique angle of 45 degrees:

• The voxel thickness along the normal is 3.003 mm. This is the L1 support
  width of the voxel box, which getVoxelThicknessAlongNormal gives.
• One slice step along the same normal is 2.413 mm, which
  getSpacingInNormalDirection gives.

A voxel is a box, so one layer of voxels reaches about 1.5 mm to each side of
the plane, and the paint covers about 6 mm along the normal. Six millimetres is
about 2.5 slice steps. The paint therefore shows on 3 or 4 oblique slices, and
it is still one layer of voxels. The 3.27 mm slice thickness of the source data
causes this, and no fill can make the layer thinner.

Measurements in a browser

I painted in the example in Chromium, and then I read the labelmap voxels with
a script. The paint is one layer, and the layer holds no hole.

One stroke at 45 degrees, 6287 voxels:

• The signed distance from a voxel centre to the plane is between -1.517 mm and
  +1.517 mm. This is 0.505 of the voxel thickness along the normal, so the
  paint is one layer of voxel centres.
• 3231 columns of the volume hold one painted voxel, and 1528 columns hold two.
  No column holds three. Two voxels at the step of the staircase is the minimum
  for an oblique layer that holds no hole.

One click at each angle. For each test the script takes about 15000 points on
the plane inside the disc, finds the nearest voxel of each point, and asks
whether the fill painted that voxel. A point that finds an unpainted voxel is a
hole:

Angle	Painted voxels	Holes
7 degrees	2120	0 of 15018
23 degrees	2232	0 of 15032
38 degrees	2068	0 of 14995
45 degrees	2004	0 of 14915
52 degrees	1748	0 of 15008
60 degrees	1550	0 of 14995

A note on the thickness that the fill uses

fillCircle reads viewport.getSlabThickness(). An orthographic volume
viewport with no slab returns RENDERING_DEFAULTS.MINIMUM_SLAB_THICKNESS,
which is 0.05 mm. That value is finite and greater than 0, so
resolveReferencePlaneThickness keeps it instead of the fallback of one voxel.
The half width of Rule M is therefore (0.05 + Tv) / 2, which is about
Tv / 2, and not the Tv that Rule M gives an annotation.

The measurements above show that this half width still leaves no hole. Tv is
an L1 width and the slice step is an L2 width, and an L1 width is never smaller
than an L2 width, so Tv / 2 always covers half a slice step. A half width of
Tv would paint a layer up to two times thicker, which a brush does not want.

A viewport with a real slab thickness is a different case, and I did not test
it. getSlabThickness returns a half thickness, so a thick slab fills half of
the depth that the viewport shows.

Checklist

PR

• My Pull Request title is descriptive, accurate and follows the
  semantic-release format and guidelines.

Code

• My code has been well-documented (function documentation, inline comments,
  etc.)

Public Documentation Updates

• The documentation page has been updated as necessary for any public API
  additions or removals.

Tested Environment

• "OS: Windows 11"
• "Node version: 24.20.0"
• "Browser: Chromium 147.0.7727.15, which Playwright 1.59.1 gives"

🤖 Generated with Claude Code

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Shared geometry helpers and an oblique integer voxel iterator are added, then wired into circle, rectangle, sphere, and region fill strategies. Polyline spacing now delegates to the shared helper. Tests and documentation cover the new behavior.

Changes

Planar spacing and fill iteration

Layer / File(s)	Summary
In-plane spacing and plane iteration packages/core/src/utilities/getInPlaneSpacingAndXYDirections.ts, packages/core/src/utilities/iterateOverPlane.ts, packages/core/src/utilities/index.ts, packages/tools/src/utilities/math/polyline/getSubPixelSpacingAndXYDirections.ts	getInPlaneSpacingAndXYDirections computes in-plane spacing and directions from image geometry, iterateOverPlane visits unique voxels across an oriented plane or slab, and the polyline spacing helper now delegates to the shared geometry path. The utilities barrel exports both helpers and related iterator types.
Oblique integer basis and voxel enumeration packages/core/src/utilities/obliqueIntegerIterator.ts, packages/core/src/utilities/index.ts	obliqueIntegerIterator defines the integer oblique basis, basis construction, range clipping, ellipsoid slice ranges, and voxel traversal helpers. The utilities barrel re-exports the module and related types.
Circle, rectangle, sphere, and region fill packages/tools/src/tools/segmentation/strategies/BrushStrategy.ts, packages/tools/src/tools/segmentation/strategies/compositions/regionFill.ts, packages/tools/src/tools/segmentation/strategies/fillCircle.ts, packages/tools/src/tools/segmentation/strategies/fillRectangle.ts, packages/tools/src/tools/segmentation/strategies/fillSphere.ts, packages/tools/src/tools/segmentation/strategies/utils/obliqueIntegerFill.ts	fillCircle, fillRectangle, and fillSphere now normalize view vectors, use voxel-center or ordered-corner geometry, and apply plane-based bounds or membership checks for oblique views. Rectangle corner ordering, oblique fill descriptors, and region fill wiring are updated accordingly.
Tests and documentation packages/core/test/utilities/*, packages/tools/src/tools/segmentation/strategies/__tests__/*, packages/docs/docs/behaviour/*, packages/docs/docs/concepts/cornerstone-tools/segmentation/planar-fill-iteration.md, packages/docs/docs/migration-guides/5x/*, packages/docs/sidebars.js	Tests cover the new geometry utilities, oblique integer iterator, and oblique fill wiring, while the docs sidebar, concept page, behaviour page, and migration notes describe planar fill iteration and oblique voxel behavior.

Estimated code review effort: 5 (Critical) | ~120 minutes

Suggested reviewers: sedghi

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title is concise, uses the semantic-release format, and accurately describes the main change: improving brush shapes for rotated images.
Description check	✅ Passed	The description is complete and relevant. It includes context, issue reference, detailed changes, expected results, automated and manual testing instructions, tested environment details, and completed…

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 4

🧹 Nitpick comments (3)

packages/tools/src/tools/segmentation/strategies/fillCircle.ts (1)

71-76: 💤 Low value

Corner ordering inconsistency with destructuring at line 314.

The return order is [topRight, topLeft, bottomRight, bottomLeft] but the destructuring at line 314 expects [topLeft, bottomRight, bottomLeft, topRight]. This mismatch will cause incorrect axis calculations when the corners are consumed.

However, looking at the usage at line 265, strokeCornersWorld is only used to compute boundsIJK via getBoundingBoxAroundShapeIJK, which is order-agnostic. The cornersInWorld used in createPointInEllipse comes from the canvas-derived corners at lines 223-226, not from createCircleCornersForCenter. So this ordering doesn't affect the actual shape test.

Consider aligning the return order for consistency

   return [
-    topRight as Types.Point3,
-    topLeft as Types.Point3,
-    bottomRight as Types.Point3,
-    bottomLeft as Types.Point3,
+    topLeft as Types.Point3,
+    bottomRight as Types.Point3,
+    bottomLeft as Types.Point3,
+    topRight as Types.Point3,
   ];

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/tools/src/tools/segmentation/strategies/fillCircle.ts` around lines
71 - 76, The return statement in the function (around lines 71-76) returns
corners in the order [topRight, topLeft, bottomRight, bottomLeft], but the
destructuring at line 314 expects them in the order [topLeft, bottomRight,
bottomLeft, topRight]. Reorder the return statement to match the expected
destructuring order at line 314. Change the return array to put topLeft first,
then bottomRight, then bottomLeft, then topRight to maintain consistency with
how the corners are consumed elsewhere in the code.

packages/tools/src/tools/segmentation/strategies/fillSphere.ts (1)

14-14: ⚡ Quick win

Unused import: getSphereBoundsInfoFromViewport.

The import is still present but the function is no longer used after the bounds computation was changed to use direct IJK calculation.

Remove unused import

-import { getSphereBoundsInfoFromViewport } from '../../../utilities/getSphereBoundsInfo';

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/tools/src/tools/segmentation/strategies/fillSphere.ts` at line 14,
Remove the unused import statement for getSphereBoundsInfoFromViewport from the
imports at the top of the fillSphere.ts file. This function is no longer needed
since the bounds computation has been changed to use direct IJK calculation
instead of relying on the getSphereBoundsInfoFromViewport utility function.

packages/tools/src/tools/segmentation/strategies/fillRectangle.ts (1)

151-160: 💤 Low value

Full projectedSpacing used as thickness may be too permissive.

The comment says "Using full projected spacing gives more stable voxel occupancy," but using the full spacing rather than half-spacing as a tolerance means voxels up to one full voxel away from the plane are included. This could include voxels from adjacent slices in thin-slice scenarios.

For consistency with fillCircle.ts which uses spacingInNormal / 2 for planeTolerance, consider whether this intentional difference is desired.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/tools/src/tools/segmentation/strategies/fillRectangle.ts` around
lines 151 - 160, The thickness variable is set to the full projectedSpacing
value, which may be overly permissive and include voxels from adjacent slices in
thin-slice scenarios. For consistency with fillCircle.ts which uses
spacingInNormal divided by 2 for planeTolerance, evaluate whether the thickness
assignment should be changed to use projectedSpacing divided by 2 instead of the
full value. Review the intention and adjust if the full spacing is not
intentionally required for this particular case.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/tools/src/tools/segmentation/strategies/fillCircle.ts`:
- Around line 330-338: The spacingInNormal calculation in the fillCircle
strategy currently assumes viewNormal is always available when calling
getSpacingInNormalDirection, but viewNormal is optional and may be undefined.
Refactor the conditional logic to explicitly check if viewNormal is defined
before attempting to call getSpacingInNormalDirection with the cast to
Types.Point3. If viewNormal is undefined, provide an appropriate fallback that
does not assume Z-axis spacing is suitable for all orientations. Additionally,
reconsider whether the current condition (spacing && direction) is sufficient,
as getSpacingInNormalDirection might handle cases where only one of these
parameters is defined.
- Around line 427-441: The ellipse equation test does not check the
strokePredicate when a point falls outside the static ellipse boundary (when
xTerm + yTerm > 1), unlike the sphere case which checks strokePredicate before
its sphere test. To fix this, after the ellipse equation check at line 434 (the
condition checking if xTerm + yTerm <= 1), add a check for the strokePredicate
similar to how it is handled in the sphere case. This will ensure that
stroke-based fills work correctly for ellipses during brush strokes by
evaluating the strokePredicate for points outside the static ellipse.

In `@packages/tools/src/tools/segmentation/strategies/fillRectangle.ts`:
- Around line 120-135: The axis construction in the fillRectangle function
assumes that after angular sorting, p0, p1, and p3 form adjacent edges from a
corner, but there is no validation that this assumption holds. If the sorting
produces a different order like diagonal pairs, then axisU and axisV would not
be orthogonal and the normal calculation would be incorrect. After obtaining
orderedPoints, validate adjacency by comparing distances from p0 to each other
point—the two shortest distances should correspond to p1 and p3 (the adjacent
edges), while the longest should be to p2 (the diagonal). If this condition is
not met, reorder the points so that p1 and p3 are correctly identified as the
adjacent corner points before constructing the axes and normal.

In `@packages/tools/src/tools/segmentation/strategies/fillSphere.ts`:
- Around line 97-112: The boundsIJK variable is computed without clamping to
valid image dimensions, unlike the fillRectangle.ts strategy. After calculating
the boundsIJK array using centerIJK and radiusIJK values, clamp each dimension's
min and max bounds to the range [0, dims[i]-1] for each of the three dimensions
I, J, and K, where dims are the image dimensions. This should be done before
assigning the clamped bounds to operationData.isInObjectBoundsIJK to prevent
out-of-bounds array access errors.

---

Nitpick comments:
In `@packages/tools/src/tools/segmentation/strategies/fillCircle.ts`:
- Around line 71-76: The return statement in the function (around lines 71-76)
returns corners in the order [topRight, topLeft, bottomRight, bottomLeft], but
the destructuring at line 314 expects them in the order [topLeft, bottomRight,
bottomLeft, topRight]. Reorder the return statement to match the expected
destructuring order at line 314. Change the return array to put topLeft first,
then bottomRight, then bottomLeft, then topRight to maintain consistency with
how the corners are consumed elsewhere in the code.

In `@packages/tools/src/tools/segmentation/strategies/fillRectangle.ts`:
- Around line 151-160: The thickness variable is set to the full
projectedSpacing value, which may be overly permissive and include voxels from
adjacent slices in thin-slice scenarios. For consistency with fillCircle.ts
which uses spacingInNormal divided by 2 for planeTolerance, evaluate whether the
thickness assignment should be changed to use projectedSpacing divided by 2
instead of the full value. Review the intention and adjust if the full spacing
is not intentionally required for this particular case.

In `@packages/tools/src/tools/segmentation/strategies/fillSphere.ts`:
- Line 14: Remove the unused import statement for
getSphereBoundsInfoFromViewport from the imports at the top of the fillSphere.ts
file. This function is no longer needed since the bounds computation has been
changed to use direct IJK calculation instead of relying on the
getSphereBoundsInfoFromViewport utility function.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: d759a66a-da29-4877-a2bd-3a73b4936c03

📥 Commits

Reviewing files that changed from the base of the PR and between 2769fcf and 756ba0a.

📒 Files selected for processing (3)

• packages/tools/src/tools/segmentation/strategies/fillCircle.ts
• packages/tools/src/tools/segmentation/strategies/fillRectangle.ts
• packages/tools/src/tools/segmentation/strategies/fillSphere.ts

[wayfarer3130]

I'm needing to make more exhaustive changes to correct a number of edge cases/flaws in how this is currently working. Will see if I can get this fixed this week still.

[sen-trenser]

I'm needing to make more exhaustive changes to correct a number of edge cases/flaws in how this is currently working. Will see if I can get this fixed this week still.

@wayfarer3130 Could you please let us know what changes and edge cases are planned? It would help us determine which of the above AI review comments to address with priority.

Thanks!

[wayfarer3130]

I'm needing to make more exhaustive changes to correct a number of edge cases/flaws in how this is currently working. Will see if I can get this fixed this week still.

@wayfarer3130 Could you please let us know what changes and edge cases are planned? It would help us determine which of the above AI review comments to address with priority.

Thanks!

The underlying problem for this PR and several others is a general handling of the oblique cut of the display versus the underlying data, and the differences between a planar oblique cut and a volume in a specific view. Without fixing that, it is hard to figure out when the correct voxels get filled and whether the computed area/setup is correct for the new display area.

Basically you have a transform view => world => integer image coordinates, but what you want is view => view voxel coordinates => integer image coordinates so that you can iterate over view voxel coordinates where the z coordinate is orthogonal to the view plane, and filling the x,y integer voxel coordinates for a specific integer z coordinate fills exactly 1 oblique plane without doubling up on any two voxels. That is a difficult setup to create, although I think I have a handle on it now. That coordinate iterator/space then needs to be used consistently.

There is also one pair of optimizations that is definitely worth considering, which is to use a view voxel bounding box and view voxel range functions that return a list of pairs of entry/exit points into the area so that the inclusion test is just done with entry/exit pairs, and not by testing every instance in the area. The fill function is then a simple integer index range fill in view voxel coordinates, with the oblique fill just being a particular instance of the volume fill. That looks something like:

for(const z in viewVoxelBounds.getZBounds()) {
  const yBound = viewVoxelBounds.getYBoundForZ(z); 
  for(const y in yBound) {
     for(const x in viewVoxelBounds.getXBounds(y,z)) {
        fillFunction(x,y,z)
     }
  }
}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/core/src/utilities/iterateOverPlane.ts`:
- Around line 77-106: Reject non-positive subPixelResolution in iterateOverPlane
before computing stepU, stepV, and stepW. Add a fail-fast check near the option
destructuring in iterateOverPlane so callers cannot pass 0 or negative values,
since those values break the sampling loop behavior. Use the existing
iterateOverPlane function and its subPixelResolution option as the place to
validate and throw an error before any spacing-derived steps are calculated.

In
`@packages/tools/src/utilities/math/polyline/getSubPixelSpacingAndXYDirections.ts`:
- Around line 26-30: The geometry helper in getSubPixelSpacingAndXYDirections
should not fall through and return nothing when viewport.getImageData() is
unavailable, since that breaks the promised { spacing, xDir, yDir } contract.
Update the function to either throw an explicit error at this guard or change
its return type to optional and propagate that contract through every caller
that destructures the result. Make the fix at the
getSubPixelSpacingAndXYDirections boundary so callers never receive an
unexpected undefined.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 48f475f0-0289-47dc-ab26-f6cc4214e082

📥 Commits

Reviewing files that changed from the base of the PR and between e93f3d0 and bdba761.

📒 Files selected for processing (9)

• packages/core/src/utilities/getInPlaneSpacingAndXYDirections.ts
• packages/core/src/utilities/index.ts
• packages/core/src/utilities/iterateOverPlane.ts
• packages/core/test/utilities/getInPlaneSpacingAndXYDirections.jest.js
• packages/core/test/utilities/iterateOverPlane.jest.js
• packages/docs/docs/concepts/cornerstone-tools/segmentation/planar-fill-iteration.md
• packages/docs/docs/migration-guides/5x/1-migration-notes.md
• packages/docs/sidebars.js
• packages/tools/src/utilities/math/polyline/getSubPixelSpacingAndXYDirections.ts

✅ Files skipped from review due to trivial changes (3)

• packages/docs/sidebars.js
• packages/core/test/utilities/getInPlaneSpacingAndXYDirections.jest.js
• packages/docs/docs/concepts/cornerstone-tools/segmentation/planar-fill-iteration.md

[coderabbitai]

Actionable comments posted: 4

🧹 Nitpick comments (1)

packages/core/src/utilities/obliqueIntegerIterator.ts (1)

972-1051: 🚀 Performance & Scalability | 🔵 Trivial

Verify call frequency of createObliqueIntegerBasis for interactive brush strokes.

The drift-bounded search in choosePrimitiveIntegerNormal scales the denominator search (needed = ceil(1/sinThreshold)+1) with volume extent — for large volumes this can run into the hundreds/low-thousands of iterations. If createObliqueIntegerBasis is rebuilt on every mouse-move while dragging a brush (rather than once per interaction/plane-normal change), this repeated search could add per-frame latency in oblique views.

Worth confirming the call site caches the basis while viewPlaneNormal/viewUp/volume geometry are unchanged during a single brush stroke.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/core/src/utilities/obliqueIntegerIterator.ts` around lines 972 -
1051, `createObliqueIntegerBasis` is doing an expensive normal search via
`choosePrimitiveIntegerNormal`, so verify it is not rebuilt on every mouse-move
during a brush drag. Update the interactive brush call site to cache and reuse
the `ObliqueIntegerBasis` for the duration of a stroke, only recomputing when
`viewPlaneNormal`, `viewUp`, `viewRight`, or volume geometry changes. Use the
`createObliqueIntegerBasis` entry point and the brush interaction handler to
locate the caching logic.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/core/src/utilities/obliqueIntegerIterator.ts`:
- Around line 477-486: `applyMatrix3` is multiplying the matrix as if the sliced
row vectors were columns, so it computes the transpose result instead of the
intended row-major transform. Update the logic in `applyMatrix3` to use each row
from `m` as a row dot product with `v`, so `orientBasisToViewUp` evaluates
`A`/`B` in the correct world-space direction for rotated or oblique volumes.

In `@packages/docs/docs/behaviour/obliqueVoxels.md`:
- Around line 50-52: The inline code span in the obliqueVoxels documentation has
a trailing space after w =, which triggers the markdown lint warning. Update the
affected sentence in the documentation so the inline code for the w assignment
is tightly written without any extra whitespace, keeping the wording and the
surrounding explanation unchanged.
- Around line 34-39: Add language tags to the fenced example blocks in
obliqueVoxels.md so markdownlint stops failing; update each affected fence near
the oblique voxel examples to declare a language such as text, including the
blocks around the equations and any other fenced snippets noted in the review.
Use the existing documentation examples in the same section to locate the fences
and ensure every triple-backtick block has a valid language tag.

In
`@packages/tools/src/tools/segmentation/strategies/utils/obliqueIntegerFill.ts`:
- Around line 89-103: The spacing math in worldLengthPerLatticeStep is incorrect
because the three lattice-axis contributions are being summed first and then
multiplied by spacing per world component, which distorts the result for rotated
volumes with anisotropic spacing. Update worldLengthPerLatticeStep so each term
in the world vector is scaled by its own axis spacing inside the iVec/jVec/kVec
contributions, keeping the logic in obliqueIntegerFill aligned with the intended
uScale/vScale calculation and the brush sizing behavior.

---

Nitpick comments:
In `@packages/core/src/utilities/obliqueIntegerIterator.ts`:
- Around line 972-1051: `createObliqueIntegerBasis` is doing an expensive normal
search via `choosePrimitiveIntegerNormal`, so verify it is not rebuilt on every
mouse-move during a brush drag. Update the interactive brush call site to cache
and reuse the `ObliqueIntegerBasis` for the duration of a stroke, only
recomputing when `viewPlaneNormal`, `viewUp`, `viewRight`, or volume geometry
changes. Use the `createObliqueIntegerBasis` entry point and the brush
interaction handler to locate the caching logic.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 13173eb3-4792-4fda-b2bf-1146fdcd38a8

📥 Commits

Reviewing files that changed from the base of the PR and between bdba761 and fe4c0d9.

📒 Files selected for processing (14)

• packages/core/src/utilities/index.ts
• packages/core/src/utilities/obliqueIntegerIterator.ts
• packages/core/test/utilities/obliqueIntegerIterator.jest.js
• packages/docs/docs/behaviour/index.md
• packages/docs/docs/behaviour/obliqueVoxels.md
• packages/docs/docs/migration-guides/5x/3-oblique-integer-voxels.md
• packages/docs/sidebars.js
• packages/tools/src/tools/segmentation/strategies/BrushStrategy.ts
• packages/tools/src/tools/segmentation/strategies/__tests__/obliqueIntegerFill.spec.ts
• packages/tools/src/tools/segmentation/strategies/compositions/regionFill.ts
• packages/tools/src/tools/segmentation/strategies/fillCircle.ts
• packages/tools/src/tools/segmentation/strategies/fillRectangle.ts
• packages/tools/src/tools/segmentation/strategies/fillSphere.ts
• packages/tools/src/tools/segmentation/strategies/utils/obliqueIntegerFill.ts

✅ Files skipped from review due to trivial changes (1)

• packages/docs/docs/migration-guides/5x/3-oblique-integer-voxels.md

🚧 Files skipped from review as they are similar to previous changes (2)

• packages/tools/src/tools/segmentation/strategies/fillRectangle.ts
• packages/tools/src/tools/segmentation/strategies/fillSphere.ts

[coderabbitai]

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check whether the oblique iterator test suite exercises orientBasisToViewUp / applyMatrix3
# with a non-identity direction matrix (rotated volume).
fd 'obliqueIntegerIterator.jest.js' | xargs -I{} rg -n -C3 'orientBasisToViewUp|applyMatrix3|direction' {}

Repository: cornerstonejs/cornerstone3D

Length of output: 3620

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the relevant helper implementations and matrix construction.
FILE='packages/core/src/utilities/obliqueIntegerIterator.ts'

echo '--- applyMatrix3 and nearby helpers ---'
sed -n '450,560p' "$FILE"

echo
echo '--- voxelToWorld matrix construction and indexNormalCovectorFromWorld usage ---'
sed -n '1000,1055p' "$FILE"

echo
echo '--- locate mat3 helpers/imports ---'
rg -n 'from "@?gl-matrix"|mat3|applyMatrix3|indexNormalCovectorFromWorld|orientBasisToViewUp|voxelToWorld' "$FILE"

Repository: cornerstonejs/cornerstone3D

Length of output: 5884

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

FILE='packages/core/src/utilities/obliqueIntegerIterator.ts'

echo '--- indexNormalCovectorFromWorld ---'
sed -n '180,250p' "$FILE"

echo
echo '--- createObliqueIntegerBasis tests around direction handling ---'
sed -n '340,560p' "$FILE"

echo
echo '--- rotation probe for applyMatrix3 vs row-major M·v ---'
python3 - <<'PY'
import math

# Row-major 3x3 matrix with columns = direction axes * spacing
direction = [
    0, -1, 0,
    1,  0, 0,
    0,  0, 1,
]
spacing = [1, 1, 1]

iVec = direction[0:3]
jVec = direction[3:6]
kVec = direction[6:9]
M = [
    iVec[0]*spacing[0], jVec[0]*spacing[1], kVec[0]*spacing[2],
    iVec[1]*spacing[0], jVec[1]*spacing[1], kVec[1]*spacing[2],
    iVec[2]*spacing[0], jVec[2]*spacing[1], kVec[2]*spacing[2],
]

v = [1, 0, 0]

# Intended row-major M·v
mv = [
    M[0]*v[0] + M[1]*v[1] + M[2]*v[2],
    M[3]*v[0] + M[4]*v[1] + M[5]*v[2],
    M[6]*v[0] + M[7]*v[1] + M[8]*v[2],
]

# Current implementation: row slices combined by component => M^T·v
mtv = [
    M[0]*v[0] + M[3]*v[1] + M[6]*v[2],
    M[1]*v[0] + M[4]*v[1] + M[7]*v[2],
    M[2]*v[0] + M[5]*v[1] + M[8]*v[2],
]

print("M =", M)
print("M·v =", mv)
print("M^T·v =", mtv)
PY

Repository: cornerstonejs/cornerstone3D

Length of output: 9669

---

applyMatrix3 uses the matrix rows as columns. m.slice(0,3)/(3,6)/(6,9) are the row vectors of the row-major voxel-to-world matrix, but this formula computes Mᵀ·v instead of M·v. That makes orientBasisToViewUp score A/B in the wrong world-space direction for rotated/oblique volumes; switch to row dot products here.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/core/src/utilities/obliqueIntegerIterator.ts` around lines 477 -
486, `applyMatrix3` is multiplying the matrix as if the sliced row vectors were
columns, so it computes the transpose result instead of the intended row-major
transform. Update the logic in `applyMatrix3` to use each row from `m` as a row
dot product with `v`, so `orientBasisToViewUp` evaluates `A`/`B` in the correct
world-space direction for rotated or oblique volumes.

[sen-trenser]

Hi @wayfarer3130 ,
Some playwright tests are failing after the Oblique offset calculations changes.
Please let us know if you are updating the playwright tests or the calculation logic to fix this.

Thanks!

[wayfarer3130]

Temporary tracking branch

I've pushed an updated copy of this branch to my own fork rather than to this PR's head:

wayfarer3130/cornerstone3D → fix-improved-brush-shape-for-rotated-images

This is a temporary tracking branch only — a convenience copy so the merged state can be built and tested. It is not a replacement for this PR, and nothing on arul-trenser/cornerstone3D was modified; this PR's head branch is untouched.

What that branch contains on top of this PR's head:

1. Merge of origin/main (at 5.8.2, 7034957b10). One conflict, in packages/docs/docs/migration-guides/5x/1-migration-notes.md — both sides appended a new section to the end of the guide, so it was a purely additive text collision. Both sections are kept, upstream's touch-action: none note first and this branch's in-plane voxel iteration note last, matching history order.
2. One cleanup commit removing two imports left unused by the oblique fill rewrite: getSphereBoundsInfoFromViewport in fillSphere.ts (bounds now come from the sphere radius in IJK) and the vec3 runtime import in obliqueIntegerIterator.ts.

Verification on the merged state: prettier clean; 183 unit tests passing (packages/core/test/utilities plus the tools strategy and spatial specs, including the new obliqueIntegerIterator, iterateOverPlane, getInPlaneSpacingAndXYDirections, obliqueIntegerFill and fillRectangle suites); packages/core typechecks clean. The packages/tools typecheck errors are pre-existing stale-dist resolution artifacts (they include untouched symbols such as growCutLog and replaceCurrentMemo) and clear after rebuilding core.

Note that I deliberately did not revert the prettier reformatting in packages/tools/src/utilities/spatial/. Those files as they stand on main fail this repo's own .prettierrc (prettier 3.6.2, trailingComma: "es5"), while the versions on this branch pass — so the reformatting here is correct and reverting it would introduce lint failures.

Feel free to pull from that branch or ignore it; it carries no changes intended for review beyond the merge and the two-line import cleanup.

[sen-trenser]

Temporary tracking branch

I've pushed an updated copy of this branch to my own fork rather than to this PR's head:

wayfarer3130/cornerstone3D → fix-improved-brush-shape-for-rotated-images
...
Feel free to pull from that branch or ignore it; it carries no changes intended for review beyond the merge and the two-line import cleanup.

Thank you for the updates and for merging the latest changes from the head.

The following Playwright tests are currently failing in this MR:

1. tests/labelmapsegmentationtools.spec.ts:508:7 › Basic manual labelmap Segmentation tools › should render and allow usage of sephere scissor
2. tests/genericViewport/genericLabelmapOverlapPlayground.spec.ts:138:7 › Labelmap Overlap Playground - Next › should render overlapping labelmaps on stack and orthographic views
3. tests/genericViewport/genericLabelmapSegmentationTools.spec.ts:275:7 › Labelmap Segmentation Tools - Next (GPU) › should paint with sphere brush (next GPU)
4. tests/genericViewport/genericLabelmapSegmentationTools.spec.ts:343:7 › Labelmap Segmentation Tools - Next (CPU) › should paint with sphere brush (next CPU)
5. tests/genericViewport/genericLabelmapSliceRenderingTools.spec.ts:143:7 › Labelmap Slice Rendering Tools - Next › should paint with sphere brush using useSliceRendering
6. tests/genericViewport/genericStackLabelmapSegmentation.spec.ts:76:7 › Stack Labelmap Segmentation - Next (GPU) › should paint a brush stroke (next GPU)

Could you also let us know whether you plan to address them or would prefer us to work on fixing these tests?

Additionally, could you please confirm whether any further actions are required to proceed with merging this MR, and indicate who will be responsible for each outstanding item?

We would also appreciate your confirmation as to whether the changes in this MR will be ready for merge after the failing tests have been resolved.

Thanks!

[sen-trenser]

@wayfarer3130 Thank you for updating the PR.
Could you please let us know if there is any next steps for this PR to unblock it from merging?

Thanks!

[wayfarer3130]

How to test that a fill writes exactly one plane

The branch now fills with Rule F, and the labelmapSegmentationTools example has the controls that show the result. This comment gives the procedure.

What Rule F is

Rule M gives a half width of (T + T_v) / 2, which is T_v for a fill of one voxel of depth. That slab holds two layers of voxels, and the user drew one layer. Rule F gives max(F, T_v) / 2 instead, where F is the depth that the fill covers.

The depth of T_v is exact, and it is not a compromise. T_v is the L1 length of the index-space normal, so a slab of thickness T_v is a standard digital plane. Such a slab holds every voxel that the plane passes through, and it shares no voxel with the fill one T_v away. Consecutive fills therefore tile the volume, with no hole and no overlap.

The procedure

1. Run the labelmapSegmentationTools example.
2. Turn on "Slice step: voxel width (L1)". A viewport then steps by T_v, which is the thickness of one digital plane.
3. Press "Reset Cameras", and set "Axial Oblique Angle" to 45 degrees.
4. Paint one circle in the centre of the axial viewport.
5. Step one slice forward, and then two slices backward.

The fill fills the painted slice completely. The slice before and the slice after show nothing.

Turn "Slice step: voxel width (L1)" off and repeat the procedure to see the historic behaviour: each neighbouring slice then shows a part of the fill.

The numbers

Turn on "Debug logs: slice step and brush fill" for the measurements. Both diagnostics are off by default, and both compute nothing when the level is above debug.

cs3d.tools.tools.segmentation.brushVoxelSlab reports one line for each fill. The line has this form:

brush fill painted=<count> T_v=<value> halfWidth=<value> halfWidth/T_v=<value>
  depthSpanInVoxels=<value> distinctLayers=<count> ONE PLANE

• depthSpanInVoxels below 1 means the fill wrote one digital plane. This is the value to check, and the line ends with ONE PLANE or MORE THAN ONE PLANE for the same test.
• halfWidth/T_v is 0.5 for a fill of one voxel of depth, which is Rule F.

cs3d.core.utilities.getTargetVolumeAndSpacingInNormalDir reports one line for each change of the step, for an oblique viewport only. The line has this form:

slice step [<viewportId>] measure=<l1|l2> used=<value> l1=<value> l2=<value>
  l1/l2=<value> fromSlabThickness=<true|false>

l1/l2 is the factor by which the historic step is too small. The factor is 1.00 for an acquisition orientation, it is 1.41 at 45 degrees about one axis, and it reaches 1.73 for a double oblique view. Issue #2912 gives the measured values.

Two limits of the current branch

The slice step. The fill writes one digital plane, and the planes are T_v apart. A viewport steps by getSpacingInNormalDirection, which is the L2 measure, and L2 is shorter than L1 for an oblique normal. Two adjacent slices therefore fall inside one digital plane without the flag. The flag rendering.sliceStepMeasure is experimental, and the flag takes the value 'l2' by default, so this branch changes no default behaviour. Issue #2912 holds the measurements and the proposed rule.

A thick slab. getSlabThickness returns a half thickness, and fillCircle passes that value as the full depth, so a fill in a thick-slab view writes half of the depth that the viewport shows. This is not new in this branch: Rule M had the same error, with a half width of (F_half + T_v) / 2. The default path is unaffected, because an orthographic viewport returns 0.05 mm and max(0.05, T_v) resolves to T_v.

A thick slab has a second problem. max(F, T_v) / 2 tiles only when F is a multiple of T_v. With F of 6 mm over voxels of 1 mm, a fill at layer 12 writes the layers 10 to 14, and a fill at layer 18 writes the layers 16 to 20. No fill writes layer 15.

I propose to correct both thick-slab problems in a separate pull request, because a brush in a thick-slab view is a different case from the oblique defect that this pull request fixes. Tell me if you prefer both in this one.

🤖 Generated with Claude Code