Skip to content

[Data] OpTask._cancel never passes force=True - #65389

Open
Hyunoh-Yeo wants to merge 6 commits into
ray-project:masterfrom
Hyunoh-Yeo:in-progress-65280
Open

[Data] OpTask._cancel never passes force=True#65389
Hyunoh-Yeo wants to merge 6 commits into
ray-project:masterfrom
Hyunoh-Yeo:in-progress-65280

Conversation

@Hyunoh-Yeo

Copy link
Copy Markdown
Contributor

Description

Current behavior of OpTask._cancel never passes force=True to ray.cancel. It is because whether the task is an actor task is determined by checking if the actor id is nil, which is false for both normal tasks and actor tasks (refer to the issue).

Slicing the hex method was withdrawn through discussions with maintainers. Instead, removed the check and lets Ray Core classify the task, with a fallback to force=False when Core rejects it for an actor task.

Related issues

Closes #65280

Additional information

Two tests added in TestOpTaskCancel
(python/ray/data/tests/test_streaming_executor.py)

  • One test for a normal task
  • One test for an actor task

Signed-off-by: Hyunoh-Yeo <hyunoh.yeo@gmail.com>
Signed-off-by: Hyunoh-Yeo <hyunoh.yeo@gmail.com>
Signed-off-by: Hyunoh-Yeo <hyunoh.yeo@gmail.com>
Signed-off-by: Hyunoh-Yeo <hyunoh.yeo@gmail.com>
@Hyunoh-Yeo
Hyunoh-Yeo marked this pull request as ready for review August 11, 2026 19:47
@Hyunoh-Yeo
Hyunoh-Yeo requested a review from a team as a code owner August 11, 2026 19:47

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the task cancellation logic in physical_operator.py to use a try-except block when calling ray.cancel, falling back to force=False if a ValueError is raised. Unit tests were also added to verify this behavior. The review feedback suggests optimizing this logic to avoid redundant calls to ray.cancel when force is already False.

Comment thread python/ray/data/_internal/execution/interfaces/physical_operator.py
@ray-gardener ray-gardener Bot added data Ray Data-related issues community-contribution Contributed by the community labels Aug 12, 2026
@Hyunoh-Yeo
Hyunoh-Yeo marked this pull request as draft August 12, 2026 01:41
@Hyunoh-Yeo
Hyunoh-Yeo marked this pull request as ready for review August 12, 2026 16:00

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the task cancellation logic in physical_operator.py to use a try-except block instead of checking if the task is an actor task beforehand. If a ValueError is raised during a force-cancellation attempt, it falls back to a non-forced cancellation. Unit tests are also added to verify this behavior. The reviewer suggested optimizing the cancellation logic to avoid a redundant call to ray.cancel when force is already False and a ValueError is raised.

Comment on lines +93 to +98
try:
ray.cancel(waitable, recursive=True, force=force)
except ValueError:
# Actor tasks can't be force cancelled.
# If the task is an actor task, fallback to force=False.
ray.cancel(waitable, recursive=True, force=False)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If force is False and ray.cancel raises a ValueError for any other reason, the current implementation will catch it and redundantly call ray.cancel with force=False again.

We can simplify this logic and avoid the redundant call by only wrapping the force=True call in the try-except block.

Suggested change
try:
ray.cancel(waitable, recursive=True, force=force)
except ValueError:
# Actor tasks can't be force cancelled.
# If the task is an actor task, fallback to force=False.
ray.cancel(waitable, recursive=True, force=False)
if force:
try:
ray.cancel(waitable, recursive=True, force=True)
return
except ValueError:
# Actor tasks can't be force cancelled.
# If the task is an actor task, fallback to force=False.
pass
ray.cancel(waitable, recursive=True, force=False)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though force is passed to _cancel, conditioning the value of force with if/else and rewriting ray.cancel looks even more redundant. Sticking with the current, clearer option. Also, ValueError does not occur when force is False.

@bveeramani bveeramani self-assigned this Aug 14, 2026
Comment on lines +2198 to +2204

# First call should let the ValueError propagate,
# and the second call should fall back to force=False
assert mock_cancel.call_args_list == [
call(ref, recursive=True, force=True),
call(ref, recursive=True, force=False),
]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Rather than checking that we first call with force and then without, can we just check that we call with force=False at least once?

If we change the implementation to explicitly check if a waitable is for an actor task and decide whether to use force accordingly, then this will test will fail

@bveeramani
bveeramani enabled auto-merge (squash) August 15, 2026 21:33
@github-actions github-actions Bot added the go add ONLY when ready to merge, run all tests label Aug 15, 2026
@bveeramani
bveeramani disabled auto-merge August 15, 2026 21:34
Hyunoh-Yeo and others added 2 commits August 16, 2026 11:08
Signed-off-by: Hyunoh-Yeo <hyunoh.yeo@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-contribution Contributed by the community data Ray Data-related issues go add ONLY when ready to merge, run all tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Data] OpTask._cancel never passes force=True

2 participants