Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/chip-level-arch.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,13 @@ level model (see [hierarchical-level-runtime.md](hierarchical-level-runtime.md))
`CallConfig` is the exception — same type used at every level, with no
`Chip*` / unprefixed split (see [task-flow.md](task-flow.md) for details).
The unified `Worker(level=N)` factory already routes to the correct backend.
At `level=2` the default `execution_mode="program"` drives `ChipWorker.init` /
`run`. `Worker(level=2, execution_mode="kernel")` routes to the kernel entries
instead: `init(config=...)` to `ChipWorker.kernel_init`,
`kernel_prepare_callable` / `kernel_launch` to `ChipWorker.kernel_*`, and
`close()` to `ChipWorker.finalize`. Its `kernel_mode_supported` property
answers through `ChipWorker.probe_kernel_mode_supported`, before init and without
touching a device. Kernel mode has no L3+ route.
When new level-specific types are added (e.g. `ChipCallable`), each concept
should follow the same pattern: a `Chip*` concrete type for L2, a prefix-less
concrete type for L3+, and optionally a factory function that routes by level.
Expand Down
73 changes: 70 additions & 3 deletions docs/user/reference/python-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ into `**config` and validated later. The recognized keys:
| --- | ---------- | ------- |
| `platform` | all | `a2a3`, `a2a3sim`, `a5`, `a5sim` |
| `runtime` | all | `tensormap_and_ringbuffer` or `host_build_graph` |
| `execution_mode` | L2 | `"program"` (default) or `"kernel"`; see [Kernel mode](#kernel-mode-l2). `"kernel"` with `level != 2` or a truthy `enable_sdma` raises `ValueError`, as does an unknown value |
| `device_id` | L2 | the single chip this worker drives |
| `device_ids` | L3+ | one chip child process per entry |
| `num_sub_workers` | L3+ | host-side Python callables to fork |
Expand All @@ -53,8 +54,8 @@ else raises. Remote-worker and remote-memory calls require `level >= 4`.
| `unregister(handle_or_slot)` | Releases a registration |
| `add_worker(worker) -> int` | Attaches a child worker; returns its id |
| `add_remote_worker(spec: RemoteWorkerSpec) -> int` | L4; see the remote-L3 design doc |
| `init(prewarm_config=None)` | Resolves runtime binaries, opens the device, forks children. First place setup errors appear |
| `close()` | Releases the device and reaps children. Put it in a `finally` — a skipped `close()` leaves the device held |
| `init(prewarm_config=None, *, config=None)` | Resolves runtime binaries, opens the device, forks children. First place setup errors appear. `config=` is kernel mode's required context config and is refused in program mode; kernel mode refuses `prewarm_config` |
| `close()` | Releases the device and reaps children. Put it in a `finally` — a skipped `close()` leaves the device held. In `execution_mode="kernel"` it releases only the context; see [Kernel mode](#kernel-mode-l2) |

### Memory

Expand Down Expand Up @@ -97,6 +98,71 @@ callable is a **Python orchestration function** `f(orch, args, cfg)`, where
| `allocate_domain(*, name, workers, window_size, buffers=())` | Context manager returning a handle indexed by domain-local rank |
| `alloc_child_tensor(worker_id, shapes, dtype) -> Buffer` | Delegates allocation to the owning Worker; target chip memory is named by the returned handle |

### Kernel mode (L2)

`Worker(level=2, execution_mode="kernel")` is the entry for a framework that
already owns the device and its streams. The Worker borrows the device that is
current on the calling thread. Each launch enqueues bounded asynchronous work
on the caller's stream and returns without synchronizing.

| Member | Notes |
| ------ | ----- |
| `init(*, config: CallConfig)` | Kernel init on the already-current `device_id`. `config` is required and validated; it is fixed for the life of the context. A positional argument binds to `prewarm_config` and is refused, so pass `config=CallConfig(...)`. There is no prewarm, no SDMA, and no replay of registrations |
| `kernel_mode_supported -> bool` | Property. `False` when `level != 2`. Otherwise it reports whether the configured platform/runtime build supports kernel mode, whatever the `execution_mode`. The build is probed once, without touching a device, and the result is cached, so it can be asked before `init()` or before choosing a mode. Missing runtime binaries raise rather than answer `False` |
| `kernel_prepare_callable(chip_callable: ChipCallable) -> int` | Uploads the callable and returns the id the runtime minted. It blocks until device-side registration finishes, so a registration failure raises from this call. There is no deduplication: the same callable prepared twice gets two ids. It is unrelated to `register()`. Call it outside ACLGraph capture. The Worker keeps the callable alive until `close()` |
| `kernel_launch(callable_id: int, args: ChipStorageTaskArgs, *, caller_stream: int) -> None` | Enqueues one invocation on `caller_stream`, a nonzero `aclrtStream` address, and returns. Returning means enqueued; device-side errors surface when the caller synchronizes its own stream. No `RunHandle`, no wait. An id this Worker did not mint or a zero stream raises `ValueError`, and any other `args` type raises `TypeError`. A launch that overlaps a prepare, launch or close raises `RuntimeError` at once rather than waiting |
| `close()` | Releases only the resources the context owns. It never synchronizes the caller's stream, resets the device, or finalizes ACL |

In kernel mode these raise `RuntimeError`: `register` (also before `init()`),
`unregister`, `run`, `submit`, `malloc`, `free`, `copy_to`, `copy_from`,
`create_buffer`, `make_tensor_arg`, `release_buffer`, and `device_memory_info`.
`committed_device_memory` stays available. In program mode, `kernel_prepare_callable`
and `kernel_launch` raise `RuntimeError`.

The caller's side of the contract:

- `device_id` is already current on the thread that calls `init()`, and that
thread also calls `close()`.
- `args` tensors are `ChipTensor.make(addr, shapes, dtype, child_memory=True)`
views of caller-owned device memory. The caller keeps `caller_stream` alive
until the work it submitted completes.
- The caller serializes `kernel_prepare_callable`, `kernel_launch` and `close()`.
- Before `close()`, the caller stops launching, synchronizes its own stream, and
destroys every graph that replays the context. Synchronizing, resetting the
device and finalizing ACL after `close()` are also the caller's job.
- `close()` raises `TimeoutError` when an in-flight prepare or launch does not
finish within the rollback grace period, and `RuntimeError` when native
teardown fails. In both cases the context is kept, and a later `close()`
retries.
- Only the process that called `init()` may prepare, launch or close; a forked
child gets `RuntimeError`. Kernel mode has no L3+ form.
- A kernel Worker that is garbage-collected, or still open at interpreter exit,
without `close()` emits a `ResourceWarning`. Its native context is leaked on
purpose rather than finalized on an arbitrary thread.

```python
from simpler import Worker
from simpler.task_interface import CallConfig, ChipStorageTaskArgs, ChipTensor, DataType

# The framework has made device 0 current on this thread and owns `stream`,
# the device buffers `x_addr` / `y_addr`, and `chip_callable`.
worker = Worker(level=2, execution_mode="kernel", device_id=0,
platform="a2a3", runtime="tensormap_and_ringbuffer")
if not worker.kernel_mode_supported:
raise RuntimeError("this runtime build has no kernel mode")
worker.init(config=CallConfig())
try:
cid = worker.kernel_prepare_callable(chip_callable)
args = ChipStorageTaskArgs()
args.add_tensor(ChipTensor.make(x_addr, (n,), DataType.FLOAT32, child_memory=True))
args.add_tensor(ChipTensor.make(y_addr, (n,), DataType.FLOAT32, child_memory=True))
args.add_scalar(1.25)
worker.kernel_launch(cid, args, caller_stream=stream)
synchronize(stream) # the framework's own stream synchronize; device errors surface here
finally:
worker.close()
```

## Callables and task args

```python
Expand All @@ -120,7 +186,8 @@ including for callables loaded from cached bytes. The tensor count is
Public `Worker` calls use `TaskArgs` containing address-free `Tensor` views at
every level. The L2 leaf resolves those views into the internal
`ChipStorageTaskArgs` / `ChipTensor` representation; callers do not pass that
internal representation to `Worker.run()`.
internal representation to `Worker.run()`. The exception is kernel mode's
`kernel_launch`, which takes a caller-built `ChipStorageTaskArgs` directly.

For L3+ graph construction, `TaskArgs.add_dep(*handles)` adds `WAIT | RETAIN`
edges: each consumer waits for its producers and keeps their task-owned
Expand Down
2 changes: 1 addition & 1 deletion docs/zh-cn/kernel-callable-residency.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ context-local `int32_t`,失败写 `-1`。当前上限为 8192。
int32_t callable_id = -1;
int rc = simpler_kernel_mode_prepare_callable(ctx, callable, size, &callable_id);
if (rc != 0) return rc;
// 在 capture 之外完成 caller 的 warmup 并同步,检查异步准备结果
// prepare 已同步 context 自己的 AICPU stream,rc 即注册结果;以下是调用者为 capture 做的 warmup 和同步
rc = caller_warmup_and_synchronize(caller_stream); // 调用者自己的逻辑
if (rc != 0) return rc;
return simpler_kernel_mode_launch(ctx, callable_id, args, caller_stream);
Expand Down
7 changes: 7 additions & 0 deletions docs/zh-cn/kernel-mode-integration-test.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,13 @@ kernel 模式下,simpler 是一个被调用的库:它借用调用方已经
直接调用 host runtime 动态库,自己扮演调用方,不经过 PyTorch,也不经过 simpler
的 Python Worker。

经过 Worker 的是另外两个文件,入口都是 `Worker(level=2, execution_mode="kernel")`:

| 文件 | 硬件 | 覆盖 |
| ---- | ---- | ---- |
| `tests/ut/py/test_worker/test_worker_kernel_mode.py` | 不需要 | 用假的 ChipWorker 检查参数校验、`init(config=...)` 的路由、program 与 kernel 两种模式的接口互斥、prepare/launch/close 之间的串行闸门,以及 teardown 失败后 `close()` 可重试 |
| `tests/ut/py/test_worker/test_worker_kernel_mode_hw.py` | a2a3 真机 | 调用方自己设卡、建 stream,经 `init(config=...)`、`kernel_prepare_callable`、`kernel_launch(..., caller_stream=...)` 和 `close()` 驱动 kernel 模式,再同步自己的 stream 核对结果 |

被执行的算子是一个 AIV 向量加标量,`y[i] = x[i] + scalar`:

| 组成 | 源文件 | 编译产物 |
Expand Down
8 changes: 8 additions & 0 deletions python/bindings/task_interface.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -3623,6 +3623,14 @@ NB_MODULE(_task_interface, m) {
"A nonzero context generation, unique and increasing within this host "
"process. Generation zero is what the C ABI rejects as invalid."
)
.def_static(
"probe_kernel_mode_supported", &ChipWorker::probe_kernel_mode_supported, nb::arg("host_lib_path"),
nb::arg("sim_context_path"), nb::call_guard<nb::gil_scoped_release>(),
"Whether the host runtime at host_lib_path can execute kernel-mode launches. Answers from the "
"runtime build alone on a fresh device context that is destroyed before returning, so it takes no "
"device and needs no init or kernel_init. Raises RuntimeError when the runtime cannot be loaded, "
"lacks a required symbol, or yields no context."
)
.def_prop_ro("device_id", &ChipWorker::device_id)
.def_prop_ro("initialized", &ChipWorker::initialized)
.def_prop_ro("pipeline_depth", &ChipWorker::pipeline_depth)
Expand Down
43 changes: 37 additions & 6 deletions python/simpler/task_interface.py
Original file line number Diff line number Diff line change
Expand Up @@ -1541,6 +1541,34 @@ def kernel_init(
with self._lifecycle_lock:
self._init_in_progress = False

@staticmethod
def probe_kernel_mode_supported(bins: Any, log_level: int | None = None) -> bool:
"""Whether the host runtime in ``bins`` can execute kernel-mode launches.

Answers before init and independently of every ChipWorker: the native
probe loads the host runtime, asks it on a fresh device context that no
init touches, and destroys that context before returning. The runtime
build alone decides the answer, so no device is taken, no thread is
attached, and the caller needs no current device.

Args:
bins: same structural type init() takes; only host_path and
sim_context_path are read.
log_level: as for init().

Raises:
RuntimeError: the host runtime cannot be loaded, lacks a required
symbol, or yields no device context.
"""
_initialize_host_log(log_level)
sim_context_path = getattr(bins, "sim_context_path", None)
return bool(
_ChipWorker.probe_kernel_mode_supported(
str(bins.host_path),
"" if sim_context_path is None else str(sim_context_path),
)
)

@property
def kernel_mode_supported(self) -> bool:
"""Whether the bound runtime can execute kernel-mode launches.
Expand All @@ -1555,12 +1583,15 @@ def kernel_mode_supported(self) -> bool:
def kernel_prepare_callable(self, chip_callable: ChipCallable) -> int:
"""Register a callable for kernel-mode launches, outside ACLGraph capture.

Returns the ID the runtime minted for it. Registration is pure: the same
callable registered twice takes two distinct, equally valid IDs, and
there is no lookup. It takes no stream — registration enqueues on the
context's own AICPU stream, which every later launch also enqueues on,
so stream FIFO orders registration ahead of each launch. The callable
stays referenced under its ID until finalize() succeeds.
Returns the context-local id simpler minted for it. Registration is
pure: the same callable registered twice takes two distinct, equally
valid ids, and there is no lookup. It takes no stream — registration
enqueues on the context's own AICPU stream, which every later launch
also enqueues on, so stream FIFO orders registration ahead of each
launch. Registration synchronizes that context stream before committing
the callable, so a device-side registration failure raises from this
call; it synchronizes no caller stream and no device. The callable stays
referenced under its id until finalize() succeeds.
"""
callable_id = int(self._impl.kernel_prepare_callable(chip_callable))
# The registry owns the image for the life of the context: the device
Expand Down
Loading
Loading