Add: constrain DSpark tool calls with XGrammar

docs/user-guide/deepseek-v4-dspark-tool-constraints.md

@@ -0,0 +1,87 @@
+# DeepSeek V4 DSpark constrained tool deployment
+
+This guide deploys structural tool-call constraints for DeepSeek V4 Flash DSpark with fused K7 decoding. The tool names and JSON schemas come from each `/v1/chat/completions` request, not from a server-side allowlist or a particular agent client. Serving compiles the request grammar on the host; the device receives a fixed-layout allowed-token mask and still runs one fused K7 decode dispatch per step.
+
+## Compatible components
+
+- Use a DeepSeek V4 Flash DSpark W8A8 checkpoint and the 16-device `--dp 4 --ep 16 --tp 4` topology described in [DSpark serving](../developer-guide/deepseek-v4-dspark.md).
+- Deploy PyPTO Serving and PyPTO-Lib as a matched pair. Their prefill and decode positional ABI now includes `grammar_mask`; fused K7 decode also includes `valid_draft_counts`. Updating only one repository will not work. The mask shape and argument order are fixed regardless of whether a request has constraints.
+- The mask also carries a row-mode header in segment 0's last padding word: `-1` selects ordinary argmax and `0` enables masking. Serving packs this header; it is not part of the provider's vocabulary bitset. Deploy the matching packer and sampler together even when their tensor shapes have not changed.
+- Install `xgrammar==0.2.7` into the Python environment that runs Serving. This is the version validated with the DeepSeek V4 structural-tag grammar and the checkpoint tokenizer. XGrammar is a host-side dependency; PyPTO-Lib does not import it. See the [XGrammar package](https://pypi.org/project/xgrammar/0.2.7/).
+- DSpark currently supports greedy sampling only. Use `temperature: 0`; non-greedy requests are rejected rather than silently changing sampling behavior.
+
+```bash
+python -m pip install 'xgrammar==0.2.7'
+python -m pip show xgrammar
+```
+
+When upgrading either repository, use a new `PYPTO_PROG_BUILD_DIR` for the changed kernels before enabling `--use-compile-cache`. The compile cache does not validate a previous executable against the new source or positional ABI.
+
+## Start the service
+
+The paths below are deployment choices, not required repository locations. Set the checkpoint and build-cache paths for your environment. The 16 listed devices must be free before starting the server.
+
+```bash
+PYPTO_PROG_BUILD_DIR=/path/to/new-compile-cache \
+python -m pypto_serving.cli \
+  --model /path/to/dsv4-flash-dspark-w8a8 \
+  --served-model-name dsv4-flash-dspark-w8a8 \
+  --backend npu --platform a2a3 \
+  --devices 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15 \
+  --dp 4 --ep 16 --tp 4 --block-size 32 \
+  --max-model-len 16384 --max-num-seqs 8 \
+  --max-num-batched-tokens 8192 \
+  --long-prefill-token-threshold 128 \
+  --speculative-config '{"method":"dspark","num_speculative_tokens":7}' \
+  --enable-prefix-caching \
+  --ring-heap 2147483648,2147483648,4294967296,8589934592 \
+  --generate-config '{"max_new_tokens":2048,"temperature":0}' \
+  --use-compile-cache --host 127.0.0.1 --port 8000
+```
+
+Start without `--use-compile-cache` for the first build if your deployment does not use a persistent compile directory. Check `/health` before sending requests:
+
+```bash
+curl --noproxy '*' http://127.0.0.1:8000/health
+```
+
+## Verify constrained and ordinary requests
+
+This request requires a `shell` tool call with a `command` string and forbids extra keys. The client still decides whether and how to execute the returned command.
+
+```bash
+curl --noproxy '*' http://127.0.0.1:8000/v1/chat/completions \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "model": "dsv4-flash-dspark-w8a8",
+    "messages": [{"role":"user","content":"Show the current directory."}],
+    "tools": [{"type":"function","function":{
+      "name":"shell",
+      "description":"Run a shell command",
+      "strict":true,
+      "parameters":{
+        "type":"object",
+        "properties":{"command":{"type":"string"}},
+        "required":["command"],
+        "additionalProperties":false
+      }
+    }}],
+    "tool_choice":"required",
+    "temperature":0,
+    "max_tokens":256
+  }'
+```
+
+Expect a completed `message.tool_calls` entry named `shell`, with `function.arguments` as a JSON string containing `command`, and `finish_reason: "tool_calls"`. A length-truncated call is not a successful invocation. For a baseline, repeat the request with `tool_choice: "auto"` and `strict` omitted: it follows the unconstrained path and may answer directly or produce a tool call. `/v1/completions` uses the same fixed kernel ABI with an all-allowed mask.
+
+`auto` enters structural constraints only when at least one declared tool has `strict: true`; `required` and a named tool choice also enter the constrained path. A schema needs `required` to require a field and `additionalProperties: false` to exclude unknown fields. Ordinary non-strict `auto` does not guarantee schema-valid parameters. `parallel_tool_calls` is passed to the grammar only for constrained requests.
+
+Ordinary batches reuse immutable device-resident all-allowed masks and default draft counts. The sampler skips bitset decoding for ordinary rows; constrained rows apply the mask before the same running-maximum reduction. These paths share the fused K7 interface and do not require a separate forward/sampling dispatch.
+
+Constrained requests reuse request-local Host bitmask storage. Draft validation and mask generation share one speculative traversal, then roll back; only committed output advances the grammar. Serving packs the valid prefix and bonus row directly into shared buffers without unpacking vocabulary bits. With asynchronous scheduling enabled, acceptance-independent metadata can prepare early, but mask planning still waits for that request's previous output to be committed. Mutable slot ownership lasts through output reclaim. Constrained masks still use the existing full-tensor runtime upload path; Host row reuse is not partial H2D transfer.
+
+Missing XGrammar, an unsupported schema, or an unsupported model path yields an HTTP 400 before streaming headers are sent. There is no fallback to unconstrained generation for a request that asked for constraints. Request completion and cancellation release request-local grammar state. Recompute preemption of a constrained request is not yet supported; under cache pressure it may wait for resources rather than preempt another constrained request.
+
+## Upgrade and rollback
+
+Roll Serving and PyPTO-Lib forward or backward together, then use a build-cache directory compiled from that pair. After a restart, check health, one constrained tool request, and one ordinary chat request before restoring traffic. Do not reuse a compile cache across the changed kernel ABI or mix the new Serving `TaskArgs` order with old Lib kernels.

docs/user-guide/online-serving.md

@@ -58,7 +58,7 @@ The server converts chat messages to a prompt with the tokenizer's `apply_chat_t
 
 ## DeepSeek V4 Function Tools
 
-Tool calling is selected by the model tokenizer; no extra launcher flag or vLLM dependency is required. Serving encodes tool definitions and parses model output. **The client executes tools**, then sends the results in a new chat request.
+Tool calling is selected by the model tokenizer; no extra launcher flag or vLLM dependency is required. Serving encodes tool definitions and parses model output. **The client executes tools**, then sends the results in a new chat request. Constrained generation for DeepSeek V4 DSpark K7 additionally requires XGrammar; see [deployment and compatibility](deepseek-v4-dspark-tool-constraints.md).
 
 Send this request to a **DeepSeek V4** server, not the Qwen server in the examples above. Set `DEEPSEEK_BASE_URL` to that server's host and port (8000 is the default serving port).
 
@@ -84,7 +84,7 @@ curl --noproxy "*" "$DEEPSEEK_BASE_URL/v1/chat/completions" \
 
 `auto` is the default when non-empty `tools` are supplied. The model can answer normally or return `message.tool_calls`, with each call containing `id`, `type: "function"`, and `function: {name, arguments}`. `arguments` is a **JSON string**, not a JSON object. `content` can be null; `reasoning`, when enabled, remains separate from both content and tools.
 
-The parser returns a model-generated function name even if that name is absent from this request's `tools`, matching vLLM's default DeepSeek V4 behavior. Serving does not provide built-in functions such as `read_file`; the client decides which calls it can execute. Check the returned name against the client's available tools before executing it.
+In ordinary unconstrained `auto` mode, the parser returns a model-generated function name even if that name is absent from this request's `tools`, matching vLLM's default DeepSeek V4 behavior. Serving does not provide built-in functions such as `read_file`; the client decides which calls it can execute. Check the returned name against the client's available tools before executing it.
 
 For a successful call, the client should validate the function name and arguments against its schema before execution. Append the returned assistant message and a tool result that references the same call ID:
 
@@ -109,8 +109,8 @@ Send that history to the same chat endpoint, including `tools` again if another
 Supported controls and limits:
 
 - `tool_choice: "none"` suppresses tool-call output. Recognized tool blocks are consumed when tools are supplied; ordinary no-tools chat keeps its existing parser behavior.
-- Multiple calls are supported. `parallel_tool_calls: false` exposes only the first call; it does not constrain sampling or execute tools serially.
-- `required`, named tool choices, and `strict: true` return HTTP 400 because constrained tool decoding is not implemented. Non-function tool types are rejected during request validation.
+- Multiple calls are supported. In unconstrained `auto`, `parallel_tool_calls: false` exposes only the first parsed call; in constrained DSpark K7 generation, the flag also enters the structural grammar. It never executes tools on the server.
+- DeepSeek V4 DSpark K7 supports structural constraints for `required`, a named tool choice, and `auto` when at least one tool has `strict: true`. These modes require XGrammar and the matching PyPTO-Lib kernel ABI. Ordinary `auto` with no strict tool remains unconstrained. Unsupported model paths or unavailable XGrammar reject constrained requests before generation. Non-function tool types are rejected during request validation.
 - Tools on a model without a registered tool parser are rejected. DSML formatting stays in the DeepSeek implementation, not the HTTP server or scheduler.
 - Tool-history argument values cannot contain the reserved `</｜DSML｜parameter>` delimiter, including inside nested JSON values. Tool-result content cannot contain `</tool_result>`. These inputs return HTTP 400 before generation rather than breaking the history encoding.
 - The parser preserves DSML parameter types without schema-based coercion or guessed JSON repairs. A length-truncated call can have incomplete arguments: do not execute it as a successful call.

mkdocs.yml

@@ -94,6 +94,7 @@ nav:
       - Inference and Serving:
           - Offline Inference: user-guide/offline-inference.md
           - Online Serving: user-guide/online-serving.md
+          - DSpark Tool Constraints: user-guide/deepseek-v4-dspark-tool-constraints.md
           - Parallelism and Scaling: user-guide/parallel.md
           - Benchmarking: user-guide/benchmarking.md
           - Profiling: user-guide/profile.md

pypto_serving/config/types.py

@@ -304,6 +304,8 @@ class PrefillBatch:
     block_ids: list[list[int]] = field(default_factory=list)
     block_ids_by_group: list[dict[str, list[int]]] = field(default_factory=list)
     cache_partitions: list[int | None] = field(default_factory=list)
+    # Opaque request-local generation constraints, interpreted only by capable runners.
+    constraint_states: dict[str, object] = field(default_factory=dict)
 
 
 @dataclass
@@ -335,6 +337,7 @@ class DecodeBatch:
     block_ids: list[list[int]] = field(default_factory=list)
     block_ids_by_group: list[dict[str, list[int]]] = field(default_factory=list)
     cache_partitions: list[int | None] = field(default_factory=list)
+    constraint_states: dict[str, object] = field(default_factory=dict)
     # Optional MTP context for models (e.g. DeepSeek V4) that decode two real
     # trailing tokens per step. ``prev_token_ids`` holds the token id at absolute
     # position ``seq_len-2`` per request (shape ``[B]``) and ``prev_hidden_states``