fix(ai): parse text tool-calls, fix guardrail/concurrent prompts, surface scope in prompt - #1415
Merged
Merged
Conversation
…s tool calls as content Some models emit tool calls as TEXT inside the message content (Hermes/XML style, e.g. `<tool_call><function=NAME><parameter=KEY>VALUE...</tool_call>`) instead of native structured `tool_calls`. litellm returns that text as `content` with an empty `tool_calls`, so the agentic loop rendered the raw XML as a user-facing message and then dead-ended (the terminal branch fires when `tool_calls` is empty), never dispatching the model's intended tool call. Add `parse_text_tool_calls()` in `secator/ai/utils.py`: when `tool_calls` is empty but `content` carries one or more `<tool_call>...</tool_call>` blocks, it recovers them into litellm-shaped calls (function name from `<function=NAME>`, args from `<parameter=KEY>VALUE` segments; also accepts a JSON body) and strips the consumed blocks from the displayed content. Values are coerced from JSON when possible, else kept as text; malformed blocks are skipped, never raised. The loop calls it right after reading `tool_calls`, so recovered calls flow through the existing `_process_tool_calls` path exactly like native ones and the raw XML is no longer shown. Native tool-call and plain-content paths are unchanged. Adds unit tests covering XML/JSON blocks, multiple blocks, malformed input, and dispatch. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…concurrent prompts from cancelling each other Two independent AI-engine reliability fixes: (a) Guardrail retry guidance. When a network action is denied because only a different form of the host is in scope (an IP was used but the hostname is allowed, or vice versa), the model retried the SAME denied target on a loop. Add concise guidance to the <guardrails> prompt: never repeat a denied target; if the denied value was an IP retry the in-scope hostname (and vice versa), otherwise pick a different in-scope target. (b) Concurrent-prompt cancellation. A permission/deny prompt and a multi-choice follow-up prompt can be outstanding at the same time. `_expire_stale_pending` (run when any new pending prompt is built) expired BOTH the `follow_up` and `permission` pending docs indiscriminately, so building one prompt flipped the other to `timed_out` — orphaning it so it stopped awaiting an answer. Scope the expiry to the incoming prompt's own type: a permission prompt now only supersedes a prior pending permission, a follow_up only a prior follow_up. The same-type supersede (recovering a dead worker's stale prompt) is preserved. Updates the test that codified the old cross-type behaviour and adds tests proving the two prompt types no longer cancel each other. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Inject the run's in-scope (and out-of-scope, if any) targets into the AI system prompt so the model knows the allowed scope up front. This reduces guardrail-denied retries where the model would otherwise guess a target form that isn't permitted. - `build_scope_section()` renders a `<scope>` block listing in-scope targets (with a hint to prefer the in-scope hostname form on retry) and out-of-scope targets; returns "" when no scope is configured, so the section is omitted. - `get_system_prompt()` gains `in_scope`/`out_of_scope` params and appends the section; the AI task threads its resolved scope run-opts through. - Test asserts the scope appears in the built prompt when provided and is absent otherwise. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
|
Note Currently processing new changes in this PR. This may take a few minutes, please wait... ⚙️ Run configurationConfiguration used: Repository: freelabz/secator/.coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (8)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Consolidates three AI-engine improvements into one PR. Supersedes #1413 and #1414 (both closed in favor of this).
Parse text tool-calls. Some models emit tool calls as
<tool_call><function=…><parameter=…>XML in message content instead of the API's structured tool-call field.parse_text_tool_calls()recovers those blocks, dispatches them like real tool calls, and strips the XML from the displayed content so the transcript stays clean. (was fix(ai): parse text/Hermes-style <tool_call> blocks when a model emits tool calls as content #1413)Guardrail retry + concurrent-prompt fixes. (a) Prompt guidance so a model whose target was denied for being out of scope does not repeat the identical denied target — it retries the in-scope host/IP form or picks a different in-scope target. (b)
_expire_stale_pendingnow scopes expiry to the same prompt type, so a permission prompt no longer cancels a concurrently-pending follow_up (and vice versa). (was fix(ai): guide guardrail-denied retries to in-scope host/IP and stop concurrent prompts from cancelling each other #1414)Surface authorized scope in the prompt. Inject the run's in-scope (and out-of-scope, if present) targets into the system prompt via a
<scope>block so the model knows the allowed scope up front — reducing guardrail-denied retries. Omitted entirely when no scope is configured.Tests: AI prompt / utils / interactivity unit suites pass together (131 passed for the touched files; new
TestScopeInPromptasserts scope appears when provided and is absent otherwise). The 6 pre-existingtest_ai_guardrails.py::TestEdgeCasesredirect-parsing failures are unrelated (present onmain).🤖 Generated with Claude Code