Fix DirtyJson dropping array elements after an unquoted value

[Osamaali313]

Problem

DirtyJson silently drops every array element after the first unquoted (bareword) value:

DirtyJson.parse_string("[foo, bar, baz]")  # -> ['foo']   (expected ['foo', 'bar', 'baz'])
DirtyJson.parse_string('[foo, "bar"]')     # -> ['foo']   (expected ['foo', 'bar'])

Arrays whose first element is quoted work by luck ('["foo", bar]' -> ['foo', 'bar']), which hides the bug in most tests.

Root cause

_parse_unquoted_string stops its scan on the delimiter that ends the bareword (,, ], }, :), then runs an unconditional self._advance() that consumes that delimiter:

while self.current_char is not None and self.current_char not in [":", ",", "}", "]"]:
    result += self.current_char
    self._advance()
self._advance()        # <-- eats the delimiter
return result.strip()

Back in _parse_array_content, the if self.current_char == "," check then never sees the comma (it was already consumed), so control falls into elif self.current_char != "]", which pops the stack and ends the array after the first bareword. Objects only survive because _parse_object_content has a fallthrough that re-enters.

Fix

Drop the extra advance so the cursor is left on the delimiter, letting the array/object loops consume it as they already do for quoted values:

    result += self.current_char
    self._advance()
-self._advance()
 return result.strip()

Impact

DirtyJson is the repair layer for malformed model tool-call / response JSON (helpers/extract_tools.py, agent.py, mcp_handler.py, memory plugins). Models that emit bareword array elements — common outside strict OpenAI-style output — silently lost their remaining tool arguments. This is data loss, not a crash, so it went unnoticed.

Tests

Added a parametrized test_unquoted_array_elements_are_all_kept covering leading/trailing/mixed bareword elements and the single-element case.

$ pytest tests/test_dirty_json.py -q
11 passed

All 7 pre-existing dirty_json tests still pass; object, number, and quoted-array parsing are unchanged.

[Copilot]

Pull request overview

Fixes a DirtyJson parsing bug where arrays were truncated after the first bareword (unquoted) element because _parse_unquoted_string() consumed the delimiter (e.g., , or ]) that the array loop relies on.

Changes:

• Stop _parse_unquoted_string() from advancing past the terminating delimiter so container parsers (_parse_array_content, _parse_object_content) can handle delimiters consistently.
• Add a parametrized regression test covering unquoted array elements (leading, trailing, mixed quoted/unquoted, and single-element).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
helpers/dirty_json.py	Removes the extra cursor advance so unquoted values do not consume ,/]/} delimiters, preventing array truncation.
tests/test_dirty_json.py	Adds regression coverage to ensure all unquoted array elements are preserved.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.