Structured log emission: JSON format option, multi-line-safe entries, ANSI stripping

[kriszyp]

Problem

The 2026-07-06 Datadog audit showed the log format itself is a major cost and signal problem:

• ~87% of "error"-status events in Datadog aren't errors. Harper writes all output to stderr, and the Docker json-file → Datadog agent path tags stderr lines as error. Of ~298M "error" events in 7 days, ~94M were [warn]-level messages and ~164M were stack-trace/object-dump fragments; only ~40M contained [error].
• Multi-line entries are billed per line. Stack traces and object dumps (e.g. the MQTT auth-failure cert dump) emit each line as a separate log event — including lines that are just } or at X (...). This destroys pattern clustering and multiplies billed events.
• ANSI color escapes ship verbatim ([32m...[39m), inflating bytes and breaking message clustering.

Proposal

1. logging.format: json config option: one JSON object per line — {time, level, thread, tags/component, audience, msg, err?} with the error stack as a single embedded string field. One event per entry by construction; the agent maps level correctly; clustering works.
2. When output is not a TTY (or when format is json), strip/disable ANSI codes.
3. Keep current plaintext as the default for local/dev; Fabric flips to json via config.
4. Route by level: only actual error/fatal to stderr (or keep everything on stdout with the level in the payload — with json format the stream no longer needs to carry severity).

Interim agent-side mitigation (multi-line aggregation rules keyed on the ISO-timestamp prefix, status remapping) is filed in host-manager; this issue is the at-source fix.

Related

harper-pro#262 (dedup/run-counts), PR #372 (status bridge), host-manager agent-pipeline issue.

🤖 Filed by Claude on behalf of Kris.