Skip to content

Classify log and status audience: operator vs user (application) #1638

Description

@kriszyp

Problem

Harper's log stream (and status surface) mixes two audiences with different responders:

  • Operator/platform — replication failures, storage errors, resource exhaustion: Harper/Fabric ops must respond.
  • User/application — errors thrown by customer components, their config mistakes, their upstream failures: the customer must respond.

Today both land in the same stream at the same levels and ship to Datadog identically. Concrete cost: on 2026-07-03 a customer app's error spam ([releaseStuckRefreshingPages] failed to release <url>...) roughly doubled fleet-wide log ingestion and stayed there — an app-audience problem billed and paged like a platform one. Conversely, user-relevant problems get buried where customers never see them.

Proposal

Add an audience dimension (operator | user) to logging and status:

  • Loggers created for user components (via the component loader / forComponent path for user apps) default to audience: user; core subsystem loggers default to operator. Explicit override available.
  • logger.status() entries (PR Add integrated logger-status system with .status() API, health checks and hierarchical view #372) carry the same field, so get_status can present an operator view and a user view.
  • Routing policy becomes possible: operator warn+ ships to Datadog/paging; user messages stay local / go to Studio's notification center / a cheap retention tier, with per-audience level config.
  • The Fabric agent config can then filter by audience tag instead of by fragile message patterns.

This is the primary noise-to-signal split for the platform: it separates "what needs us" from "what needs the customer" at emission time, where the information is known — rather than trying to reconstruct it downstream with log parsing.

Related

PR #372, structured-log issue (json format carries the field), host-manager agent-pipeline issue.

🤖 Filed by Claude on behalf of Kris.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Fields

    Priority

    P2

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions