Row anomaly detection: a second detector, and the end of `segment_by` - #1489

Open

vb-dbrks wants to merge 77 commits into

mainfrom

feat/anomaly-group-conditioning

Commits on Aug 24, 2026

Route feature metadata persistence through to_json()
vb-dbrks
committed
602d82f
Correct anomaly detection docs that misdescribe behaviour
vb-dbrks
committed
faea0c6

Commits on Aug 25, 2026

Name the segment-training thresholds in one module
vb-dbrks
committed
c53c669
Validate baseline columns before they can corrupt a key
vb-dbrks
committed
6bc0e1b
Compute one baseline key, in Python and in Spark, and pin their agreement
vb-dbrks
committed
2cdeaa4
Judge each metric against its own group's baseline, via baseline_by
vb-dbrks
committed
07cae7e
Make the anomaly benchmarks reach the published report
vb-dbrks
committed
839f637
Test offline that the relative feature separates a contextual anomaly
vb-dbrks
committed
1d4abc7
Verify baseline conditioning through Spark, MLflow and Unity Catalog
vb-dbrks
committed
1548c64
Measure whether conditioning actually detects better, and publish it
vb-dbrks
committed
4fd087e
Document baseline conditioning, and record it as a breaking change
vb-dbrks
committed
d38bf95
Record the first run of the conditioning experiment
vb-dbrks
committed
d480802
Add the plain-tabular regime to the conditioning harness
vb-dbrks
committed
9993c38
Choose a baseline grouping on the baseline cost model, not the segmented one
vb-dbrks
committed
fee682f
Mark row anomaly detection as beta
vb-dbrks
committed
7d13390
BREAKING: include baseline_by in the model configuration hash
vb-dbrks
committed
77894a3
Stop fitting a scaler that cannot affect the model
vb-dbrks
committed
df36282
Make baseline_by a queryable registry column
vb-dbrks
committed
b0658bc
Match the shipped forest configuration in the harness
vb-dbrks
committed
175533c
Record why a second detector is not added to the ensemble
vb-dbrks
committed
5ee0219
Document the heuristic path and the compatibility debt
vb-dbrks
committed
bbe9d9e
Publish the comparison against the previous release
vb-dbrks
committed
57bcebb
Correct the real-data finding, and publish the tabular regime
vb-dbrks
committed
774e5a2
Write the quality page for users, not for the team
vb-dbrks
committed
7375f9b
Refresh the quality benchmarks in the nightly
vb-dbrks
committed
8df14ab
Redact the features derived from a redacted column
vb-dbrks
committed
ccf7de8

Commits on Aug 26, 2026

BREAKING: remove the legacy segment_by anomaly path in favour of baseline_by grouping

vb-dbrks
and
Isaac
committed
f510b89
Add a reverse map from engineered feature name to source column

vb-dbrks
and
Isaac
committed
7667ffe
Close the redaction hole: drop every feature derived from a redacted column

vb-dbrks
and
Isaac
committed
9be52d4
Render contributions for a reader and reframe the prompt around baselines

vb-dbrks
and
Isaac
committed
65c3d86
Pin the group-conditioning invariants with tests

vb-dbrks
and
Isaac
committed
e6d52a7
Performance: one baseline join for all metrics; drop count() emptiness scans

vb-dbrks
and
Isaac
committed
0568704
Add the SMD estimator bake-off, and move the train/test loader to datasets

vb-dbrks
and
Isaac
committed
ca5a6c2
Step 0: settle the time-series estimator by measurement, and drop Ledoit-Wolf

vb-dbrks
and
Isaac
committed
65cc55b
Add the Mahalanobis detector, with non-negative leave-one-out attribution

vb-dbrks
and
Isaac
committed
56a423d

Commits on Aug 27, 2026

Pin the IsolationForest path against change, before wiring anything to it

vb-dbrks
and
Isaac
committed
45e0adc
Dispatch attribution by estimator, so contributions work without SHAP

vb-dbrks
and
Isaac
committed
f01c43e
Select the detector with a `profile`, defaulted so nothing changes

vb-dbrks
and
Isaac
committed
fe946a1
Decline automatic profile detection, and drop the "auto" name that implied it

vb-dbrks
and
Isaac
committed
88218e0
Verify the timeseries profile against a real workspace, settling two unknowns

vb-dbrks
and
Isaac
committed
3535f18
Fix the docs build, which commit 07cae7e1 in this PR broke

vb-dbrks
and
Isaac
committed
f2cf126
Document the profile, and rename the contract from "SHAP" to "feature contributions"

vb-dbrks
and
Isaac
committed
3758eed
Add two deck slides: comparing like with like, and correlation breaking

vb-dbrks
and
Isaac
committed
9778ee1
Repair the row anomaly demo, which this PR broke, and verify it by running it

vb-dbrks
and
Isaac
committed
966e833

Commits on Aug 28, 2026

Correct a false claim: DQX does model calendar seasonality

vb-dbrks
and
Isaac
committed
e4fed0a
Withdraw a false safety claim about trend, and measure why neither fix works

vb-dbrks
and
Isaac
committed
7d438de
Measure the fix for trend that does work: persist a fitted trend and subtract it

vb-dbrks
and
Isaac
committed
d286d08
Move the limitations out of a warning block and into one FAQ entry

vb-dbrks
and
Isaac
committed
16e94ad
Tell the LLM what a contribution measures, so explanations stop misdescribing correlation breaks

vb-dbrks
and
Isaac
committed
7e3d032
Add two demos framed on the domain problem, one per profile

vb-dbrks
and
Isaac
committed
74c73d9
Fix two broken links and a parameter that never existed in the comprehensive demo

vb-dbrks
and
Isaac
committed
5d0b995
Rewrite the anomaly demos for teaching, retire the old one, and cover both in e2e

vb-dbrks
and
Isaac
committed
dba3b05
Replace the correlation slide with an inspection line

vb-dbrks
and
Isaac
committed
dc14ef6
Correct the FAQ on single-metric series: a timestamp is context

vb-dbrks
and
Isaac
committed
3e81084
Advise rather than apply a discovered grouping on explicit columns
vb-dbrks
committed
c76eccf
Leave CHANGELOG.md to the maintainers
vb-dbrks
committed
edfb069
Publish anomaly quality in the existing benchmarks report
vb-dbrks
committed
4717277
Merge branch 'main' into feat/anomaly-group-conditioning
vb-dbrks
authored
69f4018

Commits on Sep 2, 2026

Make the detector choice reachable from a run config
vb-dbrks
committed
83b6c68
Correct the profile guidance for ordinary tabular data
vb-dbrks
committed
93b59e1
Add the temporal baseline fit
vb-dbrks
committed
a0ade1d
Wire the temporal baseline into feature engineering
vb-dbrks
committed
e1cc7d5
Expose baseline_over_time on train(), and hash it
vb-dbrks
committed
fe80305
Do not make the detector robust: measured, it costs more than it buys
vb-dbrks
committed
a53eb6c
Report extrapolation, and warn where the temporal baseline will not help
vb-dbrks
committed
d9d24d6
Document baseline_over_time, and teach the decision in the demos
vb-dbrks
committed
dfb6ae6
Publish the temporal baseline's quality in the benchmark report
vb-dbrks
committed
40e198c
Mark staleness before scoring, not after
vb-dbrks
committed
f03f76c

Commits on Sep 3, 2026

Keep the time column through the scoring select

vb-dbrks
and
Isaac
committed
c11d570
Pin the scoring select rule without needing a workspace

vb-dbrks
and
Isaac
committed
14935d0
Report detection quality honestly in the demos, and document what the threshold does

vb-dbrks
and
Isaac
committed
b418b54
Interpolate the severity tail in tail-probability space

vb-dbrks
and
Isaac
committed
0fffdce
Estimate the severity quantiles precisely enough for the tail to mean anything

vb-dbrks
and
Isaac
committed
e3515c4
Make the three comparison bases reachable from run-config YAML in the docs, not just in code

vb-dbrks
and
Isaac
committed
79e5e8b
Teach time as a basis rather than a feature, on data where it works

vb-dbrks
and
Isaac
committed
f92808e

Commits on Sep 4, 2026

Address the pre-Beta review: four blockers, four issues, four suppressions

vb-dbrks
and
Isaac
committed
b411349
Scale a discovery fixture to the floor the fit actually sees

vb-dbrks
and
Isaac
committed
af6b815