Skip to content

Fix weighted/macro AUC aggregation in rank metrics#55

Draft
Copilot wants to merge 5 commits into
mainfrom
copilot/check-auc-metric
Draft

Fix weighted/macro AUC aggregation in rank metrics#55
Copilot wants to merge 5 commits into
mainfrom
copilot/check-auc-metric

Conversation

Copy link
Copy Markdown

Copilot AI commented May 26, 2026 •
edited
Loading

  • Add test: 3 positives (0.9,0.8,0.7) and 3 negatives (0.3,0.2,0.1) → AUC = 1.0 (perfect separation, 3+3 internet example)
  • Add test: 3 positives (0.1,0.2,0.3) and 3 negatives (0.7,0.8,0.9) → AUC = 0.0 (worst case, 3+3 internet example)
  • Add test: 2 positives (0.8,0.2) and 2 negatives (0.9,0.1) where exactly half pairs are correctly ranked → AUC = 0.5 (partial/non-tie internet example)
  • All three tests verified analytically via trapezoidal ROC integration

Copilot AI linked an issue May 26, 2026 that may be closed by this pull request
Check AuC #8
Open
Copilot AI changed the title [WIP] Check AuC Metric implementation Fix weighted/macro AUC aggregation in rank metrics May 26, 2026
Copilot AI requested a review from dfuchss May 26, 2026 17:55
@dfuchss
test: cover direct AUC metric behavior
da0c849 
Agent-Logs-Url: https://github.com/ardoco/metrics/sessions/c8f62fd6-2c10-405c-b7cf-66dc81741afb

Co-authored-by: dfuchss <22198798+dfuchss@users.noreply.github.com>
@dfuchss
Add AUC tests for internet examples (3+3 perfect, 3+3 worst, half-con…
50b1c9d 
…cordant pairs)

Agent-Logs-Url: https://github.com/ardoco/metrics/sessions/aa6d93ad-149d-4a3d-a11c-ff38208946a5

Co-authored-by: dfuchss <22198798+dfuchss@users.noreply.github.com>
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented May 26, 2026

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dfuchss dfuchss Awaiting requested review from dfuchss

Assignees

@dfuchss dfuchss

Copilot code review Copilot

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Check AuC

2 participants

@dfuchss