Skip to content

Commit caeeb17

Browse files
rhacs-botmtodor
andauthored
chore(evals): Update model evaluations 2026-08-04 (#200)
Co-authored-by: mtodor <3965286+mtodor@users.noreply.github.com>
1 parent 020eb0c commit caeeb17

1 file changed

Lines changed: 15 additions & 15 deletions

File tree

‎docs/model-evaluation.md‎

Lines changed: 15 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -39,27 +39,27 @@ A task passes when **all** its assertions pass **and** the LLM judge approves th
3939

4040
<!-- model:gpt-5-mini start -->
4141

42-
### gpt-5-mini — 2026-07-28
42+
### gpt-5-mini — 2026-08-04
4343

44-
**Overall: 10/11 tasks passed (90%)**
44+
**Overall: 11/11 tasks passed (100%)**
4545

4646
#### Task Results
4747

4848
| # | Task | Result | toolsUsed | minCalls | maxCalls | Input Tokens | Output Tokens |
4949
|---|------|--------|-----------|----------|----------|--------------|---------------|
50-
| 1 | cve-cluster-does-not-exist | Pass | Pass | Pass | Pass | 2520 | 979 |
51-
| 2 | cve-detected-workloads | Pass | Pass | Pass | Pass | 533 | 1149 |
52-
| 3 | cve-cluster-list | Pass | Pass | Pass | Pass | 1698 | 1717 |
53-
| 4 | cve-nonexistent | **Fail** | Pass | Pass | Pass | 561 | 1958 |
54-
| 5 | rhsa-not-supported | Pass | — | Pass | Pass | 786 | 2760 |
55-
| 6 | cve-multiple | Pass | Pass | Pass | Pass | 2134 | 2641 |
56-
| 7 | cve-detected-clusters | Pass | Pass | Pass | Pass | 1727 | 2283 |
57-
| 8 | cve-clusters-general | Pass | Pass | Pass | Pass | 1537 | 3689 |
58-
| 9 | cve-log4shell | Pass | Pass | Pass | Pass | 2000 | 2705 |
59-
| 10 | list-clusters | Pass | Pass | Pass | Pass | 676 | 692 |
60-
| 11 | cve-cluster-does-exist | Pass | Pass | Pass | Pass | 1531 | 1102 |
61-
62-
**Total input tokens**: 15703 | **Total output tokens**: 21675
50+
| 1 | cve-cluster-does-exist | Pass | Pass | Pass | Pass | 507 | 847 |
51+
| 2 | cve-cluster-does-not-exist | Pass | Pass | Pass | Pass | 1496 | 1355 |
52+
| 3 | cve-nonexistent | Pass | Pass | Pass | Pass | 561 | 1569 |
53+
| 4 | rhsa-not-supported | Pass | — | Pass | Pass | 618 | 1670 |
54+
| 5 | cve-detected-clusters | Pass | Pass | Pass | Pass | 1513 | 1566 |
55+
| 6 | cve-log4shell | Pass | Pass | Pass | Pass | 1976 | 2654 |
56+
| 7 | cve-clusters-general | Pass | Pass | Pass | Pass | 764 | 2361 |
57+
| 8 | cve-multiple | Pass | Pass | Pass | Pass | 1110 | 2810 |
58+
| 9 | list-clusters | Pass | Pass | Pass | Pass | 668 | 700 |
59+
| 10 | cve-cluster-list | Pass | Pass | Pass | Pass | 674 | 2040 |
60+
| 11 | cve-detected-workloads | Pass | Pass | Pass | Pass | 533 | 1712 |
61+
62+
**Total input tokens**: 10420 | **Total output tokens**: 19284
6363

6464
<!-- model:gpt-5-mini end -->
6565

0 commit comments

Comments
 (0)