From 7e60939cf2db146fdacf7d1090ac91f750d8d4d8 Mon Sep 17 00:00:00 2001 From: Jack Arturo Date: Sat, 4 Jul 2026 07:34:44 +0200 Subject: [PATCH] docs: link AMB submission to provider PR #24 The neutral Agent Memory Benchmark submission now has a concrete PR (vectorize-io/agent-memory-benchmark#24). Point the README and the EXPERIMENT_LOG neutral-AMB status at it; wording stays "submitted, PR under review" since it is open, not merged. Co-Authored-By: Claude Opus 4.8 --- README.md | 2 +- benchmarks/EXPERIMENT_LOG.md | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 3adde6c..7b0839b 100644 --- a/README.md +++ b/README.md @@ -39,7 +39,7 @@ AutoMem **0.16.0** was run through the neutral [Agent Memory Benchmark](https:// - **Efficiency is architectural:** AutoMem feeds the answerer **~2.6–4.8k context tokens** at every scale (mean), versus 17–27k for the board leader on BEAM. - **The honest other half:** on conversational Core-3, AutoMem **trails** the AMB leader Hindsight — locomo 85.1% vs 92%, longmemeval 74.4% vs 94.6%, personamem 76.1% vs 86.6%. Pick AutoMem for large-context scaling and efficiency, not for top-of-board verbatim recall. -Outputs are committed and public, and [`AUTOMEM_REPRODUCE.md`](https://automem.ai/benchmarks) gives one command per split so you can **run it yourself**. AutoMem is **submitted to the neutral board (provider PR under review)** — not yet live on the public leaderboard. Full head-to-head numbers live at [automem.ai/benchmarks](https://automem.ai/benchmarks). +Outputs are committed and public, and [`AUTOMEM_REPRODUCE.md`](https://automem.ai/benchmarks) gives one command per split so you can **run it yourself**. AutoMem is **submitted to the neutral board ([provider PR #24](https://github.com/vectorize-io/agent-memory-benchmark/pull/24), under review)** — not yet live on the public leaderboard. Full head-to-head numbers live at [automem.ai/benchmarks](https://automem.ai/benchmarks). ## Should you use AutoMem? diff --git a/benchmarks/EXPERIMENT_LOG.md b/benchmarks/EXPERIMENT_LOG.md index 3288f08..53be49c 100644 --- a/benchmarks/EXPERIMENT_LOG.md +++ b/benchmarks/EXPERIMENT_LOG.md @@ -57,9 +57,9 @@ own harness (directional, not head-to-head) — only BEAM is the clean compariso Hindsight is the apples-to-apples Core-3 yardstick (same AMB harness). **Status — submitted, not official.** AMB results are submitted to the neutral -board; the vectorize-io provider PR is **under review**. Frame as "submitted, PR -under review" and "run it yourself" — never "live/official on the leaderboard" -until the PR merges. Outputs are committed and public; `AUTOMEM_REPRODUCE.md` +board; the vectorize-io provider PR ([#24](https://github.com/vectorize-io/agent-memory-benchmark/pull/24)) +is **under review**. Frame as "submitted, PR under review" and "run it yourself" +— never "live/official on the leaderboard" until the PR merges. Outputs are committed and public; `AUTOMEM_REPRODUCE.md` gives one command per split, and the public GHCR image (`ghcr.io/verygoodplugins/automem:amb-v1`) self-spins the full stack with no API keys. **No cross-system latency/speed claims** — AMB timings are AutoMem's own