Skip to content

Refresh BenchFlow docs with a local-first quickstart - #1009

Draft
ElegantLin wants to merge 1 commit into
mainfrom
agent/local-first-docs
Draft

Refresh BenchFlow docs with a local-first quickstart#1009
ElegantLin wants to merge 1 commit into
mainfrom
agent/local-first-docs

Conversation

@ElegantLin

@ElegantLin ElegantLin commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

Summary

  • replace the cloud-first onboarding path with a local Docker + subscription-auth flow
  • add a tested Start in 5 minutes guide using the real SkillsBench 3d-scan-calc task
  • add focused authentication, evaluation-running, and sandbox-selection guides
  • refresh the README, concepts, examples, agent, CLI, and Python API documentation
  • document local rubric review and fix the removed standalone skill-eval example

Why

The existing docs assumed too much prior BenchFlow knowledge and made Daytona look necessary for ordinary local use. The public onboarding path should start with one understandable local evaluation, explain what pass/fail means, and defer cloud sandboxes until remote scale or isolation is actually needed.

Validation

  • UV_CACHE_DIR=/private/tmp/benchflow-uv-cache uv run python -m pytest tests/test_cli_docs_drift.py — 8 passed
  • real local Docker run of 3d-scan-calc with Codex/gpt-5.5 — 1/1, reward 1.0, 9 tool calls, complete timing/token metadata
  • relative-link audit across all 22 public BenchFlow docs — passed
  • git diff --check — passed

Companion website PR: benchflow-ai/www.benchflow.ai#55.

@ElegantLin
ElegantLin deployed to pypi-internal-preview August 16, 2026 05:30 — with GitHub Actions Active
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant