feat(or-bench): add Harbor-based agent benchmark with two verified tasks

[devin-ai-integration]

Summary

Scaffolds or-bench, a stable-bench-v1-style benchmark (Tempo's post) measuring whether coding agents can build working OpenRouter integrations from public docs. Built on Harbor; each task ships the four stable-bench parts: instruction.md, environment/, hidden oracle in solution/, and an independent verifier in tests/.

The key idea mirroring Tempo's on-chain verification: the verifier grades against live side-effects — it reads the agent's /app/out.json artifact and cross-checks the reported generationId against GET /api/v1/generation, asserting streamed === true, model permaslug matches the pinned slug, and native_tokens_* equal the usage the agent reported. Partial-credit sub-checks sum to a 0–1 reward written to /logs/verifier/reward.txt.

Tasks (both pinned to openai/gpt-5-nano):

• streaming-chat — SSE streaming completion with usage accounting (usage: { include: true })
• structured-outputs — strict response_format: json_schema extraction with exact-match field grading

Verified end-to-end: harbor run --path tasks/<task> --agent oracle scores 1.0 on both tasks against the live OpenRouter API.

Run with:

export OPENROUTER_API_KEY=sk-or-...
harbor run --path or-bench/tasks/streaming-chat --agent claude-code --model anthropic/claude-sonnet-4-5 \
  --ae OPENROUTER_API_KEY=$OPENROUTER_API_KEY

Note: placed in this repo because the Devin GitHub App cannot create new repos; the or-bench/ directory is self-contained and can be extracted to OpenRouterTeam/or-bench verbatim. v2 roadmap (pinned-docs sidecar, LLM-judge quality rubric, MCP on/off comparisons, more tasks) is in or-bench/README.md.

Link to Devin session: https://openrouter.devinenterprise.com/sessions/74591edf9f4e4f0db4ca6b549c4be62c

[devin-ai-integration]

Original prompt from kenny.rogers

i want to make this for openrouter, how should i do it? https://tempo.xyz/developers/blog/introducing-stable-bench-v1

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR that start with 'DevinAI' or '@devin'.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring