Skip to content

feat: add AIPerf multimodal benchmark integration#13

Merged
lzx1413 merged 3 commits into
Tele-AI:mainfrom
ActivePeter:activepeter/aiperf-benchmark
Jul 21, 2026
Merged

feat: add AIPerf multimodal benchmark integration#13
lzx1413 merged 3 commits into
Tele-AI:mainfrom
ActivePeter:activepeter/aiperf-benchmark

Conversation

@ActivePeter

Copy link
Copy Markdown
Contributor

Summary

  • expose bounded target-side phase, chunk, runtime, and environment facts
  • add TeleFuser video and LingBot stream benchmark profiles plus an SGLang baseline contract
  • clone and use AIPerf only at <TeleFuser>/benchmarks/aiperf, with no checkout-path override
  • keep warmup, aggregation, semantic mapping, artifacts, resource telemetry, GreptimeDB, and UI ownership in AIPerf
  • document the user workflow and metric protocol in English and Chinese

Validation

  • changed-area unit tests: 4 passed
  • broader unit run: 754 passed and 7 skipped; 6 existing pipeline-parallel tests require CUDA on the local runner
  • integration run: 44 passed, 1 skipped, 2 GPU tests deselected
  • pre-commit passed
  • strict MkDocs build passed
  • live 2xH100 LingBot WebRTC quick profile succeeded with 6 steady chunks, 4.3565 target compute FPS, and 16.5361 client stream FPS
  • active resource telemetry merged CPU, memory, GPU, GPU memory, Ethernet, and RDMA curves into the same GreptimeDB run with zero dropped samples

Metric boundary

LingBot compute time spans actor submission through raw-frame return. Native WebRTC encode time remains unavailable. Child-actor allocator peaks are omitted instead of reporting incomplete service-process values; AIPerf process-tree GPU-memory telemetry provides the runtime curve.

Companion AIPerf PR: ActivePeter/aiperf#1

Emit synchronized phase and chunk measurements for streaming targets, propagate batch timing and accelerator peaks through task APIs, and retain exact OpenAI video dimensions for fixed workloads.

Signed-off-by: ActivePeter <1020401660@qq.com>
Add reproducible TeleFuser batch and stream workloads, a native SGLang-Diffusion LingBot baseline, active resource-reporting launchers, and fixed service examples without vendoring AIPerf.

Signed-off-by: ActivePeter <1020401660@qq.com>
Document target-versus-harness ownership, canonical metric semantics, GreptimeDB resource history, dashboard behavior, reproducibility rules, and TeleFuser/SGLang workflows.

Signed-off-by: ActivePeter <1020401660@qq.com>
@ActivePeter
ActivePeter force-pushed the activepeter/aiperf-benchmark branch from 128d80a to 4affe5a Compare July 21, 2026 11:18
@ActivePeter

Copy link
Copy Markdown
Contributor Author

Final validation for 4affe5a completed:

  • pre-commit run --all-files: passed
  • CI-equivalent CPU unit selection: 675 passed, 2 skipped
  • MkDocs strict build, shell syntax, JSON/JSONL, and YAML validation: passed
  • fresh GitHub clone + documented scripts/setup_aiperf_repo.sh: passed; it installed teleai@e977ffbb under benchmarks/aiperf and created artifacts/
  • GreptimeDB + History API/bundled Vue UI smoke test: passed; a real TeleFuser artifact imported 64 metric points and the repeat import was skipped idempotently

The Wan I2V example now defers the GPU-dependent pipeline import, fixing the two torchvision::nms failures seen on the prior CI SHA. The latest fork workflows are awaiting maintainer approval: Lint and Tests.

@lzx1413

lzx1413 commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

LGTM

@lzx1413
lzx1413 merged commit 17a9868 into Tele-AI:main Jul 21, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants