Benchmark important kernels and report results in the system card (`/kernel-bot benchmark`)

[moon-bot-app]

Motivation

We want reliable, comparable performance numbers for the kernels under hf.co/kernels-community. Rather than building a full leaderboard, keep it deliberately simple: pick a handful of important kernels, always report their benchmark results in the system card, and track whether a new version caused a speed deviation.

Prior art for the "track deviations across versions" part: https://huggingface.co/datasets/diffusers/benchmarks

Proposal

• Select a small set of important kernels under kernels-community to start with.
• Run a fixed benchmark per kernel and publish the results into the kernel's system card.
• Persist historical results (e.g. a dataset on the Hub) so version-over-version speed deviations are visible.
• Trigger: not nightly. Run after a PR is opened against an important kernel and build-upload has completed.
• Expose it as /kernel-bot benchmark, implemented as a GitHub Actions workflow with workflow_dispatch.

Notes

Discussed in Slack: @sayakpaul has a concrete idea for how to set this up; @danieldk agreed on the /kernel-bot benchmark entry point. @drbh previously worked on benchmarks (see also huggingface/kernels-benchmarks) and may have reusable pieces.

---

Requested by Sayak Paul - Slack thread - Agent trace

[Rishik00]

Is this issue open for new comers? I'd like to help in benchmarking