[Feat] toolkit: add ttft-analyze tool for UCM prefix-cache TTFT estimation - #1308
Draft
student-jhz wants to merge 5 commits into
Draft
[Feat] toolkit: add ttft-analyze tool for UCM prefix-cache TTFT estimation#1308student-jhz wants to merge 5 commits into
student-jhz wants to merge 5 commits into
Conversation
合并 develop 分支最新内容到 toolkit,同时保留 ttft-analyze 工具。 冲突解决:README.md 与 registry.py 同时保留 precheck 和 ttft-analyze 工具。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
新增 ttft-analyze 工具,用于在部署前预估 KV Cache 完全命中 UCM SSD/DRAM Prefix Cache 时的 TTFT(首 Token 时间)。基于用户给定(或实测)的 Full Prefill / Full HBM Prefix Cache TTFT,结合存储读带宽、H2D 带宽与输入长度,输出预估 TTFT、相对增益/损失与瓶颈拆分,辅助部署前的 TTFT 预判。纯计算,无外部依赖,不运行真实推理。
Modifications
建模要点:
Test
新增 toolkit/tests/test_ttft_analyze.py(329 行),覆盖:
运行测试:
cd toolkit && python -m pytest tests/test_ttft_analyze.py -v