Popular repositories Loading

1. SpecInfer SpecInfer Public

   My own simple framework to implement speculative decode

   Python

2. xllm xllm Public

   Forked from xLLM-AI/xllm

   A high-performance inference engine for LLMs, optimized for diverse AI accelerators.

   C++

3. llama.cpp llama.cpp Public

   Forked from ggml-org/llama.cpp

   LLM inference in C/C++

   C++

4. MOD MOD Public

   Python

5. ssd ssd Public

   Forked from tanishqkumar/ssd

   A lightweight inference engine supporting speculative speculative decoding (SSD).

   Python

6. HiCache HiCache Public

   Forked from fenglang918/HiCache

   HiCache: Hermite Polynomial-based Feature Cache for diffusion inference

   Python