Triton · PyTorch · CUDA concepts · LLM serving

Implemented the core mechanisms of modern LLM inference from first principles: KV caching, paged KV allocation, fused RMSNorm/RoPE/SwiGLU/softmax kernels, FlashAttention-style tiled attention, PagedAttention-style decode, continuous batching, weight-only quantization, and speculative decoding.

• 3.3–3.5× RMSNorm and 5–6.5× RoPE speedups over the documented PyTorch baselines
• 2.14× PagedAttention-style decode speedup on the smaller benchmark case with ~36% KV-memory savings
• 3.19× lower average latency in the continuous-batching simulator
• Correctness checks, benchmark methodology, hardware details, and implementation limitations are documented in the repository

Educational implementations for understanding and benchmarking the mechanisms—not replacements for production kernels such as FlashAttention, vLLM, cuBLAS, or TensorRT-LLM.

Research preprint · KV-cache memory · speculative decoding

Proposes a transactional KV-cache abstraction that separates committed paged state from a packed speculative buffer. Only accepted draft tokens are committed; rejected speculative KV is discarded without rolling back committed pages.

• Formal cache-semantic equivalence argument
• Kernel-facing paged-cache overlay and commit design
• Real-GPU capacity experiments on Tesla P100 and T4 systems
• Up to 1.78× branch concurrency in the reported capacity-limited prefix-sharing setup

TechRxiv preprint, 2026.

Local-first agentic coding harness for small and open-weight models

Solo-built full-stack system that constrains coding agents with a deterministic 12-phase state machine rather than open-ended autonomy. Includes Tree-sitter symbol extraction and code-graph context, scoped agent roles, SecretGuard credential scanning, four-tier command-risk gating, structured patching, test execution, and Docker sandboxing with network isolation.

Python · FastAPI · React · SQLite · Docker · Tree-sitter · 331 backend tests

SVD-based parameter-efficient fine-tuning package

A PyTorch/PEFT package that decomposes linear weights as W = UΣVᵀ, freezes the singular-vector bases, and learns lightweight rank-r magnitude updates instead of introducing full trainable matrices.

pip install eigentune

~221M-parameter decoder-only transformer built from scratch

Educational PyTorch implementation of RMSNorm, RoPE, multi-head attention, SwiGLU, weight tying, mixed-precision training, checkpointing, and a reproducible training pipeline demonstrated on a single ~4 GB VRAM GPU.

• vLLM PR #44693 — Added regression coverage for mixed-dtype fused_add_rms_norm RMS quant fusion; includes targeted checks and a feasible Qwen3.5 end-to-end validation. Open
• CocoIndex PR #1010 — Added SplitBySeparators across the Rust executor and Python specification; incorporated maintainer review and passed the documented test suite. Merged

• LLaMA Architecture: A Deep Dive into Efficiency and Mathematics
• PyTorch Lightning: An Introduction to the Lightning-Fast Deep Learning Framework
• More articles