ctxfit

Fit an LLM conversation into a token budget — before you hit the context limit.

ctxfit is a tiny, zero-dependency library and CLI that packs a list of chat messages so they fit within a token budget. It keeps what matters (system prompt, most recent turns, pinned messages) and drops the rest using a strategy you choose. No API keys, no network, no build step.

---

Why

Every production LLM app eventually overflows its context window. The naive fix — "keep the last N turns" — is wrong, because a 5-turn history can be 500 or 8,000 tokens depending on content. You need to truncate by tokens, not by turns, while protecting the messages you can't lose.

ctxfit does exactly that, in ~150 lines, with no dependencies:

• 🎯 Budget-aware — pack to a hard token ceiling, optionally reserving room for the reply.
• 🧠 Strategies — middle-out (keep both ends), drop-oldest, or priority.
• 📌 Pinning — mark any message pin: true and it is never dropped.
• 🛡️ System-safe — system messages are kept by default.
• 🔌 Pluggable counting — ships a fast heuristic estimator; pass your own countTokens (e.g. tiktoken) for exact counts.
• 📊 Honest stats — every call reports what was kept, dropped, and whether it fit.

Install

npm install ctxfit
# or use the CLI without installing:
npx ctxfit --help

Requires Node.js >= 18. ESM only.

Library usage

import { pack } from 'ctxfit';

const messages = [
  { role: 'system', content: 'You are a helpful assistant.' },
  { role: 'user', content: 'A very long pasted log...' },
  { role: 'assistant', content: 'An old reply nobody needs anymore...' },
  { role: 'user', content: 'What is the capital of France?' },
];

const { messages: packed, stats } = pack(messages, {
  maxTokens: 4000,
  reserveTokens: 500,     // hold back room for the model's answer
  strategy: 'middle-out', // keep system + most recent, drop the middle
});

console.log(stats);
// {
//   strategy: 'middle-out', budget: 3500, totalTokens: 5200,
//   packedTokens: 3480, keptCount: 3, droppedCount: 1,
//   droppedIndices: [2], fits: true
// }

// `packed` is ready to send to your model.

Exact token counts

The default counter is a heuristic (~4 chars/token) so the library stays dependency-free. For exact counts, inject your own counter:

import { pack } from 'ctxfit';
import { encoding_for_model } from 'tiktoken';

const enc = encoding_for_model('gpt-4o');
const countTokens = (m) => enc.encode(String(m.content)).length + 4;

const { messages } = pack(history, { maxTokens: 8000, countTokens });

CLI usage

Pipe a JSON array (or JSONL) of messages in; get the packed conversation out.

echo '[
  {"role":"system","content":"You are concise."},
  {"role":"user","content":"alpha beta gamma delta epsilon zeta eta theta"},
  {"role":"assistant","content":"a long old reply dropped first"},
  {"role":"user","content":"What is 2+2?"}
]' | npx ctxfit --max 30 --strategy middle-out --stats

$ ctxfit --max 30 --strategy middle-out --stats < conversation.json
[
  { "role": "system", "content": "You are concise." },
  { "role": "user",   "content": "What is 2+2?" }
]
# stats (stderr): packedTokens 17, droppedIndices [1,2], fits true

CLI options

Flag	Description
-m, --max <n>	Token budget / context ceiling (required)
-r, --reserve <n>	Tokens to hold back for the reply (default 0)
-s, --strategy <s>	middle-out | drop-oldest | priority (default middle-out)
--no-system	Allow dropping system messages (kept by default)
--marker [text]	Insert a placeholder where messages were dropped (default template, or custom text)
--jsonl	Emit output as JSONL (one message per line)
--stats	Print packing stats to stderr
-h, --help	Show help

Input can be a file argument or piped on stdin, as a JSON array or JSONL.

Strategies

Strategy	Keeps	Drops first	Use when
middle-out (default)	system + earliest + most recent	the middle of the conversation	general chat history
drop-oldest	system + most recent	the oldest turns	streaming chat / sliding window
priority	highest-priority messages	lowest priority first	RAG: rank retrieved chunks

Any message can also carry "pin": true to make it un-droppable (e.g. the current user question or a critical instruction), regardless of strategy.

API

pack(messages, options) → { messages, stats }

Option	Type	Default	Meaning
maxTokens	number	—	Required. Hard token ceiling.
reserveTokens	number	0	Tokens to subtract from the budget for the reply.
strategy	string	'middle-out'	One of STRATEGIES.
keepSystem	boolean	true	Never drop role: 'system' messages.
countTokens	(msg) => number	heuristic	Per-message token counter.
omittedMarker	boolean | string | (stats) => string	false	Insert a placeholder message where the gap is when messages are dropped. true uses a default template; a string/function lets you customize it. Its tokens are reserved up front so the result still fits the budget.

stats contains: strategy, maxTokens, reserveTokens, budget, totalTokens, packedTokens, keptCount, droppedCount, droppedIndices, and fits (whether the result is within budget — false if pinned/system messages alone exceed it).

Also exported: estimateTokens(text), countMessageTokens(msg, opts), contentToText(content), and STRATEGIES.

Run the tests

npm test   # node --test, 30 tests, zero dependencies

License

MIT © 2026 Ayubjon — see LICENSE.

Support

ctxfit is free and open source. If it saved you some context-window headaches, an optional tip is always welcome (never required):

• USDT — Ethereum (ERC-20): 0xad39bdf2df0b8dd6991150fcea0a156150ed19b8 (verify on Etherscan)

Please send only on the Ethereum (ERC-20) network. Thank you! ⭐ Stars are appreciated too.