Context Window Budget Planner

A context window is a hard budget: system prompt, chat history, RAG chunks, tool definitions, and the reserved output all have to fit inside it. Plan the split here and see instantly when your design blows the window.

Model window:

System prompt (tokens)

Tokens per turn (user + assistant)

History turns kept

RAG chunks retrieved

Tokens per RAG chunk

Tool definitions (tokens)

Reserved output (max_tokens)

How the window is used

System prompt Chat history RAG chunks Tool definitions Reserved output Overflow

What to trim first when you are over budget: 1) Chat history - it grows every turn and is the usual culprit; keep the last N turns and summarize the rest into a short rolling note. 2) RAG chunks - fewer, better-ranked chunks beat many mediocre ones; try halving the count before shrinking chunk size, and rerank so the best chunks survive. 3) Tool definitions - only send the tools the current request can actually use, and tighten verbose descriptions. 4) Reserved output - reserve what you realistically need, not the model maximum. 5) System prompt last - it is usually the highest-value tokens per word, but it is also sent on every single request, so any fat trimmed there pays off constantly. Remember: a "fits" verdict at 95% utilization still leaves no room for the next user message, so keep real headroom.