Token Limits & Context Windows Intermediate

Every AI model has a maximum number of tokens it can process in a single request. This limit - the context window - includes both your input and the model's output combined.

Context Windows by Model

Model	Context Window	Max Output	Approx. Pages
Claude Opus 4	200,000 tokens	32,000 tokens	~300 pages input
Claude Sonnet 4	200,000 tokens	16,000 tokens	~300 pages input
GPT-4o	128,000 tokens	16,384 tokens	~190 pages input
GPT-4o-mini	128,000 tokens	16,384 tokens	~190 pages input
Gemini 1.5 Pro	1,000,000 tokens	8,192 tokens	~1,500 pages input
Gemini 2.0 Flash	1,000,000 tokens	8,192 tokens	~1,500 pages input
LLaMA 3.1 405B	128,000 tokens	4,096 tokens	~190 pages input
Mistral Large	128,000 tokens	varies	~190 pages input

How Context Is Consumed

Your context window is shared across all parts of a request:

Context Budget Breakdown

# Total context window (e.g., 200K for Claude)
┌─────────────────────────────────────────────┐
│  System Prompt        (~500 tokens)         │
├─────────────────────────────────────────────┤
│  Conversation History (~10,000 tokens)      │
│  ├─ User message 1                          │
│  ├─ Assistant response 1                    │
│  ├─ User message 2                          │
│  ├─ Assistant response 2                    │
│  └─ ... (grows with each turn)              │
├─────────────────────────────────────────────┤
│  Current User Message  (~2,000 tokens)      │
├─────────────────────────────────────────────┤
│  Tool Results / Files  (~5,000 tokens)      │
├─────────────────────────────────────────────┤
│  ═══════════════════════════════════════     │
│  Available for Response (~182,500 tokens)   │
│  (but limited by max_tokens setting)        │
└─────────────────────────────────────────────┘

Common Mistake: Many developers forget that conversation history accumulates. After 20 back-and-forth turns, you may have used 50,000+ tokens of context just on history, leaving less room for the current request and response.

What Happens When You Exceed Limits

Scenario	What Happens	How to Handle
Input exceeds context window	API returns an error (400 Bad Request)	Reduce input size, summarize history, or use a model with a larger context
Output hits max_tokens	Response is truncated mid-sentence	Increase max_tokens or ask for shorter responses
Input + max_tokens exceeds window	API returns an error or silently reduces max_tokens	Calculate available output space: context_window - input_tokens

Strategies for Managing Context

1. Conversation Summarization

Periodically summarize the conversation history to compress it into fewer tokens while retaining key information:

Python

def manage_context(messages, max_history_tokens=50000):
    # If history exceeds limit, summarize older messages
    history_tokens = count_tokens(messages)
    if history_tokens > max_history_tokens:
        # Keep system prompt and last 5 messages
        system = messages[0]
        recent = messages[-5:]
        old = messages[1:-5]

        # Summarize old messages
        summary = summarize(old)
        return [system, {"role": "user", "content": f"Previous context: {summary}"}] + recent
    return messages

2. Sliding Window

Keep only the most recent N messages, dropping older ones as the conversation grows. Simple but effective for chat applications.

3. RAG (Retrieval-Augmented Generation)

Instead of stuffing all information into context, store documents externally and retrieve only relevant chunks for each query. This lets you work with document sets far larger than any context window.

Best Practice: For production applications, combine all three strategies: use RAG for large document sets, sliding window for conversation management, and periodic summarization for important context that should not be lost.

← Counting Tokens Cost & Pricing →

Ready to Go Deeper?

Live instructor-led courses from our partners. Affiliate disclosure.

🧠

AI & ML Courses - 30% Off

Live instructor-led AI, machine learning, data science, and cloud courses for working professionals. Use code Limited30 at checkout.

Edureka

📊

DataCamp - AI & Data Science

Hands-on Python, machine learning, and AI courses with interactive exercises and real projects.

DataCamp

🎓

edX - Top AI Courses

University-level AI courses from MIT, Harvard, Stanford. Earn certificates that employers recognize.

edX