A context window is the model's working memory for one request. Everything you send — your prompt, any files, the conversation history, the system instructions — plus everything the model writes back must fit inside that single number, measured in tokens. Most people do not need one million tokens. For everyday work — emails, short documents, coding help — 32K to 128K tokens is plenty. For long reports, legal contracts, or whole codebases, you need more. But the number on the model card is not the number you actually get. Independent testing shows models reliably use only 50% to 65% of their advertised window. Buying a bigger window does not make the model smarter. It makes it slower, more expensive, and sometimes less accurate.
The desk metaphor
The cleanest way to understand a context window is to think of it as a desk.
A language model does not carry persistent memory across conversations. Every single time you send a message, the model looks at everything that is on the desk: the system instructions that define its behavior, the full history of the chat, any files you attached, and the new message you just typed. It reads all of that, writes an answer, and then the desk is wiped clean. The next time you send a message, the model starts over — with the entire conversation history re-sent from scratch.
The context window is the size of that desk. Everything must fit. If you exceed it, the request fails — it does not silently truncate your input.
Tokens, not words: what the numbers actually mean
Context windows are measured in tokens, not words or pages. A token is roughly three-quarters of an English word, or about four characters. Code tokenizes less efficiently than prose because of brackets, indentation, and variable names.
Here is how the common window sizes translate into real work:
Context window | Approximate words | Approximate pages of text | Approximate lines of code |
|---|---|---|---|
4K tokens | 3,000 | ~10 pages | ~2,000–3,000 |
32K tokens | 24,000 | ~80 pages | ~16,000–22,000 |
128K tokens | 96,000 | ~300 pages | ~50,000–70,000 |
1M tokens | 750,000 | ~2,500 pages | ~400,000–550,000 |
Source: Morph's context window guide
A 100-page report is roughly 50,000 to 75,000 tokens, depending on density. A 30,000-line codebase is roughly 400,000 to 550,000 tokens — well within a 1M window but far beyond a 128K window. The line between "enough" and "not enough" depends entirely on what you are putting on the desk.
What counts toward the window (and what people forget)
Everything in the request counts. Not just your prompt. The parts people routinely underestimate:
System instructions. Counted on every single turn, not once per session.
Conversation history. Every prior user and assistant message you re-send. In a long chat session, this accumulates fast.
Tool definitions and results. If you use an AI agent that reads files, browses the web, or runs code, the tool schemas and the results they return can consume thousands of tokens per call.
Reasoning tokens. On models with extended thinking enabled, the model's internal reasoning counts toward the window and is billed — even when it is not returned to you.
The reply itself. If you size your input too tightly, there is no room left for the model to answer. On Claude models with thinking on by default, a request that fit on an older model can run out of room mid-response.
The practical implication: a 200K context window does not give you 200K tokens for a document. It gives you 200K tokens shared across your instructions, your history, your document, the model's reasoning, and its answer. In a long agent session, the usable document space can be half the advertised number.
The gap between advertised and effective context
This is the most important thing to understand about context windows in 2026: the number on the model card is not the number you actually get.
NVIDIA's RULER benchmark measures how much of an advertised context window a model can reliably use on tasks that require multi-hop reasoning or information aggregation across a long input. The finding is consistent across models: effective context is typically only 50% to 65% of advertised capacity.
On one retrieval test, a model scored 92.3% at 128K tokens and 29.7% at 512K. The window was real — the model accepted the input. The effective use of that input collapsed.
The "lost in the middle" problem
The reason is a well-documented phenomenon called positional bias. Models do not attend equally to all positions in their context window. They attend strongly to the beginning and the end, and weakly to the middle. Researchers call this the U-shaped attention curve.
The practical consequence: if the answer you need is buried on page 400 of a 1,000-page document, the model may miss it — even though the document fits comfortably inside the advertised window. One study found that performance can degrade by more than 30% when relevant information shifts from the start or end of the context to the middle. A more recent paper found degradation ranging from 13.9% to 85% from length alone, even when retrieval was perfect.
"Context rot"
A July 2025 study by Chroma evaluated 18 frontier models, including GPT-4.1, Claude 4, and Gemini 2.5. Every single one exhibited "context rot" — measurable performance degradation as input length increased, even on tasks the model could handle at shorter lengths.
A follow-up paper in October 2025 delivered the most counterintuitive finding: context length alone hurts performance, even when the model retrieves every relevant fact perfectly. The problem is not just retrieval failure. It is that a longer context changes how the model processes what it retrieves.
How much do you actually need?
The answer depends on your task, not on the biggest number available.
Task | Typical token need | Recommended window | Why |
|---|---|---|---|
Everyday chat, Q&A, brainstorming | 8K–32K | 32K–128K | Rarely needs more; larger windows slow responses |
Email drafting, short summaries | 4K–16K | 32K | A few thousand tokens of instruction and content |
Document Q&A (one report, <100 pages) | 50K–100K | 128K | Fits in a single request with room for output |
Legal contract review, long research brief | 100K–200K | 200K–400K | Multiple documents plus analysis |
Full codebase analysis (30K+ lines) | 400K–600K | 1M | Only if the model's effective context supports it |
Multi-document synthesis (5+ reports) | 200K–500K | 1M, with caution | Test effective retrieval before relying on it |
The "sweet spot" for most individual users is 32K to 128K tokens. That covers the overwhelming majority of everyday professional work. The 1M-token windows now shipping on frontier models are useful for genuinely long documents and large codebases, but they come with three costs that most marketing pages do not mention.
The three costs of a bigger window
1. Speed. The model re-reads the entire window on every turn. A 500K-token conversation means every reply starts by processing 500K tokens before it writes a single word. Larger windows are slower, always.
2. Memory and cost. The model stores internal notes about the whole window in a buffer called the KV cache. Larger windows consume more GPU memory and more of your budget. Filling the same 1M-token window costs $0.14 on DeepSeek V4 Flash and $10.00 on Claude Fable 5 — a 71x spread. And GPT-6 Sol reprices the entire request at a higher rate once you cross 272K input tokens, not just the excess.
3. Accuracy degradation. This is the counterintuitive one. Past a certain length — which varies by model and task — more context makes the model worse, not better. The information is there. The model just uses it less reliably. As one analysis put it, the effective context floor is roughly 50% — at 1M advertised, you get 500K to 650K of reliably usable input.
What to do about it
For everyday users: You almost certainly do not need a 1M-token window. A 128K window from Claude, GPT, or Gemini covers emails, documents up to ~300 pages, and coding help on files under ~50,000 lines. Pay attention to the model's effective context, not its advertised number.
For long documents: If you are working with a 200-page report, 128K is enough — but put the most important sections at the beginning or the end of your prompt, not in the middle. The U-shaped attention curve is real, and the middle of a long context is where information goes to be forgotten.
For high-volume work: Use the cheapest model with an effective context that fits your task. DeepSeek V4 Flash fills a 1M window for $0.14. If your task is extraction or summarization and you do not need frontier reasoning, the premium models are charging you for context you will not reliably use.
For agentic workflows: Context windows fill up fast when you are re-sending conversation history and tool results on every turn. Monitor your token usage. If you are approaching 50% of the advertised window, you are approaching the effective limit for multi-step reasoning tasks. Compress or summarize older context rather than letting it accumulate.
For anyone choosing a model: The headline context number stopped being a differentiator in 2026. Every frontier model now advertises 1M tokens or more. What separates them is effective context — how much of that window they can actually use — and cost at length. A 1M window that only works reliably at 400K is worth less than a 400K window that works reliably at 400K.
FAQ
What is a context window in plain English?
It is the model's working memory for one request. Everything you send — your message, any files, the chat history, the instructions — plus everything the model writes back must fit inside a single token budget. When the budget is full, the request fails. The model does not remember the last request unless you re-send it.
How many pages is 1 million tokens?
Roughly 2,500 pages of text, or about 750,000 English words. In code, it is roughly 400,000 to 550,000 lines, depending on formatting. In practice, models reliably use only 50% to 65% of that on tasks requiring multi-step reasoning.
Do I need a 1M-token context window?
Probably not. For emails, short documents, and everyday coding help, 32K to 128K tokens is sufficient. You need a large window for multi-hundred-page reports, full codebases, or multi-document synthesis — and even then, you should test whether the model actually uses the far end of its window reliably.
Why does my model perform worse with more context?
Because attention is not uniform. Models attend strongly to the beginning and end of a context and weakly to the middle — the "lost in the middle" problem. Longer contexts also change how the model processes retrieved information, even when retrieval itself is perfect. The result is that a model can score 92% at 128K tokens and 30% at 512K on the same retrieval task.
What is the difference between advertised and effective context?
Advertised context is how many tokens the model accepts. Effective context is how many of those tokens it still uses reliably under load. NVIDIA's RULER benchmark finds effective context is typically 50% to 65% of advertised. At 1M advertised, that means 500K to 650K of reliably usable input for complex tasks.
Does a bigger context window make a model smarter?
No. A context window determines working space, not intelligence. A bigger window lets the model keep more of your conversation in front of it, but it does not improve reasoning ability. It makes every reply slower, consumes more memory, and past a certain length, actually reduces accuracy.
How much does it cost to fill a 1M-token context window?
It depends entirely on the model. DeepSeek V4 Flash costs roughly $0.14 to fill a 1M-token window. Claude Fable 5 costs roughly $10.00 — a 71x difference for the same amount of input. GPT-6 Sol is inexpensive below 272K input tokens but reprices the entire request at a higher rate above that threshold.
What should I do if my document is longer than the model's effective context?
Chunk it. Split the document into sections that fit within the model's reliable range, summarize each section, and then ask the model to synthesize the summaries. This "map-reduce" approach is more reliable than dumping a 500-page document into a 1M window and hoping the model reads page 350.
Letters
No letters yet — be the first to write.