A context window is the information a model can consider during one generation. It includes instructions, conversation history, retrieved documents, tool descriptions, tool results, images represented as tokens, and space reserved for the response. It is closer to working memory than to the model’s learned knowledge.
This distinction prevents a common mistake. A model may know a general concept from training but lack the private fact needed for your task. The reverse can also happen: the fact may be present in the context, yet the model can fail to notice or apply it when the request is crowded with irrelevant material.
Every token competes for attention. A large window lets an application submit more material, but it does not guarantee uniform recall or reasoning across that material. Important evidence can become hard to find, repeated passages can bias an answer, and long tool outputs can displace the instructions that define success.
Budget from the output backward. Reserve enough room for the response and any reasoning or tool activity the model may generate. Then allocate space to stable instructions, the current question, recent conversation state, and supporting evidence. Do not fill the remaining capacity simply because it exists.
Prefer high-signal context. For a document question, include the relevant passages with their titles, dates, and source identifiers. For a coding task, include the interfaces, tests, error output, and nearby implementation rather than the entire repository. For an agent, make available lightweight references it can inspect when needed instead of loading every possible resource upfront.
Long conversations need an explicit state strategy. Compaction summarizes earlier work into a smaller artifact. Structured notes preserve decisions, constraints, open questions, and completed steps outside the immediate transcript. Retrieval brings back only what the next action needs. These approaches trade verbatim history for continued coherence.
Compaction can lose details, so the summary should distinguish durable facts from temporary observations. Record exact identifiers, paths, decisions, evidence links, and unresolved risks. Keep important artifacts—tests, specifications, data, and final outputs—in their native form rather than relying on a conversational summary to reproduce them.
Prompt caching can reduce cost or latency for repeated prefixes, but cached tokens still occupy the context window. Caching changes how repeated input is processed or billed; it does not turn the window into unlimited memory.
Tool use adds another pressure. Schemas, results, screenshots, and intermediate reasoning all consume capacity. Clear obsolete tool results when the platform supports it, summarize large outputs, and retain pointers to raw evidence. A useful agent should be able to re-open a source rather than carrying every byte forever.
Evaluate context design with realistic long inputs. Put important facts at different positions, add plausible distractors, and test follow-up questions after several tool calls. Measure not just whether the model can quote a fact, but whether it applies the correct version and cites the right source.
The best context is not the maximum context. It is the smallest current working set that preserves the goal, constraints, evidence, and next decision. Larger windows expand what is possible; careful context engineering determines what remains reliable.
AI Tools Radar separates product facts, editorial judgment, and commercial placement. Updated facts retain their verification date.