Context Window

A context window is the amount of text an AI language model can consider at one time, measured in tokens. Think of it as the model's working memory for a single interaction. Everything the model can take into account when it produces a response, your question, any instructions, documents you have pasted in, and the earlier back-and-forth of the conversation, has to fit inside this window. Whatever falls outside it is effectively invisible to the model at that moment, as if it never happened.

The mechanics hinge on tokens, which are the chunks of text a model actually processes. A token is often a word or a piece of a word, and as a rough guide a token corresponds to about three-quarters of an English word, so a few hundred words is roughly a few hundred tokens. A model with a large context window can hold a long document, a lengthy conversation, or a big block of reference material in view all at once, while a model with a small window can only consider a short span before older content has to be dropped. When a conversation or input exceeds the window, the earliest material typically gets pushed out, which is why a chatbot can seem to forget something said much earlier in a very long chat. Both the input you provide and the response the model generates draw on this same budget.

The term joins context, from the Latin contexere meaning to weave together, with window, capturing the idea of a span of text the model can see through at a given moment. As language models advanced, context windows grew dramatically, from a few thousand tokens in early systems to hundreds of thousands or more in newer ones, expanding what these models can handle in a single pass.

For a business, the context window matters in any practical use of AI on real material. Summarizing a long contract, analyzing a lengthy report, powering a support assistant that must remember a whole conversation, or feeding a model your documentation all depend on how much can fit in the window. A larger window means you can work with bigger inputs and richer context without awkward workarounds, which directly affects what tasks are feasible and how well the model performs on them. Knowing a model's limit helps set realistic expectations and design workflows that stay within it.

The nuances and common mistakes are worth understanding. A bigger context window is not automatically better in practice: models can pay less attention to information buried in the middle of a very long input, sometimes called the lost in the middle effect, and processing more tokens costs more and runs slower. A frequent error is assuming the model remembers everything from a long session when older turns have quietly dropped out of the window. Another is pasting enormous amounts of material and expecting flawless recall of every detail. The context window connects to tokens, the units it is measured in, to foundation models, whose designs set these limits, and to the temperature setting, which shapes how the model uses what is inside the window when it generates a reply.