What Is a Context Window? Why AI Chatbots Forget Things
A context window is the amount of text an AI model can look at in one go. Learn what counts toward it, how tokens relate to words, and why long chats can lose track of earlier details.
In this article
If you have ever had an AI chatbot lose track of something you said earlier in a long conversation, you have run into the limits of its context window. The context window is the amount of text a model can work with at one time. Everything inside it is available to the model when it writes a reply. Everything outside it might as well not exist. This article explains what the window holds, how its size is counted in tokens, and what happens when a conversation grows too long.
What a context window is
Anthropic's developer documentation defines the context window as all the text a language model can reference when generating a response, including the response itself. It is different from the large body of data the model was trained on. Instead, the documentation says, it works like the model's working memory.
That distinction is the key to why chatbots seem to forget. Training is what a model learned in the past. The context window is what it can see right now, in this request.
Tokens: how the window is measured
Context windows are measured in tokens, not words. Google's documentation for its Gemini models explains that a token is a unit of text the model processes. It can be a single character or a whole word, and long words are split into several tokens. For Gemini models, the same page gives a rule of thumb: a token is about four characters, and 100 tokens is about 60 to 80 English words. Other models split text differently, so the ratio is an estimate, not a fixed law.
Pictures and sound count too. Google's documentation says an image of 384 pixels or less in both dimensions counts as 258 tokens, video in static processing counts as 263 tokens per second, and audio as 32 tokens per second. Whatever you give the model, it all takes up room in the same window.
How big are the windows?
Sizes have grown quickly. As of 31 October 2026, Google's documentation lists a window of 1,048,576 tokens for the Gemini models it describes, and Anthropic's documentation says the window is up to 1 million tokens depending on the model, with some older Claude models at 200,000. By the rule of thumb above, a million tokens is roughly 630,000 to 840,000 English words. These figures change as new models appear, so check each company's current documentation before relying on them.
What counts toward the limit
Anthropic's documentation is specific that everything in a request counts: the system prompt, every message in the conversation, tool results, images and documents, and the definitions of any tools. The reply the model generates counts as well, including any extended thinking. Google's documentation makes similar points: system instructions and tool definitions count as input tokens, and for its newest models input, thinking and output share one window.
Why a long chat starts to forget
A model does not carry a memory of your conversation from one message to the next. In the way Anthropic's documentation describes it, each turn consists of all the previous conversation history plus your new message as input, and the reply then becomes part of the input for the next turn. The conversation keeps growing inside the window.
When it approaches the limit, something has to give. The documentation describes several outcomes. If the input alone is too long, the service refuses it with an error saying the prompt is too long. Chat apps may instead drop the oldest messages on a first-in, first-out basis. Another approach is compaction, in which earlier parts of the conversation are summarised so the conversation can go on. In each case, details from early in the chat may be lost or reduced to a summary.
More is not always better
A bigger window does not mean a model uses everything in it equally well. Anthropic's documentation says that as the number of tokens grows, accuracy and recall degrade, which it calls context rot, and that choosing what goes into the context is as important as how much space is available. Google's tools can also check the size of a request before it is sent, which helps people avoid sending more than they need.
What this means in practice
- Start a new chat for a new topic. Old, unrelated text takes up room and can distract.
- Put the important details in your latest message if a conversation has been going on for a long time.
- Ask for a summary of a long chat and start fresh with it, so the key points are close at hand.
- Be selective with pasted material. Sending only the relevant part of a long document leaves more room for the answer.
Sources
Share