Digital Lexicon Hub

The maximum amount of text, measured in tokens, that a language model can process at once, including the prompt and its response.

Context size decides how much material an AI assistant can consider at once, such as a full contract or a long chat history, and because cost scales with tokens, it also affects the price and speed of every request.

A team sends a 300-page report of about 150,000 tokens plus a 2,000-token system prompt to a model with a 200,000-token window, leaving room for the answer; for a 1,000-page archive it switches to retrieving the 20 most relevant chunks.

A bigger window does not guarantee better use: models can miss details placed in the middle of long inputs, and accuracy can degrade as context grows. Long prompts also raise latency and cost, so curating context still matters.

Everything in a request counts toward the window: system prompt, conversation history, tool definitions and results, attached documents and the generated output. Limits are set in tokens, as defined by LLM Tokenization, so the same text can use different token counts across models and languages. In standard transformers, attention compute grows roughly with the square of sequence length. When input exceeds the limit, applications truncate, summarize or retrieve selected passages for the Large Language Model (LLM).