TL;DR Context engineering is the discipline of deciding what information enters the model's finite context window, and how it is retrieved, ordered, and compressed. It is a superset of prompting, retrieval, and memory. Once the model is fixed, it is usually the single biggest lever you have on quality.
Most people reach for prompt engineering when an answer comes back wrong: reword the instruction, add a “think step by step,” try again. That is table stakes, and it stops paying off quickly. The thing that actually moves quality in a shipped system is one level up. It is the decision, made fresh on every request, about what information the model gets to see at all. That decision is context engineering, and it is where the real leverage lives.
The window is finite
A model does not read your text the way you do. It reads tokens, the small chunks covered in the tokens piece, and the context window is measured in exactly those units. That window is a fixed budget. Every token you spend on one thing is a token unavailable for another, so more context is not the same as better context. Past a point, stuffing the window actively hurts: the well-documented “lost in the middle” effect shows that models attend strongly to the start and end of a long input and much less to the material buried in the middle. The right mental model is not a bucket you fill to the brim. It is a budget you allocate on purpose.
What earns a slot
If the window is a budget, the question for every candidate token is blunt: does this change the answer? If it does not, it does not belong. What usually earns a slot is a short list: the system instructions that define the task, the handful of retrieved facts the answer actually turns on, the slice of conversation history that is still relevant, and the results of any tools the model called. What gets cut is the boilerplate, the entire document when three sentences would do, the full transcript when a summary carries the same weight. Relevance beats volume every time. Asking for structured output helps here too, because a constrained schema stops the model from spending tokens on prose you were only going to parse away.
Retrieval and memory
Two familiar techniques are really just context engineering wearing different hats. Retrieval-augmented generation, the RAG idea, is context engineering at query time: instead of hoping the answer is in the weights, you search a corpus and pull the few passages that matter into the window right before you ask. Memory is the same discipline stretched across turns. Short-term memory is the running conversation you carry forward; long-term memory is the persistent store you write to and read back from over many sessions. Different names, one job. Both are machinery for getting the right thing into the window at the right moment, and nothing more.
Order and compression
Choosing the tokens is half the work. The other half is arranging them. Because attention is uneven, placement is a lever: put the material the answer depends on where the model actually looks, near the start or the end, and do not let the critical instruction get lost mid-context behind ten pages of retrieved filler. When a history genuinely will not fit, you compress it, summarizing older turns into a dense recap that preserves the decisions and drops the chatter. And once your prefix is stable, the system prompt and the instructions that do not change per request, you cache it, so you pay to process those tokens once instead of on every call. Same information, far lower cost.
Why it matters
This is the part that matters most once something is deployed in front of real users. When a production system returns a wrong answer, the instinct is to go hunting for a cleverer prompt. In my experience the fix is almost never there. It is in the window. The model was missing the one document that held the answer, or it was drowning in ten it did not need, or the key fact was sitting in the dead middle of a long context. Fix what was, or was not, in the window and the wording of the prompt barely matters.
That is the line I keep coming back to: a demo works because someone hand-fed it the perfect context, and a product works because the system assembles that context automatically, every time, for inputs nobody anticipated. Prompting gets you the demo. Context engineering is what turns it into something you can ship.