TL;DR A token is a small chunk of text, a short word or part of a longer one, turned into a number. Models read tokens, not words, and that one fact explains how you are billed, why context has limits, and a lot of odd model behaviour.

You have seen the word everywhere: priced per token, a 128k token limit, “that used too many tokens.” It sounds technical, but the idea is simple. Here is the one sentence to keep: a language model does not read words, it reads tokens. Before the model sees anything you typed, your text is chopped into small chunks, and every chunk is swapped for a number. The model only ever works with those numbers. Everything else, the cost, the limits, the occasional weirdness, follows from that.

Why not just use words?

Words seem like the obvious unit, but they are a bad fit for a machine. There are millions of them, they change with every new slang term or product name, and different languages break into words differently. Letters go too far the other way: single characters carry almost no meaning on their own, so the model would have to work much harder to reassemble them.

Tokens sit in the sweet spot between the two. Common whole words like “the” or “model” get their own single token. Rarer or longer words get split into reusable pieces. So tokenization is two tokens, and an unusual word like unbelievably might be three. With a vocabulary of only around a hundred thousand of these pieces, the model can spell out almost any word, name, or typo it has never seen before.

A three-step diagram. Step one: the sentence 'Tokenization is surprisingly simple.' as typed. Step two: the same sentence split into seven token chunks, including 'Token' and 'ization' as separate pieces and leading spaces attached to words. Step three: each token shown as a numeric ID.
Your text is split into tokens, then each token is looked up as a number. Note that spaces usually ride along with the word that follows them, and one word can become several tokens.

A useful rule of thumb for English: one token is about four characters, and one hundred tokens is roughly seventy-five words. So a page of text is around five hundred tokens, and a long article like this one is a few thousand. Numbers, code, emoji, and languages that do not use spaces tend to cost more tokens than you would guess, which is worth remembering when something feels surprisingly expensive.

The context window: a fixed budget

A model can only look at so many tokens at one time. That limit is called the context window, and you can think of it as the size of the model's desk. Everything has to fit on the desk at once: the instructions, your question, any documents you pasted in, the back and forth of the conversation so far, and the reply the model is about to write.

A diagram of a context window drawn as a fixed-width bar labelled 128,000 tokens. It is divided into a system section, a larger section for the prompt, history and documents, a section for the reply, and some free space. Below, a second bar shows that when a long chat fills the window, the oldest tokens drop off and only the most recent are kept.
The context window holds the input and the output together. When a conversation grows past it, the oldest tokens fall off the edge, which is why a long chat can seem to forget how it started.

This is the source of a behaviour people find spooky. When a chat runs long enough to overflow the window, the earliest messages get pushed out to make room. The model is not being forgetful or lazy. It genuinely cannot see those tokens any more, because they no longer fit on the desk. A bigger context window is a bigger desk, not a better memory.

Input tokens and output tokens

Tokens come in two kinds, and the split matters because it is how you are charged. Input tokens are everything you send in: your prompt, the system instructions, the pasted document. Output tokens are what the model writes back. Almost every provider prices these two separately, and output tokens usually cost more, because generating text is the expensive part.

This explains a few things that catch people out. Pasting a long document into every message is not free; you pay for those input tokens again on each turn, because the whole conversation is resent each time. Asking for a shorter answer genuinely saves money. And a model that “thinks” before answering is spending output tokens on that thinking, which is why reasoning-heavy requests can cost more than the final answer's length suggests.

Why any of this matters

If you only remember one thing, make it this: the model works in tokens, so tokens are the unit of almost everything practical. Cost is counted in tokens. The context limit is counted in tokens. Speed is roughly how many tokens the model can produce per second. When you understand that a token is just a chunk of text turned into a number, the pricing pages, the limits, and the odd bits of behaviour stop looking arbitrary.

None of it is magic. It is bookkeeping on a very long list of numbers. That framing is the one I keep coming back to when I am designing something real, because it tells me exactly where the cost and the limits are going to come from before I have written a line of code.