What Are Tokens in AI Pricing?

If you have ever looked at an AI vendor's pricing page and seen costs quoted "per million tokens," you have run into the single most important concept in AI billing. You are not charged per question, per word, or per minute — you are charged per token. Understand what a token is and how they are counted, and AI pricing stops being mysterious. You can estimate costs before you build, explain a surprise bill, and write prompts that do the same job for less.

This explainer covers what a token actually is, why input and output tokens are billed separately, how the context window quietly inflates your costs, and how caching can claw some of that back. No specific prices appear here on purpose — they vary widely by vendor and model and change often — but the mechanics below are stable across providers.

What a token actually is

A token is a chunk of text the model reads and writes in. It is usually not a whole word: tokenization breaks text into subword pieces, so a long or unusual word like "tokenizer" might split into "token" and "izer," while common short words are a single token. Spaces and punctuation count too. As a rough rule of thumb, in English a token averages around three-quarters of a word, so a few hundred words is several hundred tokens — but the only reliable count is the one the model produces, since every vendor's tokenizer differs.

The practical takeaway: everything you send and everything you receive is measured in tokens, and that count — not the number of messages — drives your bill. A single long request can cost more than dozens of short ones.

Input tokens vs output tokens

Every request has two billed parts. Input tokens are everything you send the model: your question, the system instructions, any documents you paste in, and the entire prior conversation. Output tokens are what the model generates back.

Vendors typically price these at different rates, and output tokens usually cost more than input tokens. The reason is computational: generating text one token at a time is more expensive than reading text you already provided. This has a direct design implication. If you ask for a 2,000-word essay, you pay for a large, more expensive output. If you instead ask the model to read a 2,000-word document and reply with a one-line answer, your input is large but cheap and your output is tiny. Knowing which side of the ledger you are loading lets you shape requests for cost.

Why long prompts cost more

Here is the part that surprises people: in a conversation, the cost grows with every turn even if your individual messages stay short. Most chat applications resend the entire conversation history as input on each new turn, so the model has context. Turn ten re-sends turns one through nine. Add a long system prompt and a pasted document, and your input token count — billed every single time — balloons.

This is why "just paste the whole manual into the prompt" is convenient but expensive: that manual is re-billed as input on every follow-up question. The context window is the maximum number of tokens a model can consider at once, and filling it does not just risk hitting a limit — it maximizes your per-request input cost. Long prompts are not bad, but they are not free, and the cost is paid repeatedly.

How caching changes the math

Prompt caching is the main tool vendors offer to fight this. When a large chunk of your input stays identical across requests — a fixed system prompt, a set of tool definitions, a long reference document — caching lets the model reuse the already-processed version instead of reprocessing it from scratch. Reusing cached input is billed at a much lower rate than fresh input, and it also reduces latency because the model skips work it has already done.

The mechanics matter. Caching works on a prefix: the unchanging part must come first and match exactly, because any change invalidates the cache. Cached entries also expire after a short window, so caching pays off when you make repeated requests against the same context in a short span — multi-turn chats, repeated questions about one document, or an agent reusing the same tools across many steps. Put your stable content at the front and your variable content at the end, and caching does the rest.

Bottom line

AI is billed by the token — subword chunks of everything you send and receive. Input (your prompt, history, and documents) and output (the model's reply) are priced separately, with output usually the more expensive of the two. Long prompts and long conversations cost more because input is re-sent and re-billed on every turn, bounded by the model's context window. Caching recovers much of that cost when a large prefix stays constant. Exact prices vary by vendor and model and change often, so always check the current rate card — but the mechanics here tell you where the money goes, which is most of what you need to keep a bill under control.

Sources and further reading