Smart quotes from a word processor
Curly quotes and em dashes cost several tokens each. Their ASCII twins cost one.
Non-ASCII punctuation is encoded as multiple bytes, and multi-byte characters routinely consume two or three tokens where the ASCII equivalent takes one. A curly apostrophe in every contraction across a long prompt adds up quickly.
These characters almost always arrive by accident. Somebody drafted the prompt in a word processor, a notes app, or a document editor with smart-quote substitution on, then pasted it into code. Nobody chose them.
Zero-width characters are the worst case: invisible, sometimes injected by copy-paste from web pages, and billed like any other token. They also break exact string matching in your own tests, so removing them fixes two problems.
Don’t reveal the user’s name — use “the customer” instead…Don't reveal the user's name -- use "the customer" instead...Check your own prompt
The analyser checks this pattern along with the other 25, prices each finding against your request volume, and hands back a rewritten prompt. It runs in your browser — nothing is uploaded.
Run the analyserMore on formatting
- Markdown tables padded for alignmentColumn alignment is whitespace, and whitespace is tokens.
- Shouting is billed by the letterALL-CAPS words split into more tokens than the same word in title case.
- ASCII art in a paid channelRows of equals signs are for humans reading a terminal, not for the model.
- Whitespace you are paying to storeTrailing spaces and stacked blank lines are billable and invisible.