Skip to content
[ back to lab ]

Tokenizer comparator

Same text, three tokenizers, three different breaks. The first place a model can fail to understand you — and the place where prompts get expensive without you noticing. Runs entirely in your browser.

GPT-4o

o200k_base

[ loading ]

[ loading ]

GPT-4 / 3.5

cl100k_base

[ loading ]

[ loading ]

Llama 3

tiktoken-style BPE

[ loading ]

[ loading ]

Claude and Gemini don’t publish their tokenizers, so they can’t run in a browser. Everything here runs on this page: no network calls, no API keys, no tracking.