AJ Learning Hub logoAJ Learning Hub

Token

A token is a chunk of text (a word, part of a word, or punctuation) that a language model reads and generates; models process text as sequences of tokens, not raw characters.

Analogy

Like breaking a sentence into LEGO bricks the model can snap together, where a brick is roughly 3-4 characters of English.

Why it matters

Token counts determine your API costs and how much text you can fit in a prompt, so estimating them helps you size automations and control spend.

In practice

The phrase "GoHighLevel automation" is about 5-6 tokens, not 2 words.

Related terms:Context WindowInferencePrompt Caching