How it works
A language model produces a response in steps. At each step, it calculates scores for possible next tokens and turns those scores into a probability distribution. A token can correspond to a word, part of a word, punctuation mark, or another text fragment.
Token probability expresses how much an option is favored by the model at that point in generation, given the available context. It is not a direct measure of truth, quality, or human confidence. A very likely token can still lead to an incorrect sentence, especially when the information is not in the context or in the model's learned data.
After calculating the distribution, the generation strategy selects the next token. A more deterministic configuration tends to pick the highest scoring option. Sampling methods can choose among several options, with parameters that change the variety of the response. The choice depends on the task: predictability can be desirable for structured output, while variety can help with creative writing.
The process repeats with the chosen token added to the context. The next distribution considers both the original text and what the model has already generated. For that reason, changing an initial word can alter all subsequent choices.
Practical example
Consider an assistant completing the sentence "To make coffee, first heat the...". In context, tokens related to "water" may receive high scores. Alternatives like "wall" may receive low scores, although it is not possible to determine values without consulting a specific model output.
If the instruction asks "answer only with one option between yes and no", the system can use more controlled generation and validate whether the response belongs to the allowed set. The probability assigned to "yes", by itself, does not prove the answer is correct. It is necessary to compare the response with the rules and the problem data.
In a programming workflow, a model can suggest the next token of a line of code. The suggestion depends on the files and snippet provided. The developer must verify that the code compiles, respects the repository conventions, and passes the relevant tests.
What it's for
This concept helps explain why models can produce different alternatives for the same request. The distribution changes according to context and generation settings. It also explains why a fluent response does not guarantee accuracy: the model is choosing plausible sequences, not automatically consulting a verified fact table.
For those integrating models via API, token probabilities can be useful when the service offers that kind of output. They can support analysis of alternatives, behavior inspection, or controlled classification. The availability and format of this data depend on the model and the API, so they should be checked in the provider's documentation.
The term also avoids a misleading interpretation of "confidence". If a system shows a high probability, that refers to the calculated preference for an option in the generation process. It does not necessarily equal the chance that the statement is true in the real world.
In summary, token probability describes an internal step of text generation. It is influenced by context, the predicted distribution, and the selection strategy. To evaluate a response, combine text inspection with sources, validation rules, or tests, according to the task type.