Repetition penalty is a decoding adjustment used by some text generators to reduce the probability of reusing tokens or sequences that have already appeared. It can make a response less repetitive, but the effect depends on the implementation and the model configuration.
How it works
During generation, the model calculates scores for possible next tokens. A penalty can change the scores of tokens that have already been used, before the system selects the next token. The exact rule varies: some implementations consider tokens seen across the entire response, while others limit the analyzed window or treat frequencies differently.
The parameter does not understand the meaning of a repetition. It operates on tokens, which are pieces of text defined by the tokenizer. Therefore, it can reduce a useful repetition, such as a variable name or an important term, along with an unwanted repetition. The effect also depends on temperature, top-p, top-k and other generation options.
The penalty does not fix a poorly planned response nor guarantee natural variety. A very strong value can make the text strange, omit necessary words or disrupt predictable formats, such as code, JSON and lists. Before adjusting this parameter, check the provider documentation because names, scales and accepted values are not universal.
Practical example
Imagine that a support chatbot answers several times "restart the application" even after the customer says they already tried. The team can test a repetition penalty to reduce the literal repetition of tokens. However, this does not teach the system to understand that the suggestion was already tried.
To fix the cause, the flow also needs to keep the relevant conversation history and guide the model to consider previous attempts. An evaluation with dialogue examples helps compare two configurations: did the response become less repetitive without losing clarity, accuracy or consistency?
In code generation, the configuration must be tested carefully. Repeating identifiers, object keys or syntactic structures can be mandatory. If the penalty harms these repetitions, it is better to adjust the instructions or the generation strategy than to increase the parameter without observing the result.
What it is for
This adjustment can be useful when an application generates free text and repeats phrases or passages in a noticeable way. It serves as one of the options to control the generation style, together with instructions, context, temperature and sampling methods.
The parameter should be calibrated with real tasks. Separate examples where repetition is bad and cases where it is necessary, such as summaries with proper names or responses with fixed structure. Compare the outputs and track errors, not just the impression of variety.
If repetition comes from duplicated context, poorly organized memory or conflicting instructions, treat that source first. The penalty acts during token selection; it does not remove duplicated content from the prompt nor replace good context preparation. Record the value tested per provider and model, because identical settings can produce different effects between systems.