How it works
A language model generates output token by token. At each step, it estimates which tokens can come next. Greedy decoding picks only the most likely token at that moment and moves on. Beam search keeps several partial sequences in parallel, called hypotheses.
Beam width, also called beam width, defines how many hypotheses continue to be considered. With width 1, the process approaches the greedy choice. With a larger width, the system compares more possible paths and discards those that become less promising as the sequence grows.
Each hypothesis receives a cumulative score based on the probabilities assigned by the model. Since long sequences can accumulate scores differently from short ones, implementations may apply length normalization or other criteria. These adjustments change the algorithm's preference and need to be evaluated in the context of the task.
Beam search is not a search on the internet nor a search inside a database. It is a decoding method: a way to choose a sequence among the possibilities the model can generate. It does not create new knowledge and does not guarantee that the best scoring sequence is correct or useful.
Practical example
Imagine a translation system that needs to translate "a reunião foi adiada". After generating "the meeting", the model may consider "was postponed", "is delayed" or other continuations. Instead of committing immediately to just one, the beam preserves some options and evaluates how they develop in the next words.
A continuation that seems less likely after the first word can form, in the end, a more coherent sentence. The algorithm can notice this because it compares larger partial sequences, not just the next isolated word. Still, the result depends on the scores and the rules used to end and compare hypotheses.
In text generation, for example, a wide beam can produce a more predictable or repetitive response, depending on the model and the parameters. Therefore, increasing the width does not automatically mean getting a better response. The result should be compared with other methods on the same task.
What it is for
Beam search is used in tasks where the output is a structured sequence, such as machine translation, speech recognition, image captioning and text generation. In these cases, evaluating some complete alternatives can be more useful than choosing each token in isolation.
The method is especially interesting when quality depends on relations between distant parts of the output. A word may seem improbable locally, but contribute to a more appropriate whole sentence. Maintaining hypotheses helps postpone irreversible decisions until more context is generated.
Cost also matters. Each hypothesis needs to be expanded and scored, so larger beams can consume more memory and processing. In interactive applications, this latency can be undesirable. Developers can start with a small width and compare quality, response time and diversity on representative examples.
Beam search does not replace human evaluation nor adequate metrics. In translation, for example, it is necessary to verify fidelity and naturalness; in code, it is necessary to run tests. Use the method when the task benefits from comparing sequences, and choose the width by experimentation, not by a universal rule.