AI model providers can go offline, hit usage limits, or return errors. Model fallback is the plan B: if the primary model fails, the system tries another one, without the user noticing.

Como funciona

You define an ordered list of models. The first is the preferred one. If the call fails, the application moves on to the second, then the third, and so on.

The situations that usually trigger fallback are:

  • Provider error: unavailability or internal failure.
  • Usage limit: the rate limit error code indicates you exceeded your quota.
  • Timeout: the model took longer than acceptable.
  • Invalid response: the output does not follow the expected format.

Ideally, combine fallback with retries and increasing wait times. For temporary errors, retrying the same model can resolve it. If it persists, then you switch.

Compatibility needs attention. Different models have different context windows, tool calling formats, and behaviors. A prompt that works well in one may perform worse in another.

Exemplo prático

Your support agent uses a primary model. At peak hours, the provider returns a limit error. Without fallback, customers get no response. With fallback, the agent switches to a reserve model and keeps serving.

A configuration sketch:

JSON · config/modelos.json
{
  "ordem": ["modelo-principal", "modelo-reserva-1", "modelo-reserva-2"],
  "tentativas_por_modelo": 2,
  "timeout_segundos": 30
}

The names are illustrative. Use the models you actually subscribe to or host.

Para que serve

It is used to provide service continuity, especially in products used by customers. It also helps agents that run for hours: if a model stalls in the middle of a long task, the work continues instead of being lost.

AI gateways and routers like OpenRouter offer this feature ready to use. Code agent tools also allow configuring more than one provider, and on bridgecode you use your own keys and models, which makes it easier to have reserves.

Diferença entre fallback e roteamento

Routing chooses the ideal model before calling. Fallback kicks in after a failure. A good system uses both.

Cuidados

Test the reserve model for real, not just the primary one. Log when fallback is used, to track cost and quality. And pay attention to privacy: when switching providers, your data starts going to another company, which can matter for sensitive data.