Model Fallbacks

Specify backup models that are tried in order if the primary model fails or is unavailable.

Configure fallback models

Pass a models array in providerOptions.gateway:

{
  "model": "openai/gpt-4o",
  "messages": [{"role": "user", "content": "Hello"}],
  "providerOptions": {
    "gateway": {
      "models": ["anthropic/claude-sonnet-4-6", "groq/llama-3.1-8b-instant"]
    }
  }
}

The gateway tries:

1. openai/gpt-4o (primary model)
2. anthropic/claude-sonnet-4-6 (first fallback)
3. groq/llama-3.1-8b-instant (second fallback)

The response comes from the first model that succeeds.

Each model runs the full routing chain

For each model in the list, the gateway runs the full routing chain:

1. Operators - try operators serving this model (if available)
2. Direct provider - call the provider API directly

If both tiers fail for a model, the gateway moves to the next model in the list.

Combining with provider ordering

Use models with order to control both model fallback and provider preference:

{
  "model": "openai/gpt-4o",
  "providerOptions": {
    "gateway": {
      "models": ["anthropic/claude-sonnet-4-6"],
      "order": ["bedrock", "anthropic"]
    }
  }
}

This tries:

1. openai/gpt-4o via available providers
2. anthropic/claude-sonnet-4-6 via Bedrock first, then Anthropic direct

Observability

When fallbacks occur, the routing trace shows every model and provider attempted:

X-Tangle-Routing-Trace: openai/gpt-4o[openai(err:5001ms)], anthropic/claude-sonnet-4-6[anthropic(200:1847ms)]