Multi-provider fallback means a single logical request can be served by more than one provider. If the primary provider fails or is rate-limited, the gateway retries on the next provider in the chain. It is the mechanism that turns many independent provider SLAs into one more reliable application.

Why it matters

No single provider is always available or always best. Spreading a request across providers turns several imperfect uptimes into one more reliable service.

How it works

A logical request can be served by more than one provider. On failure or rate limiting, the gateway retries the next provider in the chain, and logs show which provider actually served the call.

Example

A request typically served by one provider completes on a second provider during an outage, with no change to the client.

← Back to the full glossary

Put the platform behind the terms

Route, evaluate, and monitor every AI request from one OpenAI-compatible platform.

Start Free → Explore the Features