A circuit breaker temporarily stops routing to a provider that is failing repeatedly, rather than letting every request pay the latency and cost of a doomed call. Combined with fallback chains, circuit breakers keep latency and error rates stable when a provider degrades.

Why it matters

Retrying a provider that is already failing wastes latency and budget. A breaker stops the bleeding by skipping a provider that is down.

How it works

After repeated failures the breaker "opens" and traffic bypasses that provider for a cooldown, then retries a probe request to see if it recovered. Combined with fallbacks, it keeps error rates and latency stable during an outage.

Example

A provider returns a burst of 500s; the breaker opens for 30 seconds, requests fall through to fallbacks, and normal routing resumes once a probe succeeds.

← Back to the full glossary

Put the platform behind the terms

Route, evaluate, and monitor every AI request from one OpenAI-compatible platform.

Start Free → Explore the Features