What is model fallback?
Model fallback is the defined alternative used when the preferred model is unavailable or declines to answer. It names which model takes over and the conditions that trigger the change, together with what differs about the output afterwards. The arrangement is decided in advance rather than improvised during an outage.
Also called: Fallback model, Failover model
Why a degraded answer and a refusal are different outcomes
Providers have capacity limits and they are reached at the worst possible times, usually when a client has scheduled a volume of work. Without a defined alternative the deployment stops, and the people depending on it return to the process it replaced. With a poorly defined alternative something worse happens, as a weaker model answers in the same voice at the same stated confidence, and nobody downstream can tell which model produced which decision. The fallback has to be recorded against the output, or the evaluation record becomes a mixture of two systems.
- A defined alternative keeps the service available when a provider degrades.
- An undeclared substitution contaminates every evaluation figure afterwards.
- Some tasks are better refused than answered by a weaker model.
- Recording which model answered is what makes a later audit possible.
How model fallback works
- The conditions that trigger a fallback are stated explicitly, including timeouts as well as refusals.
- An ordered preference of models is declared per task type.
- Each alternative is evaluated against the same golden dataset as the primary.
- Tasks unsuitable for a weaker model are configured to refuse rather than degrade.
- The model that actually answered is recorded on every output.
- Fallback frequency is monitored, since sustained use indicates a capacity problem.
How model fallback is evaluated
A fallback arrangement is assessed on what the output looks like once it is in use. Examine the following with the primary model deliberately unavailable:
- quality on the golden dataset for each declared alternative
- proportion of outputs recording which model produced them
- frequency and duration of fallback episodes
- share of tasks correctly refused rather than answered weakly
- additional latency introduced by the switch
Limitations, risks and common failure modes
- The alternative is declared and never evaluated until the day it is needed.
- Outputs omit which model answered, so the record cannot be interpreted later.
- A weaker model answers at the same stated confidence as the primary.
- Fallback becomes the normal path and nobody revisits the preference order.