Function Calling Maturity: Where Models Differ and Why It Breaks Agents

If you build agents, the model’s function-calling maturity determines how reliably the agent’s function calls work, and building on the assumption that function calling is uniformly reliable ignores the variation that breaks agents when the model’s function calling is weaker than assumed. The reliability of the model’s function calling is a property that varies by model, and an agent’s reliability depends on it, which means the function-calling maturity of the model the agent operates on is a factor in the agent’s reliability that has to be accounted for. Understanding where the variation is and how it breaks agents is what lets you build agents robust to the function-calling reliability the model provides.

Where models differ in function calling

Function calling requires the model to do several things reliably: decide when to call a function, choose the right function, and produce well-formed arguments conforming to the function’s schema, and models differ in how reliably they do each. A more mature model calls the right function with well-formed arguments consistently, and a less mature model makes more mistakes, calling the wrong function, producing malformed arguments, or failing to call a function when it should, which are the function-calling weaknesses that vary across models. The schema adherence that function calling depends on is a distribution, and the width of that distribution, how often the model produces malformed or wrong calls, varies by model’s function-calling maturity.

These differences matter because an agent’s function calls are how it acts, and the reliability of those calls determines the reliability of the agent’s actions. An agent on a model with mature function calling has reliable function calls and acts reliably, and the same agent on a model with less mature function calling has less reliable function calls, more wrong functions, more malformed arguments, and acts less reliably, breaking in ways the mature model did not. The variation in function-calling maturity is therefore a variation in agent reliability, and an agent built on one model’s function-calling maturity can break when moved to a model with less mature function calling, because it inherits the less mature model’s function-calling weaknesses. Models differ in function-calling maturity in how reliably they call the right function with well-formed arguments, and that difference breaks agents because the agent’s reliability depends on the function-calling reliability the model provides.

Why the differences break agents

The function-calling differences break agents because agents are built on function calling and inherit its reliability, so a model with less mature function calling produces an agent with less reliable actions. When the model calls the wrong function, the agent takes the wrong action, and when the model produces malformed arguments, the function call fails or is coerced into something wrong, and when the model fails to call a function it should, the agent fails to take an action it needed to, and each of these function-calling failures is an agent failure caused by the model’s function-calling weakness. An agent’s reliability is bounded by its function-calling reliability, and a model with less mature function calling lowers that bound, breaking the agent in proportion to the function-calling weakness.

This breaks agents especially when they are moved between models or built assuming a maturity the model does not have, because an agent built and tested on a model with mature function calling can break when operate on a model with less mature function calling, inheriting the function-calling weaknesses it was not built to handle. This connects to model choice, where the model’s function-calling maturity is a factor in whether a model suits an agentic use, and a model that performs well on other measures but has weaker function calling can be a poor choice for an agent that depends heavily on function calling. The differences break agents because the agent inherits the model’s function-calling reliability, and a less mature model’s function-calling weaknesses become the agent’s failures, which is why the model’s function-calling maturity is a factor in the agent’s reliability that building on an assumed maturity ignores. The differences break agents by lowering the function-calling reliability the agent depends on, which the agent inherits as less reliable actions.

Building robust to function-calling variation, and the trade-off

Building agents robust to function-calling variation means not assuming the model’s function calling is reliable but building the validation and handling that catches function-calling failures, which makes the agent robust to the function-calling reliability the model provides. Validating the model’s function calls at the boundary, checking that the called function and its arguments are correct before acting, catches the wrong functions and malformed arguments that a less mature model produces, so the agent’s actions are protected from the function-calling failures rather than inheriting them directly. This validation is what makes the agent robust to the function-calling variation, because it catches the function-calling failures regardless of the model’s maturity, so the agent’s reliability depends less on the model’s function-calling maturity.

The trade-off is that building this robustness costs engineering, and naming it keeps the robustness matched to the need. The validation and handling that make the agent robust to function-calling failures is real engineering, more than assuming the function calling is reliable, and it is justified because the alternative, assuming a function-calling maturity the model may not have, breaks the agent when the model’s function calling is weaker than assumed. The robustness is especially worth building for agents that might operate on different models or on models with less mature function calling, because those agents face the function-calling variation directly, and for agents on a model with very mature function calling the validation is still worth having as protection against the function-calling failures that even a mature model occasionally produces. The deeper point is that the model’s function-calling maturity should inform the agent’s design, building more robustness for less mature function calling and choosing models with function-calling maturity adequate for the agent’s function-calling dependence, so the agent’s reliability is matched to the function-calling reliability available. Building robust to the function-calling variation, with the robustness matched to the model’s maturity and the agent’s dependence on function calling, is what makes the agent reliable despite the variation.

Where it almost broke

We built an agent on a model with mature function calling, where the function calls were reliable, and the agent worked well, and when we moved the agent to a different model, it broke, because the new model had less mature function calling, producing more wrong functions and malformed arguments that the agent, built assuming reliable function calling, inherited directly as wrong and failed actions. The agent’s reliability had depended on the function-calling maturity of the original model, and the less mature function calling of the new model broke it, because the agent had no handling for the function-calling failures it had not faced on the mature model.

The correction built the agent robust to function-calling variation. We added validation of the model’s function calls at the boundary, checking that the called function and its arguments were correct before acting, which caught the wrong functions and malformed arguments the less mature model produced, so the agent’s actions were protected from the function-calling failures rather than inheriting them. We accounted for the model’s function-calling maturity in the design, building more robustness where the function calling was less reliable. The agent became robust to the function-calling variation, working on the less mature model because it caught the function-calling failures rather than inheriting them, and the same robustness protected it on the mature model against the occasional function-calling failure. The lesson was that models differ in function-calling maturity and the differences break agents that inherit the function-calling reliability, and building robust to the variation with boundary validation is what makes the agent reliable despite the model’s function-calling maturity.

Do not inherit the model’s function-calling weaknesses

The reframe that makes agents robust across models is not assuming the model’s function calling is reliable but validating it, so the agent does not inherit the model’s function-calling weaknesses as its own failures. Function calling maturity varies across models, and the variation breaks agents that inherit the function-calling reliability, because a less mature model’s wrong functions and malformed arguments become the agent’s failures. Building the boundary validation that catches function-calling failures is what makes the agent robust to the function-calling reliability the model provides.

The teams that build agents reliable across models are the ones that accounted for function-calling maturity, validated the model’s function calls at the boundary, and matched the robustness to the model’s maturity and the agent’s dependence on function calling. An agent built assuming reliable function calling breaks when the model’s function calling is weaker than assumed, inheriting the function-calling weaknesses it was not built to handle. Not inheriting the model’s function-calling weaknesses, by validating the function calls rather than trusting them, is what makes an agent reliable despite the variation in function-calling maturity, which is the factor in agent reliability that building on an assumed maturity ignores.