Tool calling

Compass

A model invoking a defined capability, which is the point at which answering becomes acting.

What is tool calling?

Tool calling is a model invoking a defined capability with arguments it has chosen. The capability is declared with a schema, so the arguments can be validated before anything is executed. It is the point at which a retrieval system stops answering questions and starts changing something.

Also called: Function calling

Why the first tool call changes what the system is for

A knowledge assistant that only answers can be wrong without being harmful, because a person reads the answer and decides what to do. The first tool call removes that person from the path, and the same misread question now cancels an appointment or releases a payment. Everything adequate for answering becomes insufficient once an approval step stands between an error and its consequence. Tool definitions are therefore written as narrowly as the task will allow.

  • A call executes without a person reading the answer first.
  • Narrow tool definitions limit what a misinterpreted question can reach.
  • Arguments chosen by a model need validation the caller cannot assume.
  • Every call has to be recorded with its arguments for an action to be auditable.

How tool calling works

  • Each capability is declared with a schema naming its arguments and its effects.
  • The model selects a capability and proposes arguments for it.
  • Arguments are validated against the schema and against the asker's entitlements.
  • Actions with consequence are held for an approval before execution.
  • The call is executed with an idempotency key so a retry cannot duplicate it.
  • The call and its result are recorded against the asker and the session.

How tool calling is evaluated

Tool calling is evaluated on argument correctness as much as on selection. Measure the following against real traffic:

  • accuracy of capability selection against a labelled question set
  • rate of argument validation failures per capability
  • proportion of consequential calls held for approval as configured
  • duplicate execution rate under retry and timeout conditions
  • completeness of call logging, including arguments and caller

Limitations, risks and common failure modes

  • A broadly defined tool lets a vague question reach a consequential system.
  • The model supplies a plausible argument the schema was too loose to reject.
  • Approval steps are configured per environment and omitted in the one that matters.
  • Retries without an idempotency key produce the same action several times over.

Related terms

Function callingThe implementation of tool calling, where the model emits a structured request matching a declared signature.Write-backReturning a result into the system of record, which is where an answering layer takes on real consequence.Model Context Protocol (MCP)An open way of exposing tools and data to a model, so an integration is declared once rather than per application.System of recordThe authoritative source for a given fact, which an AI layer must agree with rather than compete against.Context engineeringDeciding what a model should have in front of it for a given task, which has largely displaced prompt wording as the discipline that matters.Context windowThe amount of text a model can consider at once, which bounds how much evidence an answer can rest on.

All 315 terms