Tool calling (function calling)

Tool calling lets a model ask your program to run a function, such as a search or a calculator, and then use the result in its answer.

  • Using models
  • Updated

A model cannot browse the web or run code by itself. With tool calling you describe the available functions to it, and when one would help the model replies with a structured request, usually JSON, that names the function and its arguments. Your program runs the function, returns the result and the model carries on. It is the building block of AI agents.

Support is a property of both the model and the server: the model has to be trained for the format, and an inference server such as vLLM or SGLang needs the matching parser turned on. Open models differ a lot in how reliably they call tools, so test with your own tools and keep a human check on any action that cannot be undone.

Related terms

  • AI agent An AI agent is a system in which a model plans and carries out multi-step tasks by calling tools, such as running code, searching the web or editing files.
  • OpenAI-compatible API An OpenAI-compatible API is a server that accepts the same requests as the OpenAI API, so apps written for it can use an open model instead.
  • Reasoning model A reasoning model works through a problem step by step in a 'thinking' phase, visible or hidden, before it gives its final answer.

Keep reading

All terms