Function Calling

Connect chat models to external tools using function calling

Some models have the ability to use function calling. This allows chat models to connect to external tools to perform actions or retrieve information.

For OpenAI models: Set strict: true in your function definition to force the model to respond with the exact schema you have provided in the function definition.

Example use-cases:

• Let models fetch data: You can let models fetch data from external databases or tools. For example, if the user asks "what is my balance?" an assistant can fetch the data from an external database.
• Enabling models to take action: You can let models perform actions such as creating a new database entry or updating an existing one.
• Perform computations: You can let models perform computations such as mathematical calculations.
• Data extraction: You can let models extract data from unstructured text.
• UI modifications: You can let models modify the UI based on user input. For example, you can let the model render a pin on a map based on a user's input.

• STEP 1

  YOUR APP

  Your app calls the API with your prompt and definitions of the functions the model can use.

• STEP 2

  THE MODEL

  The model decides whether to respond to the user or call one or more functions.

• STEP 3

  API

  The API returns the function call and arguments to your app.

• STEP 4

  YOUR APP

  Your app executes the function call with the provided arguments.

• STEP 5

  YOUR APP

  Your app calls the API with the prompt and the result of the function call.

When you use function calling, the model never actually executes the function. Instead, it returns a JSON object that contains the name of the function to be executed, as well as the parameters to be passed to the function according to the schema you provided.

The starting point for function calling is choosing which function in your codebase to allow the model to generate arguments for.

In this example, we'll allow the model to call the get_order_date function in your codebase which accepts a order_id argument and queries a database for the order date. Your function could look something like this:

The next step is to define the schema for the function. This is done by creating a JSON schema that describes both what the function does and what arguments it accepts.

The parameters field of the function definition should be a JSON schema that describes the arguments the function accepts. The model will use this schema to generate valid arguments for the function.

In this example, we'll define the function schema as follows:

The final step is to pass the function schema to the model as available "tools" when calling the Chat Completions API.

As always, we will provide an array of "messages" to the model, which would contain your prompt or a whole conversation between the assistant and user.

In this example, we'll call the Chat Completions API with the function and messages for an assistant that handles customer inquiries about orders.

If the model doesn't decide to call a function

If the model decides that no function call is needed, you will receive a normal chat completion response to the user's message as usual.

For example, in this case the response.choices[0].message will contain the assistant's message to the user like so:

In an assistant use-case, you would generally want to show this response to the user and let them continue the conversation, in which case you will call the API again (with both the latest responses from the assistant and user appended to the messages array).

If we assume the user responds with their order ID, we will send a new request to the API like this:

If the model decides to call a function

If the model decides to call a function, you will receive a function call object in the response.choices[0] object.

Here is an example of what the response object looks like:

Handling the function call

You will need to handle the function call in your codebase. In this example, we'll assume that the function call contains the order ID and we'll use it to execute the function.

After you have executed the function, you will need to return the result back to the Chat Completions API so the model can generate the actual response to the user.

That is all there is to it! You've now successfully implemented function calling.