Over the last couple of years, we’ve learned just how powerful language models can be.  With the advent of gpt-35-turbo and gpt-4, these models can accomplish more than ever before and we're seeing customers be successful with an array of new scenarios. While these models are incredibly useful on their own, you can accomplish even more by integrating them with other systems and tools. 

 

Function calling is now available in Azure OpenAI Service and gives the latest 0613 versions of gpt-35-turbo and gpt-4 the ability to produce structured JSON outputs based on functions that you describe in the request. This provides a native way for these models to formulate API calls and structure data outputs, all based on the functions you specify. It's important to note that while the models can generate these calls, it's up to you to execute them, ensuring you remain in control.

 

How function calling works

The latest versions of gpt-35-turbo and gpt-4 have been fine-tuned to learn how to work with functions. If one or more functions are specified in the request, the model will then determine if any of the functions should be called based on the context of the prompt. When the model determines that a function should be called, it will then respond with a JSON object including the arguments for the function.

 

At a high level you can break down working with functions into three steps:

Step #1 – Call the chat completions API with your functions and the user’s input

Step #2 – Use the model’s response to call your API or function

Step #3 – Call the chat completions API again, including the response from your function to get a final response

 

The sections below give examples of different scenarios where function calling could be helpful or you can check out our samples to try out an end-to-end example of function calling.

 

Use cases for function calling

Function calling unlocks a variety of new use cases. You can use function calling to help you retrieve relevant data from data sources or APIs, integrate with other systems or tools, create structured outputs from your prompts, and more.

 

Retrieve data from data sources or APIs

One of the most common scenarios for function calling is to use them to write calls to retrieve data from a search index, database, or API to inform the response to the user input.  Whether you want to leverage data from the web, your own documents, or other data sources, you can use function calling to determine when you need to augment your prompts with additional data.

 

Example:

 

messages= [
    {"role": "user", "content": "Find beachfront hotels in San Diego for less than $300 a month with free breakfast."}
]

functions= [  
    {
        "name": "search_hotels",
        "description": "Retrieves hotels from the search index based on the parameters provided",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The location of the hotel (i.e. Seattle, WA)"
                },
                "max_price": {
                    "type": "number",
                    "description": "The maximum price for the hotel"
                },
                "features": {
                    "type": "string",
                    "description": "A comma separated list of features (i.e. beachfront, free wifi, etc.)"
                }
            },
            "required": ["location"],
        },
    }
]  

response = openai.ChatCompletion.create(
    engine="gpt-35-turbo-0613",
    messages=messages,
    functions=functions,
    function_call="auto", 
)

print(response['choices'][0]['message'])

 

Response:

 

{
  "role": "assistant",
  "function_call": {
    "name": "search_hotels",
    "arguments": "{\n  \"location\": \"San Diego\",\n  \"max_price\": 300, \n  \"features\": \"beachfront, free breakfast\"\n}"
  }
}

 

 

Integrating with APIs or tools

You can also use function calling to add in new skills that augment the model’s capabilities.  For example, you may want to give the model access to functions to perform calculations or do other tasks that the LLM might not be inherently good at.  By defining these functions, you can reliably interact with various external APIs or functions you’ve defined in code.

 

Function calling can also be used to take actions such as writing data to a database, making orders, or sending notifications. However, it’s important to understand the implications if something goes wrong when taking an action before you implement functionality like this. In general, we recommend always building in a user confirmation flow before allowing the model to take actions that could have harmful side effects.

 

Example:

 

messages= [
    { "role": "user", "content": "Last month Fabrikam made $73,846 in sales. Based on that, what would the annual run rate be?"}
]

functions= [  
    {
        "name": "calculator",
        "description": "A simple calculator",
        "parameters": {
            "type": "object",
            "properties": {
                "num1": {"type": "number"},
                "num2": {"type": "number"},
                "operator": {"type": "string", "enum": ["+", "-", "*", "/", "**", "sqrt"]},
            },
            "required": ["num1", "num2", "operator"],
        },
    }
]  

response = openai.ChatCompletion.create(
    engine="gpt-35-turbo-0613",
    messages=messages,
    functions=functions,
    function_call="auto", 
)

print(response['choices'][0]['message'])

 

Response:

 

{
  "role": "assistant",
  "function_call": {
    "name": "calculator",
    "arguments": "{\n  \"num1\": 73846,\n  \"num2\": 12,\n  \"operator\": \"*\"\n}"
  }
}

 

Creating structured outputs

It’s already possible to have these models output JSON, but often this requires additional parsing and can be tricky if the model occasionally starts the response with somethig like “Here is the JSON:” or adds other text in the response.

 

Function calling gives you a more reliable way to have the model create structured JSON outputs from text which can be valuable when you need to extract entities from text or when dealing with tasks that necessitate an organized and easy-to-interpret outputs.

 

Example:

 

messages= [
    {"role": "system", "content": "Assistant is a large language model designed to extract structured data from text."},
    {"role": "user", "content": "There are many fruits that were found on the recently discovered planet Goocrux. There are neoskizzles that grow there, which are purple and taste like candy. There are also loheckles, which are a grayish blue fruit and are very tart, a little bit like a lemon. Pounits are a bright green color and are more savory than sweet. There are also plenty of loopnovas which are a neon pink flavor and taste like cotton candy. Finally, there are fruits called glowls, which have a very sour and bitter taste which is acidic and caustic, and a pale orange tinge to them."}
]

functions= [  
    {  
        "name": "extract_fruit",  
        "type": "function",  
        "description": "Extract fruit names from text.",  
        "parameters": {  
            "type": "object",  
            "properties": {  
                "fruits": {  
                    "type": "array",  
                    "items": {  
                        "type": "object",  
                        "properties": {  
                            "fruit": {"type": "string",  "description": "The name of the fruit."  },  
                            "color": {"type": "string",  "description": "The color of the fruit." },  
                            "flavor": {"type": "string",  "description": "The flavor of the fruit."}  
                        },  
                        "required": ["fruit", "color", "flavor"]  
                    }  
                }  
            },  
            "required": ["fruits"]  
        }  
    }  
]  

response = openai.ChatCompletion.create(
    engine="gpt-35-turbo-0613",
    messages=messages,
    functions=functions,
    function_call={"name": "extract_fruit"}, # force the model to call the function
)

print(response['choices'][0]['message'])

 

Response:

 

{
  "fruits": [
    {"fruit": "neoskizzles", "color": "purple", "flavor": "candy"},
    {"fruit": "loheckles", "color": "grayish blue", "flavor": "tart"},
    {"fruit": "pounits", "color": "bright green", "flavor": "savory"},
    {"fruit": "loopnovas", "color": "neon pink", "flavor": "cotton candy"},
    {"fruit": "glowls", "color": "pale orange", "flavor": "sour and bitter"}
  ]
}

 

Using function calling responsibly

Like any AI system, using function calling to integrate language models with other tools and systems presents potential risks. It’s important to understand the risks that function calling could present and take measures to ensure you use the capabilities responsibly.

 

Here are a few tips to help you use functions safely and securely:

• Validate Function Calls: Always verify the function calls generated by the model. This includes checking the parameters, the function being called, and ensuring that the call aligns with the intended action.
• Use Trusted Data and Tools: Only use data from trusted and verified sources. Untrusted data in a function’s output could be used to instruct the model to write function calls in a way other than you intended.
• Follow the Principle of Least Privilege: Grant only the minimum access necessary for the function to perform its job. This reduces the potential impact if a function is misused or exploited. For example, if you’re using function calls to query a database, you should give your application read-only access to the database. You also shouldn’t depend solely on excluding capabilities in the function definition as a security control.
• Consider Real-World Impact: Be aware of the real-world impact of function calls that you plan to execute, especially those that trigger actions such as executing code, updating databases, or sending notifications.
• Implement User Confirmation Steps: Particularly for functions that take actions, we recommend including a step where the user confirms the action before it is executed.

To learn more about our recommendations on how to use Azure OpenAI models responsibly, see the Overview of Responsible AI practices for Azure OpenAI models.

 

Getting Started

You can get started with function calling today if you have access to the Azure OpenAI Service. Here are some steps to help you get up and running:

• Apply now for access to Azure OpenAI Service, if you don’t already have access.
• Review the new documentation on function calling.
• Check out our samples on working with function calling.