In the first two posts of this series we have developed an Outlook add-in that, using the Open AI models, helps users to generate professional business mails starting from a simple sentence. In the first post, we achieved this goal by directly using the OpenAI APIs; in the second one, instead, we have leveraged the Azure OpenAI service. In both cases, however, we didn't use the model that powers up ChatGPT, because it wasn't available yet at the time of writing, but we leveraged a similar one called Davinci.
Since a few days, both OpenAI and Azure OpenAI have finally made available the ChatGPT model, so that developers can integrate it into their experiences. In the previous posts, we said that the when this would have happened, we wouldn't have needed to change much code, probably just the model's name. This turned out to be partially true: the overall architecture of the add-in is unchanged, but the way you provide the prompt to the ChatGPT model is slightly different than the way you do it with the Davinci one. The different approach comes from the fact that, while Davinci was created mainly for content generation scenarios, ChatGPT targets specifically conversational ones. As such, you must be able to send entire conversations to the API, so that it can gather the required context to handle a conversation without having to specify all the information at each prompt.
In this post, we're going to learn the changes we must make based on the service we're using.
Using ChatGPT with the OpenAI APIs
I've already anticipated this a few days ago, when I updated the original post, but I wanted to consolidate everything in a single article, so let's review the required changes when you want to directly use the OpenAI platform.
OpenAI released the ChatGPT model as gpt-3.5-turbo and the good news is that it's 10 times cheaper than the Davinci one. The model, in fact, is priced $0.002 / 1K tokens, while the Davinci one we have used in the original version of the project is priced $0.0200 / 1K tokens.
Before starting to make the changes which are required to use this model, we must open a terminal on the folder which contains our project and upgrade the OpenAI library with the following command:
npm update openai
This command will install version 3.1.0 of the library, which includes a new API to interact with the ChatGPT model. Now let's take a look at the updated generateText() function which uses this new API:
generateText = async () => {
var current = this;
const configuration = new Configuration({
apiKey: "your-API-key",
});
const openai = new OpenAIApi(configuration);
const response = await openai.createChatCompletion({
model: "gpt-3.5-turbo",
messages: [
{
role: "system",
content: "You are a helpful assistant that can help users to create professional business content.",
},
{ role: "user", content: "Turn the following text into a professional business mail: " + this.state.startText },
],
});
current.setState({ generatedText: response.data.choices[0].message.content });
};
There are three main differences compared to the previous implementation. The first one is that we have a new API to interact with ChatGPT, called createChatCompletion(), which we must use instead of createCompletion(). The second difference, even if it's a small one, is that we have to change the model name to gpt-3.5-turbo. The third difference is the most important one. To support the conversational scenario, the prompt you pass to the model to generate text isn't based any more on a single sentence, but on on a collection of messages object. Each of them has two properties:
role, which is used to define who generated the prompt. It could be the user, the system or the assistant itself.content, which is the actual text.
In the code sample, you can see two types of messages: one with system as role, which we're using to instruct the model on the type of outcome we want to achieve (you are an assistant that can generate business content); one with user as role, which is instead the actual ask (turn the following text into a business mail).
This scenario might not be the best one to explain this approach, since we're using ChatGPT to generate content and not to handle a conversation. Let's see another example taken from the documentation, which should be more helpful to better understand how it works.
const response = await openai.createChatCompletion({
model: "gpt-3.5-turbo",
messages: [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Who won the world series in 2020?"},
{"role": "assistant", "content": "The Los Angeles Dodgers won the World Series in 2020."},
{"role": "user", "content": "Where was it played?"}
],
});
As we mentioned, ChatGPT is born to support conversational scenarios, however the conversation history isn't managed by OpenAI, but it must be managed by the developer. This means that we must use the messages collection to provide the whole history, including the messages generated by ChatGPT using assistant as a role. This way, the model is able to provide an answer to the last question (Where was it played?) even if it doesn't include the full context.
However, even if we aren't in a conversational scenario, ChatGPT is very powerful and works great for content creation. Additionally, the ChatGPT model is much cheaper than the Davinci one, so it's great fit for our Outlook add-in.
Using ChatGTP APIs with Azure Open AI
Since a few days Microsoft has announced the public preview of ChatGPT inside Azure Open AI. This means that, if you have been approved to use the Azure Open AI service (see my previous post for more details on the approval process), now you can head to the Model deployments section and, in the dropdown, you will see a new option called gpt-35-turbo.
Please note: make sure that your service instance is deployed in East US. At the time of writing, this is the only region where the ChatGPT model is available.
From a code perspective, working with Azure OpenAI requires less changes compared to the OpenAI APIs, since the format is the same. You must submit a HTTP POST request to the openai/deployments/{deploymentName}/completions?api-version=2022-12-01 URL, prefixed by the URL of your instance. Inside the HTTP request, you must provide the API key as header and the payload with the prompt you want to submit as body. The main difference is the way you create the prompt. Azure OpenAI follows the same conversational approach we have seen with OpenAI but, instead of using a collection of messages, it uses a single message with special placeholders to define the history. Let's look at the updated version of the generateText() function:
generateText = async () => {
var current = this;
const apiKey = "your-api-key";
const endpoint = "your-url";
const prompt =
"<|im_start|>system\nThe system is an AI assistant that helps people to write professional business mails.\n<|im_end|>\n<|im_start|>user\nTurn the following text into a professional business mail: " +
this.state.startText +
"\n<|im_end|>\n<|im_start|>assistant";
const deploymentName = "your-deployment-name";
const url = endpoint + "openai/deployments/" + deploymentName + "/completions?api-version=2022-12-01";
const payload = {
prompt: prompt,
max_tokens: 1000,
};
var response = await fetch(url, {
method: "POST",
headers: {
"Content-Type": "application/json",
"api-key": apiKey,
},
body: JSON.stringify(payload),
});
var data = await response.json();
current.setState({ generatedText: data.choices[0].text });
};
As you can see, the code to perform the HTTP request is the same, including the way we handle the response. However, the prompt property of the payload combines multiple prompts, which are delimited by the <|im_start|> and <|im_end|> placeholders. After the <|im_start|> placeholder, before including the actual prompt, we specify who generated it, using keywords like system, user or assistant. The prompt ends with an open sentence: the <|im_start|> placeholder, followed by the assistant role. By not closing it, we tell Azure Open AI that we want the AI model to take the input we have provided and use it to generate something for us. If you want to know more details on how to work with this model, you can read this good article from the official documentation.
The downside of this approach is that it make slightly more complicated the prompt generation compared to Open AI. The good news, however, is that you won't have to change much your code to use the ChatGPT model, while Open AI requires more work on the code.
Wrapping up
The advent of ChatGPT is a game changer in the AI ecosystem, since it opens new and exciting scenarios to change the way we work, and we create content. Thanks to OpenAI and Azure OpenAI, now we can bring the power of this model right into our applications. And if we have already built experiences using the Davinci model, like our business mail generator, moving to ChatGPT is quite simple, as we have learned in this post.
As usual, you can find the source code of the project on GitHub.
Microsoft
