Azure OpenAI GPT model to review Pull Requests for Azure DevOps

In recent months, the use of Generative Pre-trained Transformer (GPT) models for natural language processing (NLP) has gained significant traction. GPT models, which are based on the Transformer architecture, can generate text from arbitrary sources of input data and can be trained to identify errors and detect anomalies in text. As such, GPT models are increasingly being used for a variety of applications, ranging from natural language understanding to text summarization and question-answering.

In the software development world, developers use pull requests to submit proposed changes to a codebase. However, reviews by other developers can sometimes take a long time and not accurate, and in some cases, these reviews can introduce new bugs and issues. In order to reduce this risk, During my research I found the integration of GPT models is possible and we can add Azure OpenAI service as pull request reviewers for Azure Pipelines service.

The GPT models are trained on developer codebases and are able to detect potential coding issues such as typos, syntax errors, style inconsistencies and code smells. In addition, they can also assess code structure and suggest improvements to the overall code quality.

Once the GPT models have been trained, they can be integrated into the Azure Pipelines service so that they can automatically review pull requests and provide feedback. This helps to reduce the time taken for code reviews, as well as reduce the likelihood of introducing bugs and issues.

How to use it

1. Create Azure OpenAI instance

Azure OpenAI Service provides REST API access to OpenAI's powerful language models including the GPT-3, Codex and Embeddings model series. In addition, the new GPT-4 and ChatGPT (gpt-35-turbo) model series have now reached general availability. These models can be easily adapted to your specific task including but not limited to content generation, summarization, semantic search, and natural language to code translation. Users can access the service through REST APIs, Python SDK, or our web-based interface in the Azure OpenAI Studio.

Use this article to get started making your first calls to Azure OpenAI - How-to - Create a resource and deploy a model using Azure OpenAI Service - Azure OpenAI | Microsoft Learn

 2. Deploy Model

Set up a deployment to make API calls against a provided base model or a custom model in Azure OpenAI Studio.

3. Install the extension

To use the GPT Pull Request Review Task, first install the extension in your Azure DevOps organization. Click on the "Get it free" button and follow the prompts to install it. You may need to authorize the extension to access your Azure DevOps account.

GitHub Repository Link - mlarhrouch/azure-pipeline-gpt-pr-review: Azure DevOps extension adding tools to review Pull Requets using Azure Pipelines (github.com)

Visual Studio Marketplace - GPT Pull Request Review - Visual Studio Marketplace

4. Add the task to the build pipeline

After installing the extension, add the task to your build pipeline. Go to your build pipeline, click on the "+" icon to add a new task, and search for "Review PullRequest by GPT". Select it and add it to your pipeline.

5. Configure the task

Once you have added the task to your pipeline, configure it. In the task configuration, provide your API key for OpenAI API. To create an API key, go to Azure OpenAI instance > Key and Endpoints as shown in below screenshot. 

Notes: These keys are used to access your Cognitive Service API. Do not share your keys. Store them securely– for example, using Azure Key Vault. We also recommend regenerating these keys regularly. Only one key is necessary to make an API call. When regenerating the first key, you can use the second key for continued access to the service.

6. Review Pull Requests

When the build is triggered from a Pull Request, the task will review it. If there is feedback on the changed code, the task will add comments to the Pull Request. If the build is triggered manually, the task will be skipped.

7. Permission Setup

Before using this task, ensure that the build service has permissions to contribute to Pull Requests in your repository, and allow the task to access the system token.

8. Give permission to the build service agent

9. Allow Task to access the system token

Depending on the type of pipeline you are using, follow one of the two steps below:

10. Yaml pipelines

Add a checkout section with persistCredentials set to true.

steps:- checkout: self  persistCredentials: true

11. Classic editors

Enable the option "Allow scripts to access the OAuth token" in the "Agent job" properties.

Overall, the integration of GPT models as pull request reviewers for Azure Pipelines can help to reduce the time taken for code reviews and improve the overall quality of code. This is a great step forward for the development process and can help to make the process of software development much more efficient and secure.

Feel free to provide your feedback and any suggestions advice on this topic.