Integrating Power Apps with Azure Machine Learning & OpenAI using Power Automate

aaryan2134 · ‎Jun 05 2023

About the Author

I am a pre-final year student, currently pursuing B Tech in Computer Engineering at Delhi Technological University. I love to learn new things through exploration and connecting with people along the journey. My key skills include Machine Learning, Web Development, Data Structures & Algorithms. I am also an active open source contributor.

Connect with me

LinkedIn Profile
GitHub Profile

Introduction

Are you interested in harnessing the power of artificial intelligence and machine learning to predict and plan for your health expenses? Are you curious about how Power Apps, Azure Machine Learning, and OpenAI can work together to create innovative solutions? Look no further! In this blog post, we will explore how to integrate Power Apps with Azure Machine Learning and OpenAI using Power Automate, creating a cutting-edge Health Expense Planner Application.

The Project

The Health Expense Planner Application allows users to leverage a machine learning model to predict their health expenses accurately. By inputting various parameters such as age, sex, BMI, number of children, and smoking status, users can generate personalized predictions for their health expenses. But that's not all—the application goes a step further by providing a detailed plan to save funds specifically tailored to the predicted expenses.

The Health Expense Planner Application serves as an excellent illustration of how these technologies can be combined to solve real-world challenges. While the focus is on health expenses in this application, the concepts and techniques discussed can be applied to various domains and use cases.

Features

Predict Health expenses based on various parameters like age, sex, BMI, number of children and whether they are a smoker or not
Uses a custom built Azure Auto ML model integrated to Power Apps using Power Automate Flow
Provides a customized plan to save for health expenses based on inputs and the result of the model using OpenAI's API

Prerequisites

You need to get an API key from OpenAI
You need to have an account for Power Apps & Power Automate
You need an active account in Azure
You need to have a deployed Azure Auto ML Model’s end point URL for predictions

Components

The solution is composed of several key components. First, the Power App itself serves as the interface for users to input their information and receive the predictions and savings plan. The heart of the application lies in the integration of Azure Auto ML which allows us to create powerful, custom Machine Learning models without writing a single line of code using automated machine learning in the Azure Machine Learning studio.

To enable seamless communication between Power Apps and Azure Auto ML, we employ Power Automate Flow. This flow acts as the bridge, passing the user's inputs to the model's endpoint and retrieving the results. Through HTTP requests and JSON parsing, the flow handles the data exchange and ensures a smooth user experience.

In addition to Azure Machine Learning, we leverage the capabilities of OpenAI's API (OpenAI Connector) to enhance the health expense planning process. The application connects to the OpenAI API, utilizing the information to generate a personalized health expense plan. This integration allows users to have a comprehensive understanding of their predicted expenses and offers tailored savings strategies.

Power App

The Health Expense Power App is the main user interface for interacting with the project. It uses some basic UI elements like text labels and input boxes for making the app interactive.

The interesting part in the App though is the predict button. It is responsible for two tasks: calling the AutoML Flow and The OpenAI Connector.

ClearCollect(Result, AzureAutoMLFlow.Run(TextInput1.Text, TextInput1_1.Text, TextInput1_2.Text, TextInput1_3.Text, TextInput1_4.Text));

ClearCollect(explain, 'OpenAI(IndependentPublisher)'.Completion("text-davinci-003", "Use the given information about a person to create a plan to save " & First(Result).response & " USD for their health expenses" & "Their age is " & TextInput1.Text & " they are a " & TextInput1_1.Text & " having " & TextInput1_2.Text & " children.\n", {temperature: 0.3, max_tokens: 100, best_of: 1.0, frequency_penalty: 0.5}).choices);

The ClearCollect function is used to store only the last result obtained. The Azure AutoML Flow is pretty simple. It only passes the input parameters to the Flow which returns the response back and that is stored in Result.

The OpenAI connector uses the Text Completion method. The text-davinci-003 model was used based on the results and documentation provided on the OpenAI website. Finally, a prompt is passed to the connector along with some parameters. The parameters were based on some trial and error. For the prompt, it was created in such a way that the input parameters are also included along with the resultant expenses. They are linked together in a meaningful way for the Large Language Model of OpenAI to process properly.

Power Automate Flow

The flow is called when the Predict button is pressed in the Power App.

It uses a Power Automate Flow to pass the input to the model and get the results back.

The flow uses HTTP to POST the inputs to the Model Endpoint
It then uses Parse JSON to parse the output
Finally sends it back to the Power App

HTTP uses the POST method. We need to provide the deployed endpoint URL of the model along with the input scheme and the input parameters from the Power App. This is then sent as a request to the endpoint using a POST request.

Parse JSON uses the response scheme which can be auto generated by providing a sample response

Next, we pass the results back to the Power App as a response

Azure Automated ML

For this project, we used AutoML to make the ML Model using Low Code in the Azure Machine Learning Studio.

The model creation process was extremely easy and intuitive using AutoML as it provides a GUI where we only need to select the required parameters, set the target variable (in case of regression) and some other basic details like cross validation and metrics.

Instructions to Deploy your own Azure AutoML Model

Go to Azure Machine Learning Studio and create a new Workspace
Select Automated ML and Select +New automated ML job
Follow the instructions on the screen to add dataset
After that, configure the job and forecast settings
Run the experiment
Once training is completed, select the best model from the results
Select Deploy
This process might take a while
Once deploying is completed, go to the deployed model and copy the end point URL
This end point URL can be used directly in the Power Automate Flow to connect the model with Power Apps

The service automatically selects the best model. Then we can deploy the best model or make some changes if required. Once deployed, we get an endpoint to be used in the Power Automate Flow.

We used the Health Expenses Dataset from Kaggle to create the Prediction Model (Regression based). A snapshot of the dataset and the distribution is shown below:

Dataset

OpenAI Connector

We used the OpenAI Independent Publisher Connector to access the API. We need to add the connector by adding it as a Data Source and then creating a connection. To create the connection, we need to provide the API key from the OpenAI Platform as well.

Once the connection is created, the connector allows us to use the API in the Power App like any other function or method.

Results

Let us have a look at few interesting results from the Power App that we just created.

Here, we can see in the first example, the person is a bit old in comparison to the 2nd example (even though their metrics suggest them to be pretty healthy which is accounted in predicted expenses through the Regression Model). Due to them being older, the Proposed Plan is pretty different and takes into account that they need to prepare for health expense at an urgent basis.

The second example is in stark contrast with the person being younger though unhealthy. This leads to an increase in the predicted Health Expenses as expected. Though the interesting difference is that the Proposed Plan takes into account the age factor and provides long term planning strategies!

Note: The OpenAI's API though very powerful comes at a cost of reliability which can be observed by the fact that it confuses the BMI with children and assumes that the person has 26 children!

Resources

Check out the Project Demo Video at Microsoft Power Platform community call – May 2023 - here to see it in action!

You can also have a look at the Power App Sample Pull Request here to get more technical details regarding the Power App Sample.

Products (49)

Special Topics (28)

Video Hub (462)

Most Active Hubs