Integrating AI: Prototyping a No-Code solution with Power Apps
Published Mar 09 2021 04:50 AM 11K Views
Microsoft

You might have the cutting edge AI features but it is hard to know how useful it will be before letting your users beta test your prototype. You can build fast, deploy and deliver your app and iterate without writing any code, using AI Builder and Power Platform.

 

This article explains what Power Platform is, as well as go through a step by step process to create an application that detects objects from photos using Power Apps and AI Builder. Check out the video below to see the app we will build to detect different Mixed Reality Headsets such as HoloLens version 1 and 2 Augmented Reality and Virtual Reality headsets and their hand controllers.

 

 

Yonet_0-1615005261042.gif

 

 

 

What is Power Platform?

 

Power Platform is a set of tools, API's and SDK's that helps you analyze your data and build automations, applications and virtual agents with or without having to write any code.

 

powerPlatform.png

 

 

What are Power Apps?

 

Power Apps, allows you to create applications with a drag and drop UI and easy integration of your data and 3rd party APIs through connectors.

 

A connector is a proxy or a wrapper around an API that allows the underlying service to talk to Microsoft Power Automate, Microsoft Power Apps, and Azure Logic Apps. It provides a way for users to connect their accounts and leverage a set of pre-built actions and triggers to build their apps and workflows. For example, you can use the Twitter connector to get tweet data and visualize it in a dashboard or use the Twilio connector to send your users text messages without having to be an expert in Twitter or Twilio APIs or having to write a line of code.

 

Check out the list of connectors for Power Apps to see all the APIs that are available. Notice Power Automate or Logic App connectors might not be the same.

 

 

What is AI Builder?

 

AI Builder is one of the additional features of Power Apps. With AI Builder, you can add intelligence to your apps even if you have no coding or data science skills.

 

aiBuilderAppView.png

 

 

 

 

Is AI Builder the right choice?

 

 

Can I use Power Apps and AI Builder for production?

 

Yes you can. As any tool that does things magically, AI Builder in Power Apps comes with a cost. That does not mean you can't try your ideas out for free.

 

 

What will my production app cost?

 

If you want to go to production with Power Apps, it is a good idea to consider the costs. Thankfully there is an app for that. AI Builder Calculator lets you input what AI tools you will need and how many users will be accessing your app's AI features and gives you the price it will cost you.

 

aiBuilderCalculate.png

 

What are preview features?

 

 

AIBuilderPreview.png

 

What is Object Detection?

 

AI Builder Object detection is an AI model that you can train to detect objects in pictures. AI models usually require that you provide samples of data to train before you are able to perform predictions. Prebuilt models are pre-trained by using a set of samples that are provided by Microsoft, so they are instantly ready to be used in predictions.

 

testResultSmall.gif

 

 Object detection can detect up to 500 different objects in a single model and support JPG, PNG, BMP image format or photos through the Power Apps control.

 

How to try out Object Detection capabilities?

 

You can try out and see how object detection works before having to create and accounts or apps yourself on the Azure Computer Vision page.

 

seeItinAction.png

 

 

What can you do with Object Detection?

 

  • Object counting and inventory management

  • Brand logo recognition

  • Wildlife animal recognition

 

How to detect objects from images?

 

  • To start creating your AI model for your app, sign in to Power Apps and click on AI Builder on the left hand menu. Select Object Detection from the "Refine Model for your business needs" option.

 

buildAI.png

 

 

  •  Name your new AI model with a unique name. Select Common Objects and proceed to next section.

 

commonObj.png

 

 

  • Name the objects that you are going to detect.

 

 

namedObjects.png

 

  • Upload images that contain the object you will detect. To start with you can upload 15 images for each object.

 

imageDetectionFormat.png

 

 

  • Make sure each object has approximately the same amount of images tagged. If you have more examples of one object, the training data will be likely to detect that object when it is not.
  • Tag your objects by selecting a square that your object is in and choosing the name of the object.

 

 

tagging.png

 

 

  • Once you are done, choose Done Tagging and Train. Training process will take some time.
  • If you choose to not use an image or clear any tags, you can do that at any time by going back to your model under the AI Builder on the left hand side menu and choose your model and choose edit.

 

dontUseImage.png

 

 

  • AI Builder will give you a Performance score over 100 and a way to quickly test your model before publishing. You can edit your models and retrain to improve your performance. Next section will give you some best practices to improve your performance.

 

performance.png

 

How to Improve Your Custom Model Performance?

 

Co-Authors
Version history
Last update:
‎Mar 15 2021 02:14 PM
Updated by: