Run big unattended automation jobs across multiple VMs dynamically at scale, with the new Azure Virtual Desktop integration starter kit. Ashvini Sharma, from the Power Automate engineering team, joins host Jeremy Chapman to share recent updates to Power Automate such as the addition of dynamically, scalable unattended Robotic Process Automation (RPA). Power Automate flows can run in a virtual machine using an automation account, and that VM doesn’t need to be constantly running. It can wake up, run your automation, and then stop on a schedule.
Use the Azure Virtual Desktop integration starter kit to run large-scale automations and scale out the number of virtual machines to distribute your flows across multiple computers. Once the automation is complete, it can shut down your VMs, so you don’t have to pay for idle compute.
QUICK LINKS:
02:08 — How to build and scale out automation
05:00 — Directly parse information to a PDF doc
06:11 — Unattended RPA
08:18 — Scaling strategy with Azure Virtual Desktop
11:00 — Wrap up
Link References:
To learn more, check out https://www.PowerAutomate.com
Watch our previous show on RPA with Power Automate at https://aka.ms/MechanicsRPA
Find the Azure Virtual Desktop integration starter kit at https://aka.ms/AVDPA
Unfamiliar with Microsoft Mechanics?
We are Microsoft’s official video series for IT. You can watch and share valuable content and demos of current and upcoming tech from the people who build it at Microsoft.
- Subscribe to our YouTube: https://www.youtube.com/c/MicrosoftMechanicsSeries?sub_confirmation=1
- Join us on the Microsoft Tech Community: https://techcommunity.microsoft.com/t5/microsoft-mechanics-blog/bg-p/MicrosoftMechanicsBlog
- Watch or listen via podcast here: https://microsoftmechanics.libsyn.com/website
Keep getting this insider knowledge, join us on social:
- Follow us on Twitter: https://twitter.com/MSFTMechanics
- Follow us on LinkedIn: https://www.linkedin.com/company/microsoft-mechanics/
Video Transcript:
- Coming up, we’ll take a look at the recent updates to Power Automate and how you can run big unattended automation jobs across multiple VMs dynamically at scale with the new Azure Virtual Desktop integration starter kit. And to walk us through all of this, I’m joined by Ashvini Sharma from the Power Automate engineering team. Welcome back to Mechanics.
- Thank you, Jeremy, it’s great to be here. I’ve got tons of updates to share.
- And I really can’t wait to see them. Last time you were on, you showed how Power Automate, along with AI, can actually do automated processing for invoices. And if you missed that show, you can check it out at aka.ms/MechanicsRPA. So what’s new in the world of Power Automate?
- Yeah, we’ve seen a huge uptake of Power Automate and how you are using it to automate repetitive tasks. More than 500,000 organizations are using it. There are 15 million bots and 1.5 billion actions that execute on our service every day. And one of the reasons for this momentum is the deep integration we now have across the apps and platforms that you use, including Microsoft Excel to automate data processing or use for personal data store as you read and write data. In Microsoft Teams, popular workflows are available in the store to help you get started quickly. And in Windows 11, Power Automate for desktop is an inbox app that helps you get more done on your machines.
- And it’s really great to see the community rally behind Power Automate and really use it in the context of your work.
- That’s right, and we’ve got a new milestone with the addition of dynamically scalable unattended RPA. What that means is that you no longer have to use a user dedicated device. Your Power Automate flows can run in a virtual machine using an automation account, and that VM doesn’t need to be constantly running. It can wake up, run your automation, and then stop on a schedule. And using Azure Virtual Desktop integration starter kit, you could even run large-scale automations and scale out the number of virtual machines to distribute your flows across multiple computers. And then once the automation is complete, it can shut down your VMs so you don’t have to pay for idle compute. And this really is another important step in our goal to make Power Automate the premier cloud-first automation solution.
- And as a huge automation fan, I’ve got to say this sounds really amazing. But this is Mechanics, so I’d love to see you build out some automation and how you’d scale that out.
- Yeah, that’s right, let’s do it. For this scenario I’m going to use a very common scenario we hear from customers every day, which is take data from Excel and put it into a web form. In this case, the sales orders arrive via email as PDF documents, and currently people are manually entering information into individual Excel spreadsheets. And from there, data is added into a web form. We see this a lot because it’s typically faster to manually enter raw data into Excel, but we can automate these processes to make things more efficient and reduce errors in the process. Because we are continually receiving and entering orders, the automation needs to look for updates a few times per day and enter them into the web form. And that’s where Power Automate comes in. In this case, we are going to be using desktop flows and cloud flows, and let me quickly explain the difference here. Desktop flows are used to automate tasks in the browser or in other apps on your desktop, and cloud flows run online and connect to cloud services, like email or on-premises services, to perform automated tasks. Importantly, both types of flows can work together. Here you see a few of the desktop flows I’ll be showing today. This is the first one that moves data from our Excel spreadsheet to our web-based form. It’s opening the browser and the URL for our form, then populates all the fields in the web forms using the fields mapped from the Excel spreadsheet. Now, if I jump back to the Excel spreadsheet, to avoid duplication, our flow will also write the Excel sheet and change the processed cell from no to yes once the web form entries are completed. Now, if I open Power Automate from the browser, you’ll see I have a cloud flow at the bottom to trigger the data entry. I’ll open it and hit Edit to show you how I’ve set up the trigger to start this flow. In recurrence, I’ve set the flow to run every six hours. Then it runs my desktop flow on the left, called Excel to web form. Now, the run mode is important here. I’ve set it to Attended so we can watch this automation run. To save time, and instead of waiting up to six hours, I’ll manually trigger this operation. And here you see it’s launching the browser. It’s opening our form. It’ll proceed to input all the fields from the Excel file, starting with the order in the first sheet. And notice our four line items are getting entered automatically. Now, because our Excel file has a second sheet, it’s entering that as a second sales order, working through all of the customers and sales order information, and then the single line item. And back in Power Automate, we can see the of flow succeeded. We sped it up a little in the interest of time. It completed in about two minutes. And if I hop back into Excel one more time, you’ll see the same processed cell that we saw earlier is marked as Yes.
- Now this already saves, by the way, a ton of time. And as you saw, the steps and the logic to set all this up are pretty straightforward.
- That’s right, and we can eliminate Excel from the process altogether by directly parsing the information in the PDF document using automation and forms recognition with AI Builder. Let me show you how to get this working. I’m back in Power Automate. Here, I’ll be using a couple of cloud flows chained together, and I’ll explain how each works. Instead of using a time-based schedule every six hours, the process here begins right when the email arrives. It looks for and uploads the attachment for further processing, and that kicks off the second flow in this chain. This one gets everything ready for AI builder, which is a Power Platform capability that gives prebuilt AI models designed to apply intelligence to your business processes. In this case, we’ll use it to recognize our sales order form and parse the information in it. And this predict step is where the AI finds and extracts the data field from the PDF. That continues to the next flow in our chain, which validates the extracted information further. Then if everything checks out, it will hand off the extracted information to the same Power Automate desktop flow we used earlier. And from here, it’ll enter the sales order information and line items, just like we saw in the web form before.
- Okay, so now you’ve got additional automation steps that you’ve put in place, but the big difference here is that this is going to trigger data entry in real-time every time an email comes in. But what happens though, maybe if your Windows PC is powered down or turned off and can’t run that desktop flow?
- That’s right, and this is where unattended RPA comes in. Until now, I’ve been running all of this in the context of my user account on my local PC, but here I’m using my Surface Pro. So it’s portable, it might go offline when I’m not working, but the orders will keep coming in. Remember our run mode here in our cloud flow? I can use the unattended run mode along with an always-on virtual machine to run desktop flows 24/7. I’ll start by connecting this to a VM. Under the ellipse, I’ll add a new connection reference. From the connect dropdown, I’ll select Directly to machine. Now, I’ll choose my VM, RPAVM1. Then I need to add the local account’s username, demoadmin, then the password, there we go. And next, I’ll hit Create. So now, I have the virtual machine configured. And when you do this, it’s a good idea to leave the run mode as attended just to test out the automation and so you can troubleshoot any issues. Once you’ve tested everything and know it’s working, now you can switch the run mode to unattended with this dropdown here. And everything will run by itself in the context of that account. Now you can monitor each run from Power Automate. I’m on the machine monitoring page, and you can see the VMs I have connected here. I configured my desktop flow to run on RPAVM1. So I’ll select it. We still have emails with sales orders coming in and in the run queue tab, we’ll be able to see any orders being processed. To see them in real time, I’ll enable auto-refresh. And again, we’ll speed things up a little, because remember, each run takes about two minutes. Now we can see a few more orders roll in. I have one running and one that’s next to run. And the third order comes in, its status is now queued. Notice, as each run completes, it’s removed from the top of the list. At this point, there’s nothing left in the queue, and to check if everything worked, I can go to the Overview tab and see all of the order entries succeeded.
- Okay, so three for three, not bad. But what happens if your business really takes off, and we know that a single VM can process about 30 orders per hour in our case, and you need more capacity?
- Great question, so in this case, what you really want is elasticity that grows and shrinks the virtual machines based on demand. Now that’s where the real power comes in. We’ve built in dynamic load balancing across a group of machines using Azure Virtual Desktop. What you’re seeing here is the integration starter kit we’ve built for this scenario and you can find it at aka.ms/AVDPA. Once you have an Azure Virtual Desktop environment running with multiple virtual machines, you can use it with this Power App. And here’s where things get really interesting with scaling strategy. This environment can dynamically scale out and scale back in your VM count based on the number of items in the Power Automate queue. Here, the scaling orchestration is set to run every 15 minutes. The VM start threshold is set to one in the queue for our demo, but you’ll want to increase this threshold depending on how long your flows run on a single VM to balance your costs against performance. The Always on VM, like it sounds, says how many VMs are always warm and ready to go. Then the shutdown method for scaling back in can be deallocated or stopped. A deallocated VM will take longer to start back up compared to a stopped VM, but you won’t be charged if it’s deallocated. And that’s it, now we can go back to our flow and connect to this machine group so that all our automations can scale. Here, I’ll click on the ellipses again and go to Create a connection reference. I’ll choose to connect directly to the machine again, but this time I’ll choose the group called AVDMachineGroupForSales, which in my case has two VMs in it. Again, I need to add the local account for our automation to use in order to log into those Vms, and the password. Once I click Create, the group is ready to start processing orders. This time, you’ll see quite a few more come in. We’ll let this run for a moment. And in this case, we’ll process 10 orders. You can see there are eight concurrent requests now. They get distributed automatically between the two machines in our group, and these will start to pare down as our jobs run. In real time, this takes about 10 minutes to run all the 10 jobs across two VMs. Once it’s complete, you’ll see there are no more runs in our queue. And again, I can see how everything went in the Overview tab. Everything succeeded, and we can go to See all runs, which shows that each machine processed an equal number of sales orders. So the load was distributed across both VMs. And, of course, we could scale this to dozens of VMs if we needed to.
- That’s what I like to see. This is next generation automation infrastructure. It’s elastic. It’s going to efficiently scale to whatever demand hits it, and it can scale both out and in dynamically, all using the familiar Azure Virtual Desktop infrastructure. So what would you recommend for all the folks that are watching right now who want to get started with this?
- Yeah, the great news is everything I’ve shown today is available to try out right now. To learn more, check out PowerAutomate.com, and you’ll find the Azure Virtual Desktop integration starter kit that we showed you earlier at aka.ms/AVDPA.
- Thanks so much, Ashvini, for joining us today and sharing all the increased power in Power Automate. And of course, keep checking back to Microsoft Mechanics for all the latest tech updates. Subscribe, if you haven’t already, to our channel, and thank you so much for watching.
