Stateful serverless automation with PowerShell support in Azure Durable Functions

Published 05-26-2021 09:00 AM 6,825 Views
Microsoft

This week at Microsoft’s annual Build conference, we made two announcements related to Azure Durable Functions: Two new backend storage providers, and the General Availability of Durable Functions for PowerShell. In this post, we’ll go into more details about the new capabilities that Durable Functions brings to PowerShell developers. 

 

Stateful workflows with Durable Functions

 

Durable Functions is an extension to Azure Functions that lets you write stateful workflows in a serverless compute environment.

 

Using a special type of function called an orchestrator function, you can write PowerShell code to describe a stateful workflow that orchestrates other PowerShell Azure Functions that perform activities in the workflow. Using familiar PowerShell language constructs such as loops and conditionals, your orchestrator function can execute complex workflows that consist of activity functions running in sequence and/or concurrently. An orchestration can be started by any Azure Functions trigger. Additionally, it can wait for timers or external input and handle errors using try/catch statements.

 

Some patterns supported by Durable FunctionsSome patterns supported by Durable Functions

 

Uses for Durable Functions in PowerShell

 

With a large ecosystem of modules, PowerShell Azure Functions are extremely popular in automation workloads. Many modules integrate with managed identity—making PowerShell Azure Functions especially useful for managing Azure resources and calling the Microsoft Graph. Durable Functions allows you to extend Azure Functions’ capabilities by composing multiple PowerShell Azure Functions together to perform complex automation workflow scenarios.

 

Here are some examples of what you can achieve with Durable Functions and PowerShell.

 

Automate resource provisioning and application deployment

 

PowerShell Azure Functions are commonly used to perform automation of Azure resources. This can include provisioning and populating resources like Storage accounts and starting and stopping virtual machines. Often, these operations can extend beyond the 10-minute maximum duration supported by Azure Functions in the Consumption plan.

 

Using Durable Functions, you can decompose your sequential workflow into a Durable Functions orchestration that consists of multiple shorter functions. The orchestration can last for hours or longer, and you write it in PowerShell. It can include logic for retries and custom error handling. In addition, Durable Functions automatically checkpoints your progress so if your orchestration is interrupted for any reason, it can automatically restart and pick up where it left off.

 

param($Context)

$Group = Invoke-ActivityFunction -FunctionName 'CreateResourceGroup'
$VM = Invoke-ActivityFunction -FunctionName 'CreateVirtualMachine' -Input $Group

do {
    $ExpiryTime = New-TimeSpan -Seconds 10
    $TimerTask = Start-DurableTimer -Duration $ExpiryTime
    $VMStatus = Invoke-ActivityFunction -FunctionName 'CreateVirtualMachine' -Input $VM
}
until ($VMStatus -eq 'started')

Invoke-ActivityFunction -FunctionName 'DeployApplication' -Input $VM
Invoke-ActivityFunction -FunctionName 'RunJob' -Input $VM
Invoke-ActivityFunction -FunctionName 'DeleteResourceGroup' -Input $Group

 

Orchestrate parallel processing

 

Durable Functions makes it simple to implement fan-out/fan-in. Many workflows have steps that can be run concurrently. You can write an orchestration that fans out processing to many activity functions. Using the power of the Cloud, Durable Functions automatically schedules the functions to run on many different machines in parallel, and it allows your orchestrator to wait for all the functions to complete and access their results.

 

param($Context)

# Get a list of work items to process in parallel.
$WorkBatch = Invoke-ActivityFunction -FunctionName 'GetWorkItems'

# Fan out
$ParallelTasks =
    foreach ($WorkItem in $WorkBatch) {
        Invoke-ActivityFunction -FunctionName 'ProcessItem' -Input $WorkItem -NoWait
    }
$Outputs = Wait-ActivityFunction -Task $ParallelTasks

# Fan in
Invoke-ActivityFunction -FunctionName 'AggregateResults' -Input $Outputs

 

Audit Azure resource security

 

Any of Azure Functions' triggers can start Durable Functions orchestrations. Many events that can occur in an Azure subscription, such as the creation of resource groups and Azure resources, are published to Azure Event Grid. Using the Event Grid trigger, you can listen for resource creation events and kick off a Durable Functions orchestration to perform checks to ensure permissions are correctly set on each created resource and automatically apply role assignments, add tags, and send notifications.

 

Create an Azure Event Grid subscription that invokes a PowerShell Durable FunctionCreate an Azure Event Grid subscription that invokes a PowerShell Durable Function

 

Try PowerShell Durable Functions

 

PowerShell Durable Functions are generally available and you can learn more about them by reading the documentation or by trying the quickstart.

 

 

2 Comments
Senior Member

Do not know about you, but I think it is AWESOME!!!!!!!!!!!!! PowerShell FTW!!!

 

Happy Azure Stacking!!!

New Contributor

Yes!!!

%3CLINGO-SUB%20id%3D%22lingo-sub-2384942%22%20slang%3D%22en-US%22%3EStateful%20serverless%20automation%20with%20PowerShell%20support%20in%20Azure%20Durable%20Functions%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2384942%22%20slang%3D%22en-US%22%3E%3CP%20data-unlink%3D%22true%22%3EThis%20week%20at%20Microsoft%E2%80%99s%20annual%26nbsp%3BBuild%26nbsp%3Bconference%2C%20we%20made%20two%20announcements%26nbsp%3Brelated%20to%26nbsp%3BAzure%20Durable%20Functions%3A%20%3CA%20href%3D%22https%3A%2F%2Fazure.microsoft.com%2Fen-us%2Fupdates%2Fpublic-preview-introducing-new-durable-functions-storage-provider-options%2F%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3ETwo%20new%20backend%20storage%20providers%3C%2FA%3E%2C%20and%20the%20%3CA%20href%3D%22https%3A%2F%2Fazure.microsoft.com%2Fen-us%2Fupdates%2Fpowershell-support-in-durable-functions-is-now-generally-available%2F%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3EGeneral%20Availability%20of%20Durable%20Functions%20for%20PowerShell%3C%2FA%3E.%26nbsp%3BIn%20this%20post%2C%20we%E2%80%99ll%20go%20into%20more%20details%26nbsp%3Babout%20the%20new%20capabilities%20that%20Durable%20Functions%20brings%20to%20PowerShell%20developers.%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId--464075734%22%20id%3D%22toc-hId--464075734%22%20id%3D%22toc-hId--464075734%22%20id%3D%22toc-hId--464075734%22%20id%3D%22toc-hId--464188294%22%3EStateful%20workflows%20with%20Durable%20Functions%3C%2FH2%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fazure%2Fazure-functions%2Fdurable%2Fdurable-functions-overview%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3EDurable%20Functions%3C%2FA%3E%20is%20an%20extension%20to%20Azure%20Functions%20that%20lets%20you%20write%20stateful%20workflows%20in%20a%20serverless%20compute%20environment.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EUsing%20a%20special%20type%20of%20function%20called%20an%20orchestrator%20function%2C%20you%20can%20write%20PowerShell%20code%20to%20describe%20a%20stateful%20workflow%20that%20orchestrates%20other%20PowerShell%20Azure%20Functions%20that%20perform%20activities%20in%20the%20workflow.%20Using%20familiar%20PowerShell%20language%20constructs%20such%20as%20loops%20and%20conditionals%2C%20your%20orchestrator%20function%20can%20execute%20complex%20workflows%20that%20consist%20of%20activity%20functions%20running%20in%20sequence%20and%2For%20concurrently.%20An%20orchestration%20can%20be%20started%20by%20any%20Azure%20Functions%20trigger.%20Additionally%2C%20it%20can%20wait%20for%20timers%20or%20external%20input%20and%20handle%20errors%20using%20try%2Fcatch%20statements.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22powershell-durable-patterns.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F283645iFF407721D25C0AC4%2Fimage-size%2Flarge%3Fv%3Dv2%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22powershell-durable-patterns.png%22%20alt%3D%22Some%20patterns%20supported%20by%20Durable%20Functions%22%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-caption%22%20onclick%3D%22event.preventDefault()%3B%22%3ESome%20patterns%20supported%20by%20Durable%20Functions%3C%2FSPAN%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId-2023437099%22%20id%3D%22toc-hId-2023437099%22%20id%3D%22toc-hId-2023437099%22%20id%3D%22toc-hId-2023437099%22%20id%3D%22toc-hId-2023324539%22%3EUses%20for%20Durable%20Functions%20in%20PowerShell%3C%2FH2%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EWith%20a%20large%20ecosystem%20of%20modules%2C%20PowerShell%20Azure%20Functions%20are%20extremely%20popular%20in%20automation%20workloads.%20Many%20modules%20integrate%20with%20managed%20identity%E2%80%94making%20PowerShell%20Azure%20Functions%20especially%20useful%20for%20managing%20Azure%20resources%20and%20calling%20the%20Microsoft%20Graph.%20Durable%20Functions%20allows%20you%20to%20extend%20Azure%20Functions%E2%80%99%20capabilities%20by%20composing%20multiple%20PowerShell%20Azure%20Functions%20together%20to%20perform%20complex%20automation%20workflow%20scenarios.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EHere%20are%20some%20examples%20of%20what%20you%20can%20achieve%20with%20Durable%20Functions%20and%20PowerShell.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH3%20id%3D%22toc-hId--1580968723%22%20id%3D%22toc-hId--1580968723%22%20id%3D%22toc-hId--1580968723%22%20id%3D%22toc-hId--1580968723%22%20id%3D%22toc-hId--1581081283%22%3EAutomate%20resource%20provisioning%20and%20application%20deployment%3C%2FH3%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EPowerShell%20Azure%20Functions%20are%20commonly%20used%20to%20perform%20automation%20of%20Azure%20resources.%20This%20can%20include%20provisioning%20and%20populating%20resources%20like%20Storage%20accounts%20and%20starting%20and%20stopping%20virtual%20machines.%20Often%2C%20these%20operations%20can%20extend%20beyond%20the%2010-minute%20maximum%20duration%20supported%20by%20Azure%20Functions%20in%20the%20Consumption%20plan.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EUsing%20Durable%20Functions%2C%20you%20can%20decompose%20your%20sequential%20workflow%20into%20a%20Durable%20Functions%20orchestration%20that%20consists%20of%20multiple%20shorter%20functions.%20The%20orchestration%20can%20last%20for%20hours%20or%20longer%2C%20and%20you%20write%20it%20in%20PowerShell.%20It%20can%20include%20logic%20for%20retries%20and%20custom%20error%20handling.%20In%20addition%2C%20Durable%20Functions%20automatically%20checkpoints%20your%20progress%20so%20if%20your%20orchestration%20is%20interrupted%20for%20any%20reason%2C%20it%20can%20automatically%20restart%20and%20pick%20up%20where%20it%20left%20off.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-powershell%22%3E%3CCODE%3Eparam(%24Context)%0A%0A%24Group%20%3D%20Invoke-ActivityFunction%20-FunctionName%20'CreateResourceGroup'%0A%24VM%20%3D%20Invoke-ActivityFunction%20-FunctionName%20'CreateVirtualMachine'%20-Input%20%24Group%0A%0Ado%20%7B%0A%26nbsp%3B%26nbsp%3B%26nbsp%3B%20%24ExpiryTime%20%3D%20New-TimeSpan%20-Seconds%2010%0A%26nbsp%3B%26nbsp%3B%26nbsp%3B%20%24TimerTask%20%3D%20Start-DurableTimer%20-Duration%20%24ExpiryTime%0A%26nbsp%3B%26nbsp%3B%26nbsp%3B%20%24VMStatus%20%3D%20Invoke-ActivityFunction%20-FunctionName%20'CreateVirtualMachine'%20-Input%20%24VM%0A%7D%0Auntil%20(%24VMStatus%20-eq%20'started')%0A%0AInvoke-ActivityFunction%20-FunctionName%20'DeployApplication'%20-Input%20%24VM%0AInvoke-ActivityFunction%20-FunctionName%20'RunJob'%20-Input%20%24VM%0AInvoke-ActivityFunction%20-FunctionName%20'DeleteResourceGroup'%20-Input%20%24Group%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH3%20id%3D%22toc-hId-906544110%22%20id%3D%22toc-hId-906544110%22%20id%3D%22toc-hId-906544110%22%20id%3D%22toc-hId-906544110%22%20id%3D%22toc-hId-906431550%22%3EOrchestrate%20parallel%20processing%3C%2FH3%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EDurable%20Functions%20makes%20it%20simple%20to%20implement%20fan-out%2Ffan-in.%20Many%20workflows%20have%20steps%20that%20can%20be%20run%20concurrently.%20You%20can%20write%20an%20orchestration%20that%20fans%20out%20processing%20to%20many%20activity%20functions.%20Using%20the%20power%20of%20the%20Cloud%2C%20Durable%20Functions%20automatically%20schedules%20the%20functions%20to%20run%20on%20many%20different%20machines%20in%20parallel%2C%20and%20it%20allows%20your%20orchestrator%20to%20wait%20for%20all%20the%20functions%20to%20complete%20and%20access%20their%20results.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-powershell%22%3E%3CCODE%3Eparam(%24Context)%0A%0A%23%20Get%20a%20list%20of%20work%20items%20to%20process%20in%20parallel.%0A%24WorkBatch%20%3D%20Invoke-ActivityFunction%20-FunctionName%20'GetWorkItems'%0A%0A%23%20Fan%20out%0A%24ParallelTasks%20%3D%0A%26nbsp%3B%26nbsp%3B%26nbsp%3B%20foreach%20(%24WorkItem%20in%20%24WorkBatch)%20%7B%0A%26nbsp%3B%26nbsp%3B%26nbsp%3B%26nbsp%3B%26nbsp%3B%26nbsp%3B%26nbsp%3B%20Invoke-ActivityFunction%20-FunctionName%20'ProcessItem'%20-Input%20%24WorkItem%20-NoWait%0A%26nbsp%3B%26nbsp%3B%26nbsp%3B%20%7D%0A%24Outputs%20%3D%20Wait-ActivityFunction%20-Task%20%24ParallelTasks%0A%0A%23%20Fan%20in%0AInvoke-ActivityFunction%20-FunctionName%20'AggregateResults'%20-Input%20%24Outputs%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH3%20id%3D%22toc-hId--900910353%22%20id%3D%22toc-hId--900910353%22%20id%3D%22toc-hId--900910353%22%20id%3D%22toc-hId--900910353%22%20id%3D%22toc-hId--901022913%22%3EAudit%20Azure%20resource%20security%3C%2FH3%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EAny%20of%20Azure%20Functions'%20triggers%20can%20start%20Durable%20Functions%20orchestrations.%20Many%20events%20that%20can%20occur%20in%20an%20Azure%20subscription%2C%20such%20as%20the%20creation%20of%20resource%20groups%20and%20Azure%20resources%2C%20are%20published%20to%20Azure%20Event%20Grid.%20Using%20the%20Event%20Grid%20trigger%2C%20you%20can%20listen%20for%20resource%20creation%20events%20and%20kick%20off%20a%20Durable%20Functions%20orchestration%20to%20perform%20checks%20to%20ensure%20permissions%20are%20correctly%20set%20on%20each%20created%20resource%20and%20automatically%20apply%20role%20assignments%2C%20add%20tags%2C%20and%20send%20notifications.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22create-event-grid-subscription.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F283644i2ABD4E02A0181EA8%2Fimage-size%2Flarge%3Fv%3Dv2%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22create-event-grid-subscription.png%22%20alt%3D%22Create%20an%20Azure%20Event%20Grid%20subscription%20that%20invokes%20a%20PowerShell%20Durable%20Function%22%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-caption%22%20onclick%3D%22event.preventDefault()%3B%22%3ECreate%20an%20Azure%20Event%20Grid%20subscription%20that%20invokes%20a%20PowerShell%20Durable%20Function%3C%2FSPAN%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId--911413457%22%20id%3D%22toc-hId--911413457%22%20id%3D%22toc-hId--911413457%22%20id%3D%22toc-hId--911413457%22%20id%3D%22toc-hId--911526017%22%3ETry%20PowerShell%20Durable%20Functions%3C%2FH2%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EPowerShell%20Durable%20Functions%20are%20generally%20available%20and%20you%20can%20learn%20more%20about%20them%20by%20reading%20the%20%3CA%20href%3D%22https%3A%2F%2Faka.ms%2Fdurable-powershell-docs%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3Edocumentation%3C%2FA%3E%20or%20by%20trying%20the%20%3CA%20href%3D%22https%3A%2F%2Faka.ms%2Fdurable-powershell-quickstart%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3Equickstart%3C%2FA%3E.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-2384942%22%20slang%3D%22en-US%22%3E%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22functions-dark-blue-cover.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F283641i8F9B37BE34E2DE0E%2Fimage-size%2Flarge%3Fv%3Dv2%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22functions-dark-blue-cover.png%22%20alt%3D%22functions-dark-blue-cover.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EPowerShell%20support%20in%20Durable%20Functions%20is%20now%20generally%20available!%3C%2FP%3E%3C%2FLINGO-TEASER%3E%3CLINGO-LABS%20id%3D%22lingo-labs-2384942%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EAzure%20Functions%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2390312%22%20slang%3D%22en-US%22%3ERe%3A%20Stateful%20serverless%20automation%20with%20PowerShell%20support%20in%20Azure%20Durable%20Functions%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2390312%22%20slang%3D%22en-US%22%3E%3CP%3EDo%20not%20know%20about%20you%2C%20but%20I%20think%20it%20is%20AWESOME!!!!!!!!!!!!!%20PowerShell%20FTW!!!%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EHappy%20Azure%20Stacking!!!%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2394377%22%20slang%3D%22en-US%22%3ERe%3A%20Stateful%20serverless%20automation%20with%20PowerShell%20support%20in%20Azure%20Durable%20Functions%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2394377%22%20slang%3D%22en-US%22%3E%3CP%3EYes!!!%3C%2FP%3E%3C%2FLINGO-BODY%3E
Co-Authors
Version history
Last update:
‎May 25 2021 11:33 AM
Updated by: