Blog Post

Azure Integration Services Blog
3 MIN READ

Automated Testing with Logic Apps Standard

hongzli's avatar
hongzli
Icon for Microsoft rankMicrosoft
Nov 13, 2021

Automated testing used to be difficult with Logic Apps, but that’s no longer the case with the addition of Logic Apps Standard. As it is based on the Azure Functions runtime, it can run anywhere, and we can write tests for workflows that can be executed locally or in a CI/CD pipeline. A sample project demonstrating how this can be done is located in our GitHub repo here: https://github.com/Azure/logicapps/tree/master/LogicAppsSampleTestFramework

 

This post will first highlight the capabilities this provides, walk through how to get this sample project up and running, and then dive into the details of the provided sample. 

 

Capabilities provided 

  1. Write automated tests for end-to-end Logic Apps functionality. 
  2. Fine-grained validation at the run and action levels. 
  3. Tests can be checked into a git repo and run either locally or as part of CI/CD pipelines. 
  4. Mocking capabilities for HTTP actions and Azure connectors. 
  5. Configurability that allows tests to use different setting values from production.  

Prerequisites

Running the sample tests 

  1. Open LogicAppsSampleTestFramework.sln in Visual Studio 
  2. Build the solution. 
  3. In the Test Explorer pane, three tests should be present 
  4. Start Azurite
  5. Run the test cases 

 

Anatomy of a test case 

All three sample test cases can be found in TestCases/TestCases.cs. Starting with the first one, RequestResponse, it tests a simple http request-response workflow.  

 

The class doing the heavy lifting is WorkflowTestHost, and we pass it a list of WorkflowTestInputs, which are the individual workflows definitions that we want to run. WorkflowTestHost creates a new temporary Logic Apps project folder, starts the Logic App by invoking Functions Core Tools (func.exe) on that folder, monitors the output to verify that the runtime starts up successfully, and handles cleaning up this temporary folder after the test case finishes. 

 

After the runtime starts up, we can interact with it using REST APIs. We use them to get the trigger callback URL, start the workflow by sending a POST request to that URL, and when the run finishes, we get the run status and action histories for fine-grained validation. 

 

The HttpAction and ApiConnectionAction test cases build upon the first example by adding mocking capabilities. HttpAction, as its name suggests, adds an HTTP action that sends a request to the URL “@appsetting(‘httpuri’)”, which tells the runtime to get the value from the httpuri app setting. To set this app setting in local development, we pass the WorkflowTestHost a localSettings parameter, in which we set the httpuri key-value-pair to a localhost endpoint, and this content is placed in a local.settings.json to be loaded at runtime.  

 

In addition to using WorkflowTestHost, we also construct a MockHttpHost which is used for handling mocked responses. In the HttpAction test case, we set it to return a constant response, but logic can be added to mock more complex behavior. 

 

The ApiConnectionAction test case is similar to the aforementioned HttpAction one, but it mocks the response from a managed connector instead. We pass WorkflowTestHost an additional connectionDetails parameter which gets saved to connections.json and contains details for an ARM API connection. Its connectionRuntimeUrl is set to "@appsetting(arm-connectionRuntimeUrl)" and in localSettings we specify that value to be a localhost endpoint, the same way as we did for the HttpAction test case. Using a MockHttpHost once more, we mock the response for the connector. 

 

We hope this sample provides a helpful starting point for setting up unit tests for your Logic Apps, and as always, we will appreciate any feedback or suggestions that you may have. 

Updated Apr 27, 2022
Version 4.0
  • The test case’s anatomy section is super helpful. Thank you!

  • Wow, this is such an amazing project! I have struggled to setup automation tests with my Logic Apps for so long. This will definitely help me get started, and it looks very simple too. Thank you for the very helpful documentation, Henry! 🙂 Awesome job.

  • SocInABox's avatar
    SocInABox
    Iron Contributor

    Hi hongzli  or anyone,

     

    Who could I talk to about creating a function that matches a simple string from the output of the previous logic app/operator.

     

    I just want to know if the string 'ntDomain' exists.

     

    I've tried this:

    contains('ntDomain','Accounts')
     
    Here's the raw output I'm trying to match on:
     
    {
    "statusCode": 200,
    "headers": {
    "Pragma": "no-cache",
    "Transfer-Encoding": "chunked",
    "Vary": "Accept-Encoding",
    "x-ms-request-id": "xxxxxxxx-c835-4e00-a15f-xxxxxxxxxxxx",
    "Strict-Transport-Security": "max-age=31536000; includeSubDomains",
    "X-Content-Type-Options": "nosniff",
    "X-Frame-Options": "DENY",
    "Cache-Control": "no-store, no-cache",
    "Set-Cookie": "ARRAffinity=xxx;Path=/;HttpOnly;Secure;Domain=azuresentinel-eus2.azconn-eus2.p.azurewebsites.net,ARRAffinitySameSite=xxx;Path=/;HttpOnly;SameSite=None;Secure;Domain=xxx",
    "Timing-Allow-Origin": "*",
    "x-ms-apihub-cached-response": "true",
    "Date": "Sat, 13 Nov 2021 15:22:43 GMT",
    "Content-Type": "application/json; charset=utf-8",
    "Expires": "-1",
    "Content-Length": "185"
    },
    "body": {
    "Accounts": [
    {
    "accountName": "nnnnnnnn",
    "ntDomain": "bob",
    "isDomainJoined": true,
    "displayName": "bob\\12345678",
    "friendlyName": "bob\\12345678",
    "Type": "account",
    "Name": "12345678"
    }
    ]
    }
    }
  • mayurmacwan's avatar
    mayurmacwan
    Copper Contributor

    Super cool. Thanks for sharing. Being able to run this in either local or CI/CD would be a great plus. 

  • StigX's avatar
    StigX
    Copper Contributor

    The concept looks promising but I cannot get the sample project to work successfully. 

     

    I used Visual Studio 2019 16.11.18 and ran Azurite that come with Visual Studio 2022 17.3.2, I could see on Azurite output when running test is non-stop outputting messages, and the unit test eventually failed after 2 mins. Exception from unit test:  

    System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request.. ---> System.Net.Sockets.SocketException: The I/O operation has been aborted because of either a thread exit or an application request

     

    if I set HttpClient with Timeout something like 5 mins, the test still fail after 2 mins but coming with http response code 429 (too many requests).

     

    Could anyone help me what's wrong ?

  • mschellig_1722's avatar
    mschellig_1722
    Copper Contributor

    Hi there,

     

    this looks like an awesome concept for logic app test automation.

    I tried to implement this in my CI/CD Pipelines. Everything was fine with the explaned HTTP triggers.

     

    Nevertheless, I tried to implement a test automation for servicebus triggers as well. The problem I run into is, that using the "When_a_message_is_received_in_a_queue_(auto-complete)" trigger. There are no exceptions at all, just a debug message that says: "Workflow '<workflowname>' does not have a mapped function trigger, skipping function metadata injection." The workflow won't be loaded afterwards.

     

    I guess this trigger is not supported yet? Is there a list of supported triggers or is there anything I could have done wrong?

     

    I am using the newest version of Azure Function Core Tools and the newest version of azurite and so on. Further this is the workflow I used for testing.

    In portal this runs as expected, so I don't see where I could be wrong. 

    {
        "definition": {
            "$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
            "actions": {},
            "contentVersion": "1.0.0.0",
            "outputs": {},
            "triggers": {
                "When_a_message_is_received_in_a_queue_(auto-complete)": {
                    "inputs": {
                        "host": {
                            "connection": {
                                "referenceName": "serviceBus"
                            }
                        },
                        "method": "get",
                        "path": "/@{encodeURIComponent(encodeURIComponent('queuename'))}/messages/head",
                        "queries": {
                            "queueType": "Main"
                        }
                    },
                    "recurrence": {
                        "frequency": "Second",
                        "interval": 15
                    },
                    "type": "ApiConnection"
                }
            }
        },
        "kind": "Stateful"
    }

     

     

    Really would appreciate any help/information about this.

  • marka514's avatar
    marka514
    Copper Contributor

    Hi mschellig_1722 , the automated test framework created by Henry only works with HTTP triggers, this is because these can be easily invoked in an isolated test environment. Any other type of trigger has a hard dependency on a resource or service, an Azure Service Bus Queue in your case. 

     

    Check out LogicAppUnit testing framework, this is based on Henry's work but includes many improvements that make it easier to unit test workflows in an isolated development environment, or as part of a CI/CD pipeline. For example, any non-HTTP triggers are automatically replaced with HTTP triggers to allow the workflow to be triggered. Any actions using Built-In service connections or Managed API connections are automatically replaced with HTTP actions to remove the dependencies. These changes are done so that the actual functionality and behaviour of the workflow is not affected. Check out the readme file in the Git repo for more information.