Synthetic Monitoring in Application Insights Using Playwright: A Game-Changer

Monitoring the availability and performance of web applications is crucial to ensuring a seamless user experience. Azure Application Insights provides powerful synthetic monitoring capabilities to help detect issues proactively. However, Microsoft has deprecated two key features:

• (Deprecated) Multi-step web tests: Previously, these allowed developers to record and replay a sequence of web requests to test complex workflows. They were created in Visual Studio Enterprise and uploaded to the portal.
• (Deprecated) URL ping tests: These tests checked if an endpoint was responding and measured performance. They allowed setting custom success criteria, dependent request parsing, and retries.

With these features being phased out, we are left without built-in logic to test application health beyond simple endpoint checks. The solution? Custom TrackAvailability tests using Playwright.

What is Playwright?

Playwright is a powerful end-to-end testing framework that enables automated browser testing for modern web applications. It supports multiple browsers (Chromium, Firefox, WebKit) and can run tests in headless mode, making it ideal for synthetic monitoring.

Why Use Playwright for Synthetic Monitoring?

• Simulate real user interactions (login, navigate, click, etc.)
• Catch UI failures that simple URL ping tests cannot detect
• Execute complex workflows like authentication and transactions
• Integrate with Azure Functions for periodic execution
• Log availability metrics in Application Insights for better tracking and alerting

Step-by-Step Implementation (Repo link)

1. Set Up an Azure Function App

• • Navigate to the Azure Portal.
  • Create a new Function App.
  • Select Runtime Stack: Node.js.
  • Enable Application Insights.

2. Install Dependencies

        In your local development environment, create a Node.js project:

mkdir playwright-monitoring && cd playwright-monitoring
npm init -y
npm install /functions playwright applicationinsights dotenv

3. Implement the Timer-Triggered Azure Function

         Create timerTrigger1.js:

const { app } = require('@azure/functions');
const { runPlaywrightTests } = require('../playwrightTest.js'); // Import the Playwright test function

app.timer('timerTrigger1', {
    schedule: '0 */5 * * * *', // Runs every 5 minutes
    handler: async (myTimer, context) => {
        try {
            context.log("Executing Playwright test...");
            await runPlaywrightTests(context);
            context.log("Playwright test executed successfully!");
        } catch (error) {
            context.log.error("Error executing Playwright test:", error);
        } finally {
            context.log("Timer function processed request.");
        }
    }
});

4. Implement the Playwright Test Logic

     Create playwrightTest.js:

require('dotenv').config();
const playwright = require('playwright');
const appInsights = require('applicationinsights');

// Debugging: Print env variable to check if it's loaded correctly
console.log("App Insights Key:", process.env.APPLICATIONINSIGHTS_CONNECTION_STRING);

// Initialize Application Insights
appInsights
  .setup(process.env.APPLICATIONINSIGHTS_CONNECTION_STRING || process.env.APPINSIGHTS_INSTRUMENTATIONKEY)
  .setSendLiveMetrics(true)
  .setDistributedTracingMode(appInsights.DistributedTracingModes.AI_AND_W3C)
  .setAutoDependencyCorrelation(true)
  .setAutoCollectRequests(true)
  .setAutoCollectPerformance(true)
  .setAutoCollectExceptions(true)
  .setAutoCollectDependencies(true)
  .setAutoCollectConsole(true)
  .setUseDiskRetryCaching(true) // Enables retry caching for telemetry
  .setInternalLogging(true, true) // Enables internal logging for debugging
  .start();
const client = appInsights.defaultClient;
async function runPlaywrightTests(context) {
    const timestamp = new Date().toISOString();

    try {
        context.log(`[${timestamp}] Running Playwright login test...`);

        // Launch Browser
        const browser = await playwright.chromium.launch({ headless: true });
        const page = await browser.newPage();

        // Navigate to login page
        await page.goto('https://www.saucedemo.com/');

        // Perform Login
        await page.fill('#user-name', 'standard_user');
        await page.fill('#password', 'secret_sauce');
        await page.click('#login-button');

        // Verify successful login
        await page.waitForSelector('.inventory_list', { timeout: 5000 });

        // Log Success to Application Insights
        client.trackAvailability({
            name: "SauceDemo Login Test",
            success: true,
            duration: 5000, // Execution time
            runLocation: "Azure Function",
            message: "Login successful",
            time: new Date()
        });

        context.log("✅ Playwright login test successful.");
        await browser.close();
    } catch (error) {
        context.log.error("❌ Playwright login test failed:", error);

        // Log Failure to Application Insights
        client.trackAvailability({
            name: "SauceDemo Login Test",
            success: false,
            duration: 0,
            runLocation: "Azure Function",
            message: error.message,
            time: new Date()
        });
    }
}
module.exports = { runPlaywrightTests };

5. Configure Environment Variables

         Create a .env file and set your Application Insights connection string:       

 APPLICATIONINSIGHTS_CONNECTION_STRING=<your_connection_string>

6. Deploy and Monitor

• Deploy the Function App using Azure CLI:

func azure functionapp publish <your-function-app-name>

• Monitor the availability results in Application Insights → Availability.

Setting Up Alerts for Failed Tests

To get notified when availability tests fail:

1. Open Application Insights in the Azure portal.
2. Go to Alerts → Create Alert Rule.
3. Select Signal Type: Availability Results.
4. Configure a condition where Success = 0 (Failure).
5. Add an action group (email, Teams, etc.).
6. Click Create Alert Rule.

Conclusion

With Playwright-based synthetic monitoring, you can go beyond basic URL ping tests and validate real user interactions in your application. Since Microsoft has deprecated Multi-step web tests and URL ping tests, this approach ensures better availability tracking, UI validation, and proactive issue detection in Application Insights.