Automating End-to-End testing with Playwright and Azure Pipelines

This article outlines how to run the tests in a pipeline in Azure DevOps. 

Playwright has a number of different "Reporters". To have the executed tests show up in the "Tests" tab, you would need to https://playwright.dev/docs/test-reporters#junit-reporter. After the pipeline executes the tests, you need to publish the JUnit xml file using the https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/publish-test-results-v2?view=azure-pipelines&tabs=trx%2Ctrxattachments%2Cyaml task.

1. Modify the playwright.config.ts file to include the junit reporter:

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [['junit', { outputFile: 'results.xml' }]],
});

2. Add a "Publish Test Results" task to the pipeline as the last step. The YAML definition for the task would look like:

# Publish test results to Azure Pipelines.
- task: PublishTestResults@2
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: '**/results.xml'