Publish Code Coverage Report in Azure DevOps Services pipeline execution summary

Chandra Sekhar Viswanadha

Ahetejazahmad Khan

May 18th, 20230 0

This blog content is compiled by @Ahetejaz from Azure DevOps CSS support team. He helped a customer in implementing Code Coverage report as part of Pipeline execution summary page in Azure DevOps Service.

Below screenshot shows the code coverage report published in pipeline summary page:

Steps to generate code coverage:

Create .NET Core application using Visual Studio.
From NuGet package download ‘coverlet.msbuild’.

Coverage tool can create coverage file required by either Jacoco or Cobertura.

Jacoco or Cobertura can populate the data to code coverage tab in Azure Pipeline run (refer above screenshot).

‘coverlet.msbuild’ is one such tool which can create such coverage to be used by the Cobertura and Jacoco.

– Pipeline uses below tasks to generate code coverage report.

trigger:

– none

jobs:

– job: Job_1

displayName: Agent job 1

pool:

vmImage: windows-latest

steps:

– checkout: self

– task: UseDotNet@2

displayName: Net Core sdk 2

inputs:

includePreviewVersions: true

– task: DotNetCoreCLI@2

inputs:

command: ‘build’

projects: ‘**/*.csproj’

arguments: ‘/t:restore’

– task: DotNetCoreCLI@2

displayName: Test

condition: succeededOrFailed()

inputs:

command: test

projects: ‘**/*.csproj ‘

arguments: /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=$(Build.SourcesDirectory)/TestResults/Coverage/

– task: reportgenerator@4

displayName: ReportGenerator

condition: succeededOrFailed()

inputs:

reports: $(Build.SourcesDirectory)/TestResults/Coverage/coverage.cobertura.xml

targetdir: $(Build.SourcesDirectory)\coveragereport

– task: PublishCodeCoverageResults@1

displayName: Publish code coverage

condition: always()

inputs:

codeCoverageTool: Cobertura

summaryFileLocation: $(Build.SourcesDirectory)\coveragereport\cobertura.xml

reportDirectory: $(Build.SourcesDirectory)\coveragereport\

additionalCodeCoverageFiles: $(Build.SourcesDirectory)\coveragereport*.html

Points to Remember:

1. The feature to have code coverage as part of DevOps UI is under development. Reference link – https://developercommunity.visualstudio.com/idea/366142/support-vstest-coverage-code-coverage-build-result.html There is no fixed timeline as to when the feature would be available.

2. If we use .NET Core projects, then we could use .NET Core task which would give code coverage in the build logs. Not in DevOps UI.

3. Publish code coverage result task supports coverage result formats such as Cobertura and JaCoCo. To use the task we would need to use a code coverage tool as part of the project solution. We may add a Nuget package to get the tool added to generate code coverage file required by either Cobertura or JaCoCo. Cobertura or JaCoCo could then populate the data to the code coverage tab. Coverlet.msbuild is one such tool which can create such coverage to be used by the Cobertura and JaCoCo.

4. Before using publish code coverage result task we might need to understand that this task comes with certain limitation as mentioned in the article – https://github.com/coverlet-coverage/coverlet#quick-start

Coverlet supports only SDK-style projects https://docs.microsoft.com/en-us/visualstudio/msbuild/how-to-use-project-sdk?view=vs-2019

5. For .NET Framework (4.7.2) project which does not use SDK style will not publish code coverage result in DevOps UI.

6. For .NET Core project, we will be able to publish code coverage result using 3rd party component mentioned in point 3. Also, we used an extra task ‘ReportGenerator’ which is NOT owned by Microsoft to publish the results. Support for this task will not be provided by Microsoft.

Two link that might be handy for reference:

– https://docs.microsoft.com/en-us/visualstudio/msbuild/how-to-use-project-sdk?view=vs-2019
– https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/KnownIssues.md#1-vstest-stops-process-execution-earlydotnet-test

Cheers!!

Ahmad