Smart Reporting for UI Automation

Published Jun 08 2020 03:46 AM 3,634 Views

Author : Anusha Ammaluru 

 

For UI Test Automation, once a test script runs successfully, the next step is to report the test execution results. Reports shows the ratio of pass/fail of the test cases, duration of a test run, and the total number of test scripts executed etc. It is the main document to be shared with stakeholders. Don’t you think it should be a user-friendly report with easy readability and customizable pie-charts?

With the DevOps speed, Test reporting becomes critical. With so many releases and permutations, you’re probably sitting on a mountain of data. And that makes it hard to focus on what’s important — plus it slows you down. You need the right test reporting to cut through the chaos.

I came across a user-friendly reporting tool called ExtentReports.

 

ExtentReports is one of the best built-in ways to generate customizable HTML reports with a good user interface which can be shared with all the major project stakeholders. With Extent Framework, you can create beautiful, interactive, and detailed reports for your tests. Add events, screenshots, tags, devices, authors, or any other relevant information you decide is important to create an informative and a stunning report.

It is an open source library that can be easily configured with Selenium, thereby making it the best choice for automation testers.

 

Advantages of using Extent Reports:

  • Rich HTML reports that can be customized to offer a graphical representation of each step.
  • Depicts the time required for test execution.
  • If required, screenshots can be taken and displayed for each step in a test.
  • Allows easy tracking of multiple test case runs in a single test suite.
  • Tracks the exceptions of failure test cases
  • Easy to integrate with numerous frameworks, especially TestNG and JUnit.
  • Historical reports with test case failure rates across builds are presented out in a single dashboard with pie chart/ graph representation

 

Snapshots of Extent reportinghttps://extentreports.com/

 

Report levels: 

  • Tests - Individual Test step logs are being recorded 
  • Tags - Individual test categories level view is generated
  • Exception - List of exceptions occurred report 
  • Dashboard - Dashboard of the complete test run

 

Dashboard for Extent report: 

dashboard snap.png

 

Dashboard for Klov

This is the extensive reporting for showing historical reports of tests from all the builds and provides a history of test case failures graph. 

 

klov snap.png

 

 Using Extent Reports in Selenium:

 Extent Reports in Selenium contain two major, frequently used classes:

  • ExtentReports class
  • ExtentTest class

 

Syntax:

 

 

ExtentReports reports = new ExtentReports("Path of directory to store the resultant HTML file", true/false); 
ExtentTest test = reports.StartTest("TestName");

 

 

 

The first line, ExtentReports class generates HTML reports based on a user-specified path. The Boolean flag notes if the existing report needs to be overwritten or a completely new report needs to be generated. ‘True’ is the default value, which indicates that all existing data will be overwritten.

 

The second line, ExtentTest class is required to log test steps onto the previously generated HTML report.

Both these classes can be used with several frequently used built-in methods:

  • StartTest
  • EndTest
  • Log
  • Flush

 

StartTest and EndTest methods are utilized to execute preconditions and postconditions of a test case. The log method is utilized to log the status of each test step onto the resultant HTML report. The flush method is used to erase any previous data on a relevant report and create a whole new report.

 

A Test Status can be denoted by any of the following values:

  • Pass
  • Fail
  • Skip
  • Info

 

Syntax:

 

 

reports.EndTest();
test.Log(LogStatus.Pass,"Test Passed");
test.Log(LogStatus.Fail,"Test Failed");
test.Log(LogStatus.Skip,"Test Skipped");
test.Log(LogStatus.Info,"Test Info");

 

 

 

How to capture screenshots in Extent Report:

By capturing screenshots, testers can better identify bugs since they can detect every issue encountered during test execution. However, since the screenshots consume quite a bit of memory, it is best to capture them only if a test fails.

Usage:

 

 

 

if (status == TestStatus.Failed)
            {
                string screenShotPath = GetScreenShot.Capture(driver, "ScreenShotName");
                test.Log(LogStatus.Fail, stackTrace + errorMessage);
                test.Log(LogStatus.Fail, "Snapshot below: " + 
                test.AddScreenCapture(screenShotPath));
            }

 

 

 

 

Historical Reporting:

The reports we discussed earlier was an HTML report which is offline and every time while the test executes, a new HTML file will be generated. However, to achieve the historical reporting of multiple test runs combined, we need to use Klov Reporter.

Using ExtentKlovReporter:

KlovReporter works with the Extent Framework to push data into a MongoDB instance and the Klov server (media).

Steps to Integrate Klov reporting with existing Automation framework: 

 

  1. Add the Nugget reference “Extent reports” to existing automation solution: 
    1.     Report1.jpg
  2. Install MongoDB 3.2 (other versions may not work correctly)  
    1. Go to URL https://www.mongodb.com/download-center/community?jmp=docs
    2. Select the version “3.2.22” version and download 
    3. Extract the file and keep it in a specified folder. 
  3. Download the ‘Reporting’ repo from GitHub: https://github.com/anu-01/Reporting 
  4. Update the path of MongoDb to the downloaded folder in App.Config 

Untitled.jpg

 

 

 

HTML Reports Integration to Azure pipeline:

 

Step 1 : Copy Reports into staging directory

 

copyReports.png

 

 

Step 2 : Publish Reports

 

Publishreport.png

Step 3: Queue the build. Once published, reports will be available under artifacts in the pipeline

 

publishedArtifacts.png

2 Comments
Version history
Last update:
‎Feb 11 2021 11:25 AM