Smart Reporting for UI Automation

Iron Contributor

Jun 08, 2020

Author : Anusha Ammaluru

For UI Test Automation, once a test script runs successfully, the next step is to report the test execution results. Reports shows the ratio of pass/fail of the test cases, duration of a test run, and the total number of test scripts executed etc. It is the main document to be shared with stakeholders. Don’t you think it should be a user-friendly report with easy readability and customizable pie-charts?

With the DevOps speed, Test reporting becomes critical. With so many releases and permutations, you’re probably sitting on a mountain of data. And that makes it hard to focus on what’s important — plus it slows you down. You need the right test reporting to cut through the chaos.

I came across a user-friendly reporting tool called ExtentReports.

ExtentReports is one of the best built-in ways to generate customizable HTML reports with a good user interface which can be shared with all the major project stakeholders. With Extent Framework, you can create beautiful, interactive, and detailed reports for your tests. Add events, screenshots, tags, devices, authors, or any other relevant information you decide is important to create an informative and a stunning report.

It is an open source library that can be easily configured with Selenium, thereby making it the best choice for automation testers.

Advantages of using Extent Reports:

Rich HTML reports that can be customized to offer a graphical representation of each step.
Depicts the time required for test execution.
If required, screenshots can be taken and displayed for each step in a test.
Allows easy tracking of multiple test case runs in a single test suite.
Tracks the exceptions of failure test cases
Easy to integrate with numerous frameworks, especially TestNG and JUnit.
Historical reports with test case failure rates across builds are presented out in a single dashboard with pie chart/ graph representation

Snapshots of Extent reporting: https://extentreports.com/

Report levels:

Tests - Individual Test step logs are being recorded
Tags - Individual test categories level view is generated
Exception - List of exceptions occurred report
Dashboard - Dashboard of the complete test run

Dashboard for Extent report:

Dashboard for Klov

This is the extensive reporting for showing historical reports of tests from all the builds and provides a history of test case failures graph.

Using Extent Reports in Selenium:

Extent Reports in Selenium contain two major, frequently used classes:

ExtentReports class
ExtentTest class

Syntax:

ExtentReports reports = new ExtentReports("Path of directory to store the resultant HTML file", true/false); 
ExtentTest test = reports.StartTest("TestName");

The first line, ExtentReports class generates HTML reports based on a user-specified path. The Boolean flag notes if the existing report needs to be overwritten or a completely new report needs to be generated. ‘True’ is the default value, which indicates that all existing data will be overwritten.

The second line, ExtentTest class is required to log test steps onto the previously generated HTML report.

Both these classes can be used with several frequently used built-in methods:

StartTest
EndTest
Log
Flush

StartTest and EndTest methods are utilized to execute preconditions and postconditions of a test case. The log method is utilized to log the status of each test step onto the resultant HTML report. The flush method is used to erase any previous data on a relevant report and create a whole new report.

A Test Status can be denoted by any of the following values:

Pass
Fail
Skip
Info

Syntax:

reports.EndTest();
test.Log(LogStatus.Pass,"Test Passed");
test.Log(LogStatus.Fail,"Test Failed");
test.Log(LogStatus.Skip,"Test Skipped");
test.Log(LogStatus.Info,"Test Info");

How to capture screenshots in Extent Report:

By capturing screenshots, testers can better identify bugs since they can detect every issue encountered during test execution. However, since the screenshots consume quite a bit of memory, it is best to capture them only if a test fails.

Usage:

if (status == TestStatus.Failed)
            {
                string screenShotPath = GetScreenShot.Capture(driver, "ScreenShotName");
                test.Log(LogStatus.Fail, stackTrace + errorMessage);
                test.Log(LogStatus.Fail, "Snapshot below: " + 
                test.AddScreenCapture(screenShotPath));
            }

Historical Reporting:

The reports we discussed earlier was an HTML report which is offline and every time while the test executes, a new HTML file will be generated. However, to achieve the historical reporting of multiple test runs combined, we need to use Klov Reporter.

Using ExtentKlovReporter:

KlovReporter works with the Extent Framework to push data into a MongoDB instance and the Klov server (media).

Steps to Integrate Klov reporting with existing Automation framework: