Author : Anusha Ammaluru
For UI Test Automation, once a test script runs successfully, the next step is to report the test execution results. Reports shows the ratio of pass/fail of the test cases, duration of a test run, and the total number of test scripts executed etc. It is the main document to be shared with stakeholders. Don’t you think it should be a user-friendly report with easy readability and customizable pie-charts?
With the DevOps speed, Test reporting becomes critical. With so many releases and permutations, you’re probably sitting on a mountain of data. And that makes it hard to focus on what’s important — plus it slows you down. You need the right test reporting to cut through the chaos.
I came across a user-friendly reporting tool called ExtentReports.
ExtentReports is one of the best built-in ways to generate customizable HTML reports with a good user interface which can be shared with all the major project stakeholders. With Extent Framework, you can create beautiful, interactive, and detailed reports for your tests. Add events, screenshots, tags, devices, authors, or any other relevant information you decide is important to create an informative and a stunning report.
It is an open source library that can be easily configured with Selenium, thereby making it the best choice for automation testers.
Advantages of using Extent Reports:
- Rich HTML reports that can be customized to offer a graphical representation of each step.
- Depicts the time required for test execution.
- If required, screenshots can be taken and displayed for each step in a test.
- Allows easy tracking of multiple test case runs in a single test suite.
- Tracks the exceptions of failure test cases
- Easy to integrate with numerous frameworks, especially TestNG and JUnit.
- Historical reports with test case failure rates across builds are presented out in a single dashboard with pie chart/ graph representation
Snapshots of Extent reporting: https://extentreports.com/
Report levels:
- Tests - Individual Test step logs are being recorded
- Tags - Individual test categories level view is generated
- Exception - List of exceptions occurred report
- Dashboard - Dashboard of the complete test run
Dashboard for Extent report:
Dashboard for Klov
This is the extensive reporting for showing historical reports of tests from all the builds and provides a history of test case failures graph.
Using Extent Reports in Selenium:
Extent Reports in Selenium contain two major, frequently used classes:
- ExtentReports class
- ExtentTest class
Syntax:
ExtentReports reports = new ExtentReports("Path of directory to store the resultant HTML file", true/false);
ExtentTest test = reports.StartTest("TestName");
The first line, ExtentReports class generates HTML reports based on a user-specified path. The Boolean flag notes if the existing report needs to be overwritten or a completely new report needs to be generated. ‘True’ is the default value, which indicates that all existing data will be overwritten.
The second line, ExtentTest class is required to log test steps onto the previously generated HTML report.
Both these classes can be used with several frequently used built-in methods:
- StartTest
- EndTest
- Log
- Flush
StartTest and EndTest methods are utilized to execute preconditions and postconditions of a test case. The log method is utilized to log the status of each test step onto the resultant HTML report. The flush method is used to erase any previous data on a relevant report and create a whole new report.
A Test Status can be denoted by any of the following values:
- Pass
- Fail
- Skip
- Info
Syntax:
reports.EndTest();
test.Log(LogStatus.Pass,"Test Passed");
test.Log(LogStatus.Fail,"Test Failed");
test.Log(LogStatus.Skip,"Test Skipped");
test.Log(LogStatus.Info,"Test Info");
How to capture screenshots in Extent Report:
By capturing screenshots, testers can better identify bugs since they can detect every issue encountered during test execution. However, since the screenshots consume quite a bit of memory, it is best to capture them only if a test fails.
Usage:
if (status == TestStatus.Failed)
{
string screenShotPath = GetScreenShot.Capture(driver, "ScreenShotName");
test.Log(LogStatus.Fail, stackTrace + errorMessage);
test.Log(LogStatus.Fail, "Snapshot below: " +
test.AddScreenCapture(screenShotPath));
}
Historical Reporting:
The reports we discussed earlier was an HTML report which is offline and every time while the test executes, a new HTML file will be generated. However, to achieve the historical reporting of multiple test runs combined, we need to use Klov Reporter.
Using ExtentKlovReporter:
KlovReporter works with the Extent Framework to push data into a MongoDB instance and the Klov server (media).
Steps to Integrate Klov reporting with existing Automation framework:
- Add the Nugget reference “Extent reports” to existing automation solution:
-
- Install MongoDB 3.2 (other versions may not work correctly)
- Go to URL https://www.mongodb.com/download-center/community?jmp=docs
- Select the version “3.2.22” version and download
- Extract the file and keep it in a specified folder.
- Download the ‘Reporting’ repo from GitHub: https://github.com/anu-01/Reporting
- Update the path of MongoDb to the downloaded folder in App.Config
HTML Reports Integration to Azure pipeline:
Step 1 : Copy Reports into staging directory
Step 2 : Publish Reports
Step 3: Queue the build. Once published, reports will be available under artifacts in the pipeline
