Introduction
In this article, I will review the ability to add Git version control data into our test automation report. The technological stack used to implement the solution is:
Programing language: Java
Build tool: Maven
Testing Framework: TestNG
Reporting Framework: Allure
Dependencies:
- maven-git-commit-id-plugin
- allure-environment-writer developed by the AutomatedOwl
The Problem
Sometimes passing tests becomes flaky, this can happen for many reasons:
- The version of the product has changed.
- The version of the automation has changed.
- Changes related to the system under tests
- Test synchronization issues.
There are, of course, more reasons, and we need a way to get all the relevant information into our reporting system to debug these reasons, and an important piece of the puzzle is our version control data.
The Solution – Adding version control data into Allure report
Allure reporting framework review
First, let’s review the allure framework.Here is a demo of the framework capabilities: https://demo.qameta.io/allure/
Why did we choose Allure?
- Free and open source.
- Ability to attach various file types (images, data files, etc.) to the test cases according to our needs.
- Attach links to test cases (for example, relevant test cases in Jira).
- Does not require to manage the report object – unlike other testing frameworks.
- Built-in plugin for Jenkins.
- Uses a reflection-based approach in order to extract the data passed into test steps and show them in the report.
- Option to set severity levels for test cases and to filter results based on severity – this adds another layer that helps to quickly diagnoses our test results.
- Understandable ability to analyze the type of failure - whether of a runtime error or real failure that occurred as a result of Assert.
The feature that we will use in this article to add our version control data is the ability to add environment information into the report.
Implementing the Solution
First, we need to add maven-git-commit-id-plugin into our pom.xml file. Then, after a successful Maven build a git property file is created under target/classes directory. The file contains the following Properties that are available for extraction:
git.branch, git.build.host, git.build.time, git.build.user.name, git.build.version, git.closest.tag.commit.count, git.closest.tag.name, git.commit.id, git.commit.id.abbrev, git.commit.id.describe, git.commit.id.describe-short, git.commit.message.full, git.commit.message.short, git.commit.time, git.commit.user.name, git.dirty, git.remote.origin.url, git.tags.
After the property file is created, we need to implement a mechanism in our test framework that extracts the data and inject it into the report. The tool that helps us to create this mechanism is allure-environment-writer.
In the following example code, I have created a test method that runs after the entire suite is completed. We are creating a property object that is loaded from an input stream which is our git properties file, then we create an immutable map of property names and the data from the property object.
@AfterSuite
public void WriteAllureEnvironmente() throws IOException {
String path = System.getProperty("user.dir") + "\\target\\classes\\git.properties";
Properties prop = new Properties();
InputStream input = new FileInputStream(path);
prop.load(input);
allureEnvironmentWriter(ImmutableMap.<String, String>builder()
.put("git.branch", prop.getProperty("git.branch"))
.put("git.build.host", prop.getProperty("git.build.host"))
.put("git.build.time", prop.getProperty("git.build.time"))
.put("git.commit.id", prop.getProperty("git.commit.id"))
.put("git.commit.message.short", prop.getProperty("git.commit.message.short"))
.put("git.commit.time", prop.getProperty("git.commit.time"))
.put("git.remote.origin.url", prop.getProperty("git.remote.origin.url"))
.build());
}The outcome of the run in the generated report is:
The relevant version control data is displayed under the environment section. Please notice that during the creation of the map we decide which properties we want to extract.
In conclusion
In this article, we reviewed the problem we were facing when needing to debug failing tests and not knowing the version control data of our automation code, then implemented a solution for adding the version control data directly to our reporting framework.
Happy testing!