Introduction
In this article, I will review the ability to add Git version control data into our test automation report.
The technological stack used to implement the solution is:
Programing language: Python
Testing Framework: pytest
Reporting Framework: Allure
link to the project that was developed together with Elias Shourosh.
Sometimes passing tests becomes flaky, this can happen for many reasons:
- The version of the product has changed.
- The version of the automation has changed.
- Changes related to the system under tests
- Test synchronization issues.
There are, of course, more reasons, and we need a way to get all the relevant information into our reporting system to debug these reasons, and an important piece of the puzzle is our version control data.
The Solution – Adding version control data into Allure report
Allure reporting framework review
First, let’s review the allure framework.Here is a demo of the framework capabilities:https://demo.qameta.io/allure/
Why did we choose Allure?
- Free and open source.
- Ability to attach various file types (images, data files, etc.) to the test cases according to our needs.
- Attach links to test cases (for example, relevant test cases inJira).
- Does not require to manage the report object – unlike other testing frameworks.
- Built-in plugins for GitHub Actions and Jenkins.
- The report extracts the data passed into test steps and shows them in the report.
- Option to set severity levels for test cases and to filter results based on severity – this adds another layer that helps to quickly diagnoses our test results.
- Understandable ability to analyze the type of failure - whether of a runtime error or real failure that occurred as a result of Assert.
The feature that we will use in this article to add our version control data is the ability to add environment information into the report.
Implementing the Solution
This first part consists of the AllureEnvironmentParser class, which has the write_to_allure_env method. A constructor specifies the file name of the XML file that will contain the data to be written. It writes key-value pairs separated by newlines from a dictionary to an XML file.
Second, the write_allure_environment fixture uses the GitPython library to access Git repository information and write it to the XML file. The AllureEnvironmentParser class writes information about the current commit, such as commit ID, author name, commit message, and commit date, to the XML file. This is after other fixtures have finished executing. In addition, it appends the browser version, base URL, and active branch name to the XML file.
The outcome of the run in the generated report is located under the environment section:
In conclusion
In this article, we reviewed the problem we were facing when needing to debug failing tests and not knowing the version control data of our automation code, then implemented a solution for adding the version control data directly to our reporting framework.
Happy testing!