Test Automation - How To Perform Automated Accessibility Checks Using Playwright Python And Axe

Introduction

In today’s digital landscape, ensuring web accessibility is not just a moral obligation but a crucial aspect of delivering inclusive experiences to users with diverse needs. Playwright, a powerful browser automation tool, coupled with the Axe accessibility testing library, provides a robust solution for automating accessibility checks during your testing pipeline. By integrating these two powerful tools, you can proactively identify and address accessibility issues.

The solution presented in this article is exemplified in my Playwright Python example project, developed in collaboration with Elias Shourosh.

Implementing the Solution

The AxeHelper class contains the check_accessibility the method that runs Axe on a given Playwright page, checks for accessibility violations based on the specified maximum allowed violations per impact level, and provides detailed reporting.

The solution code can be found here.

import json
from collections import Counter
from typing import Dict

import allure
from axe_playwright_python.sync_playwright import Axe
from playwright.sync_api import Page

class AxeHelper:

    def __init__(self, axe: Axe):
        self.axe = axe

    def check_accessibility(
        self, page: Page, maximum_allowed_violation_by_impact: Dict[str, int] = None
    ) -> None:
        """Checks accessibility of the page using playwright axe.

        :param page: Playwright Page object
        :param maximum_allowed_violations_by_impact: A dictionary
            specifying the maximum allowed violations for each impact
            level (e.g., {'minor': 2, 'moderate': 1, 'serious': 0, 'critical': 0}). If None, no violations will be allowed for
            any impact level.
        """
        if maximum_allowed_violation_by_impact is None:
            maximum_allowed_violation_by_impact = {
                "minor": 0,
                "moderate": 0,
                "serious": 0,
                "critical": 0,
            }
        results = self.axe.run(page)
        violations_count = dict(
            Counter(
                [violation["impact"] for violation in results.response["violations"]]
            )
        )
        if violations_exceeded := {
            impact_level: violation_count
            for impact_level, violation_count in violations_count.items()
            if violation_count
            > maximum_allowed_violation_by_impact.get(impact_level, 0)
        }:
            allure.attach(
                body=json.dumps(results.response["violations"], indent=4),
                name="Accessibility Violation Results",
                attachment_type=allure.attachment_type.JSON,
            )
            assert not violations_exceeded, (
                f"Found accessibility violations exceeding the maximum allowed: "
                f"{violations_exceeded}"
            )

The conftest file defines a pytest fixture that creates an instance of AxeHelper with Axe initialized. This fixture has a session scope, meaning it’s created once per test session and shared across all tests.

import pytest
from axe_playwright_python.sync_playwright import Axe

from utilities.axe_helper import AxeHelper

@pytest.fixture(scope="session")
def axe_playwright():
    """Fixture to provide an instance of AxeHelper with Axe initialized.

    This fixture has a session scope, meaning it will be created once per test session
    and shared across all tests.

    Returns:
        AxeHelper: An instance of AxeHelper with Axe initialized.
    """
    yield AxeHelper(Axe())

Test Usage

The test usage can be found here.

import allure

class TestAccessibility:

    @allure.title("Test Accessibility with Default Counts")
    def test_accessibility_default_counts(self, axe_playwright, page):
        axe_playwright.check_accessibility(page)

    @allure.title("Test Accessibility with Custom Counts")
    def test_accessibility_custom_counts(self, axe_playwright, page):
        axe_playwright.check_accessibility(
            page,
            maximum_allowed_violation_by_impact={
                "minor": 2,
                "moderate": 5,
                "serious": 0,
                "critical": 0,
            },
        )

The first test, test_accessibility_default_counts, invokes the check_accessibility method of the axe_playwright fixture, which scans the given page for accessibility violations, allowing no violations of any impact level. The second test, test_accessibility_custom_counts, demonstrates how to customize the maximum allowed violations per impact level using the maximum_allowed_violations_by_impact parameter.

In conclusion

Integrating Axe with Playwright empowers automation engineers to incorporate accessibility testing into their workflows seamlessly.

Happy testing!