Using accessibility guidelines to improve test quality and robustness

There's not a lot of talk about Web Accessibility around here. It is viewed as an uninteresting topic, dealt with on an as-needed basis. But it shouldn't be this way!

Let me give you an intro to WAI-ARIA from an angle you probably have never seen before: How to utilize accessibility guidelines to drastically improve your unit tests?

What do Unit Testing and Accessibility have in common? Much more than you would think!

Unit Testing

Let's begin with some (opinionated) Do-s & Don't-s on Unit Testing. I am assuming you are already familiar, this list is not meant to be complete or perfect.

• Test the behavior of components. Don't test implementation details
• Be resistant to non-user-visible changes in HTML / React structure. Assert from the users' perspective.
• Focus on the semantics, don't break if visuals or the copy changes.
• Use the app like a regular user would. Don't query artificial constructs (like CSS selectors or ids). Trigger events a user would trigger themselves.

Accessibility

Now what about some guidelines for accessibility?

• Write semantic HTML that screen-readers can understand. Don't rely on CSS to convey structure.
• Non-user-visible changes in HTML structure should not change screen-reader output either.
• Focus first on the core value/functionality provided for your user, styling & fluff comes second.
• Your site by should be discoverable by screen-reader the same way it is for regular users. Don't treat headless user-agents as second-class citizens.

Unit Testing + Accessibility = 💞

Now back to the question I asked before:

What do Unit Testing and Accessibility have in common?

Let's put the two lists side by side:

Testing	Accessibility
Test the behavior of components. Don't test implementation details	Write semantic HTML that screen-readers can understand. Don't rely on CSS to convey structure.
Be resistant to non-user-visible changes in HTML / React structure.	Assert from the users' perspective. Non-user-visible changes in HTML structure should not change screen-reader output either.
Focus on the semantics, don't break if visuals or the copy changes.	Focus first on the core value/functionality provided for your user, styling & fluff comes second.
Use the app like a regular user would. Don't query artificial constructs (like CSS selectors or ids). Trigger events a user would trigger themselves.	Your site by should be discoverable by screen-reader the same way it is for regular users. Don't treat headless user-agents as second-class citizens.

At this point, looking back at both lists one can start to see the common pattern:

• Describe your semantics well in HTML.
• Treat headless agents as first-class citizens.

To put it in another way, screen-readers and test environments are very much alike: They are headless user-agents interpreting your DOM structure without putting much if any effort into calculating layout, styling and visuals in general.

Enough talk, let's see some code

Let's start with a simple example, non-semantic vs semantic HTML. I'm sure most of you are familiar with this one already, but let's see the testing implications.

Let's start with this HTML:

<div class="nav">
  <div class="nav-item">
    <strong>First Item</strong>
  </div>
  <div class="nav-item">
    <strong>Second Item</strong>
  </div>
</div>

Let's say in a test we would like to retrieve the first navigation element. We can do that with a simple CSS query like querySelector('.nav > :first-child'), or a text-based selector (most testing frameworks should include one) getByText('First Item').

While both of them work at the moment, both are fragile as they break the constraints we set out earlier:

• The first one depends on exact CSS classes, if someone renames .nav to .navigation, the test breaks.
• It also depends on the exact CSS structure, adding another div in-between - like grouping the items - or adding a logo before the items will break it as well.
• The second one is somewhat better, but depends on the exact words of the copy, which you don't usually want to test.

Let's rewrite this to be more textbook-like semantic HTML:

<nav>
  <ul>
    <li>First Item</li>
    <li>Second Item</li>
  </ul>
</nav>

At this point your CSS query becomes querySelector('nav li:fist-of-type') which is much more robust, it is ignorant of grouping the items or adding a logo. (Unless of course you add the logo as another li which is the wrong thing to do).

Even better would be to query by WAI-ARIA role, which we'll briefly discuss below: getAllByRole(getByRole('navigation'), 'listitem')[0]. This does almost exactly what our plain English requirements said: Retrieves the navigation menu, and within that menu, the first list item.

Roles

In WAI-ARIA, every HTML element can have something called a role. This is either set implicitly, e.g. nav tags have the navigation role by default, or explicitly by applying the role="some-role" attribute.

Let's imagine you are testing some JavaScript that shows this tooltip on some condition. Your goal is to assert that the tooltip is visible.

<div class="text-center w-96">
  Tooltip content
</div>

The simple solution would be to append a data-testid="some-tooltip" and the div then write the assertion as expect(screen.getByTestId('some-tooltip')).toBeVisible(), or similar in another testing framework. But now you are querying an artificial construct - a test-id -, and you've given yourself another string to maintain, which doesn't even add any value from the perspective of someone writing (or reading) the HTML.

What should we do instead? Think about what our goal is. This is a tooltip. We need to assert a tooltip is visible. Then let's do just that: WAI-ARIA has a role called tooltip that is defined as:

A contextual popup that displays a description for an element.

Instead of the test-id, you can add role="tooltip" to the div, then your assertion becomes expect(screen.getByRole('tooltip')).toBeVisible(), which is almost exactly the business-requirement.

I bet if you show this to a business person who has zero JavaScript knowledge they would still understand it.

And this is the power of using roles and possibly other ARIA attributes for testing!

Of course the full WAI-ARIA spec is much more in depth, with many other aspects (attributes, states, etc.) being just as useful for testing. Describing those is beyond the scope of this introductory article, but I encourage you to at least go through the MDN intro on WAI-ARIA to get an overview of the tools available to you.

Thank you for reading, happy testing!

Resources

MDN - Accessibility overview
MDN - HTML accessibility
MDN - Intro to WAI-ARIA
W3 - Definition of Roles