[SPIKE] Check all pages' accessibility

[domoscargin]

Popped this together to check how long it'd take to run accessibility tests on ALL our pages. Just a hacky function to get all directories with an index.html file, then running an axe validation test on each of those.

Takes around 80 seconds on my machine, but open for ways to speed it up.

There's a few failures flagged, mostly to do with our heading order.

If taken forward, the code would need to be cleaned up and optimised.

Edit: looks like the tests took about 2m 45s.

[netlify]

✅ You can preview this change here:

Name	Link
🔨 Latest commit	969033f
🔍 Latest deploy log	https://app.netlify.com/sites/govuk-design-system-preview/deploys/6694e8f49d6c4100082882a3
😎 Deploy Preview	https://deploy-preview-2957--govuk-design-system-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[domoscargin]

Had a brief look at it.concurrent.each, but at first glance this doesn't play nicely with Puppeteer. Maybe something with promise.all?

[colinrotherham]

Hey @domoscargin I've rebased this one, seems a shame to leave it

I've brought over the Axe rules from alphagov/govuk-frontend@ebb9e4e added in:

• Switch Accessibility tests to @axe-core/puppeteer govuk-frontend#3522

Locally I'm seeing Puppeteer (with Axe) disconnect after a while but let's hope it's resolved by:

• axe-core-npm v4.8.0 Release dequelabs/axe-core-npm#827
• puppeteer: Investigate if Puppeteer headless: new and unloaded iframes are still an issue dequelabs/axe-core-npm#828

[colinrotherham]

@domoscargin Just rebased this to see if the @axe-core/puppeteer update fixed the disconnect issue

Looking good locally

[domoscargin]

Darn, failed here.

[colinrotherham]

Darn, failed here.

How frustrating 😭

Worth trying a separate await browser.newPage() per test (just pushed)

Otherwise, let's check again when more next major updates come out 🤦‍♂️

[colinrotherham]

Another Puppeteer update so rebasing this again

[colinrotherham]

Had some more Axe and Puppeteer updates in #3468 so giving this one a rebase

[colinrotherham]

@domoscargin Looks like it's all working 🙌

Status check failures are now accessibility report issues, as intended

[domoscargin]

I've reduced failures on this PR down to zero BUT only by disabling rules across the board.

Just wanted to get this down to zero and allow us to think about how to address each of the special cases (which are all documented in __tests__/accessibility-audit.test.js).

The rules that cause failures unless disabled:

[
  'region',
  'color-contrast-enhanced',
  'aria-allowed-attr',
  'target-size',
  'aria-allowed-role'
]

[domoscargin]

Failures notes

aria-allowed-role

This flags our use of role="region" on the <video> element, which doesn't allow that role.

This was done on purpose, but unclear if it's still relevant.

target-size

Bit of a weird one, this. It's flagging the "Open this example in a new tab" links, but only on certain examples:

#table-second-example in /components/table/index.html
#select-second-example in /components/select/default/index.html
#panel-second-example in /components/panel/default/index.html
#details-second-example in /components/details/default/index.html

There are plenty more examples with the second-example suffix, but maybe there's something about these that's confusing Axe? When I check with the web version, I don't get the same error.

It's also flagging:

button[value="yes"] (Accept analytics cookies) in /styles/page-template/custom/code/index.html
.govuk-header__service-name (Service name) in /styles/page-template/custom/code/index.html
.govuk-header__navigation-item--active > .govuk-header__link[href="#"] (Navigation item 1, 2 & 3) in /styles/page-template/custom/code/index.html
.govuk-footer__inline-list-item:nth-child(1) > .govuk-footer__link[href="#"] (Footer links) in /styles/page-template/custom/code/index.html

Again, I can't recreate this on the web.

aria-allowed-attr

This is flagging a bunch of conditional radios on the Equality Information examples. It doesn't like them having aria-expanded="false". Again, I can't recreate this on the web, and there are other examples using conditionals that don't get flagged.

color-contrast-enhanced

This flags 180+ issues, but is a AAA rule and I think we can potentially ignore it for now in our automated testing and deal with the important cases manually.

region

"All page content should be contained by landmarks" - this flags over 100 issues, but we already ignore a few instances separately.

@selfthinker If you're ever bored, feel free to have a gander at these and have a reckon (this isn't currently prioritised work).