Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: core(axe): new audits empty-table-header, frame-focusable-content #12536

Closed
wants to merge 19 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions lighthouse-cli/test/cli/__snapshots__/index-test.js.snap
Original file line number Diff line number Diff line change
Expand Up @@ -255,9 +255,15 @@ Object {
Object {
"path": "accessibility/duplicate-id-aria",
},
Object {
"path": "accessibility/empty-table-header",
},
Object {
"path": "accessibility/form-field-multiple-labels",
},
Object {
"path": "accessibility/frame-focusable-content",
},
Object {
"path": "accessibility/frame-title",
},
Expand Down Expand Up @@ -588,11 +594,21 @@ Object {
"id": "duplicate-id-aria",
"weight": 10,
},
Object {
"group": "a11y-tables-lists",
"id": "empty-table-header",
"weight": 2,
},
Object {
"group": "a11y-names-labels",
"id": "form-field-multiple-labels",
"weight": 2,
},
Object {
"group": "a11y-navigation",
"id": "frame-focusable-content",
"weight": 3,
},
Object {
"group": "a11y-names-labels",
"id": "frame-title",
Expand Down
12 changes: 12 additions & 0 deletions lighthouse-cli/test/fixtures/a11y/a11y_tester.html
Original file line number Diff line number Diff line change
Expand Up @@ -135,12 +135,24 @@
<div id="duplicate-id-aria"></div>
<input type="text" aria-labelledby="duplicate-id-aria"/>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this needs to be an appropriate use of aria-labelledby or this now gets flagged by aria-allowed-attr. Nice upgrade for that audit

</section>
<p>empty-table-header</p>
<section>
<table>
<tr>
<th></th>
</tr>
</table>
</section>
<p>form-field-multiple-labels</p>
<section>
<label for="form-field-multiple-labels" id="label1">label1</label>
<label for="form-field-multiple-labels">label2</label>
<input type="checkbox" id="form-field-multiple-labels" />
</section>
<p>frame-focusable-content</p>
<section>
<iframe title="some title" tabindex="-1" src="../dobetterweb/focusable-iframe.html"></iframe>
</section>
<p>frame-title</p>
<section>
<iframe id="frame-title"></iframe>
Expand Down
9 changes: 9 additions & 0 deletions lighthouse-cli/test/fixtures/dobetterweb/dbw_tester.html
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,15 @@ <h2>Do better web tester page</h2>
<button class="small-button">Do something</button>
<button class="small-button">Do something else</button>

<!-- FAIL - frame-focusable-content -->
<iframe tabindex="-1" src="focusable-iframe.html"></iframe>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use srcdoc instead to avoid the 1-line extra file? :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was my first try, didn't work! Not supported I suppose.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was my first try, didn't work! Not supported I suppose.

interesting, are srcdoc iframes a unique origin and so not available in the parent via js (so axe wouldn't be able to see inside)?

<!-- FAIL - empty-table-header -->
<table>
<tr>
<th></th>
</tr>
</table>

<!-- Some websites overwrite the original Error object. The captureJSCallUsage function
relies on the native Error object and prepareStackTrace from V8. When overwriting the stack
property the e.stack inside the gatherer won't be in the correct format
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
<a href='/'>Home</a>
29 changes: 25 additions & 4 deletions lighthouse-cli/test/smokehouse/test-definitions/a11y.js
Original file line number Diff line number Diff line change
Expand Up @@ -476,7 +476,7 @@ const expectations = {
node: {
'type': 'node',
'selector': 'body > section > input#form-field-multiple-labels',
'path': '2,HTML,1,BODY,43,SECTION,2,INPUT',
'path': '2,HTML,1,BODY,45,SECTION,2,INPUT',
'snippet': '<input type="checkbox" id="form-field-multiple-labels">',
'explanation': 'Fix all of the following:\n Multiple label elements is not widely supported in assistive technologies. Ensure the first label contains all necessary information.',
'nodeLabel': 'body > section > input#form-field-multiple-labels',
Expand All @@ -487,7 +487,7 @@ const expectations = {
{
relatedNode: {
'type': 'node',
'path': '2,HTML,1,BODY,43,SECTION,0,LABEL',
'path': '2,HTML,1,BODY,45,SECTION,0,LABEL',
'selector': 'body > section > label#label1',
'snippet': '<label for="form-field-multiple-labels" id="label1">',
'nodeLabel': 'label1',
Expand All @@ -496,7 +496,7 @@ const expectations = {
{
relatedNode: {
'type': 'node',
'path': '2,HTML,1,BODY,43,SECTION,1,LABEL',
'path': '2,HTML,1,BODY,45,SECTION,1,LABEL',
'selector': 'body > section > label',
'snippet': '<label for="form-field-multiple-labels">',
'nodeLabel': 'label2',
Expand Down Expand Up @@ -532,7 +532,7 @@ const expectations = {
node: {
'type': 'node',
'selector': 'body > section > h3',
'path': '2,HTML,1,BODY,47,SECTION,1,H3',
'path': '2,HTML,1,BODY,51,SECTION,1,H3',
'snippet': '<h3>',
'explanation': 'Fix any of the following:\n Heading order invalid',
'nodeLabel': 'sub-sub-header',
Expand Down Expand Up @@ -751,6 +751,27 @@ const expectations = {
],
},
},
'frame-focusable-content': {
score: null,
scoreDisplayMode: 'notApplicable',
},
'empty-table-header': {
score: null,
details: {
items: [
{
node: {
'type': 'node',
'selector': 'table > tbody > tr > th',
'path': '2,HTML,1,BODY,43,SECTION,0,TABLE,0,TBODY,0,TR,0,TH',
'snippet': '<th>',
'explanation': 'Fix any of the following:\n Element does not have text that is visible to screen readers',
'nodeLabel': 'table > tbody > tr > th',
},
},
],
},
},
},
},
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -391,11 +391,11 @@ const expectations = {
},
'dom-size': {
score: 1,
numericValue: 153,
numericValue: 158,
details: {
items: [
{statistic: 'Total DOM Elements', value: 153},
{statistic: 'Maximum DOM Depth', value: 4},
{statistic: 'Total DOM Elements', value: 158},
{statistic: 'Maximum DOM Depth', value: 5},
{
statistic: 'Maximum Child Elements',
value: 100,
Expand Down
44 changes: 44 additions & 0 deletions lighthouse-core/audits/accessibility/empty-table-header.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
/**
* @license Copyright 2021 The Lighthouse Authors. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
*/
'use strict';

/**
* @fileoverview Ensures table headers have discernible text.
* See base class in axe-audit.js for audit() implementation.
*/

const AxeAudit = require('./axe-audit.js');
const i18n = require('../../lib/i18n/i18n.js');

const UIStrings = {
/** Title of an accesibility audit that checks if there are any table headers without visible text. This title is descriptive of the successful state and is shown to users when no user action is required. */
title: '`table` headers have visible text',
/** Title of an accesibility audit that checks if there are any table headers without visible text. This title is descriptive of the failing state and is shown to users when there is a failure that needs to be addressed. */
failureTitle: '`table` headers do not have visible text',
/** Description of a Lighthouse audit that tells the user *why* they should try to pass. This is displayed after a user expands the section to see more. No character length limits. 'Learn More' becomes link text to additional documentation. */
description: '`table` headers should have visible text so assistive technologies like screen readers may describe table contents appropriately. [Learn more](https://dequeuniversity.com/rules/axe/4.2/empty-table-header).',
};

const str_ = i18n.createMessageInstanceIdFn(__filename, UIStrings);

class EmptyTableHeader extends AxeAudit {
/**
* @return {LH.Audit.Meta}
*/
static get meta() {
return {
id: 'empty-table-header',
title: str_(UIStrings.title),
failureTitle: str_(UIStrings.failureTitle),
description: str_(UIStrings.description),
scoreDisplayMode: AxeAudit.SCORING_MODES.INFORMATIVE,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

empty-table-header only ever passes or is incomplete. Based on #10072, we should be handling those audits as scoreDisplayMode: 'informative', so here we are.

I was out on leave for that PR so I don't have all the background, but today it does seems a little strange to not have it count for anything when passing, even if axe can't be sure the failing case is really failing.

That said, it seems like our a11y_tester example really is a failure, and it's not clear to me why "Unable to determine if element has children" is the best we can do (the <th> is plainly empty) and if it's useful to show that to users.

Maybe for now we should drop this audit after all and re-add it when/if it gains some certainty?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

requiredArtifacts: ['Accessibility'],
};
}
}

module.exports = EmptyTableHeader;
module.exports.UIStrings = UIStrings;
43 changes: 43 additions & 0 deletions lighthouse-core/audits/accessibility/frame-focusable-content.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
/**
* @license Copyright 2021 The Lighthouse Authors. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
*/
'use strict';

/**
* @fileoverview No `<iframe tabindex=-1>` has focusable content.
* See base class in axe-audit.js for audit() implementation.
*/

const AxeAudit = require('./axe-audit.js');
const i18n = require('../../lib/i18n/i18n.js');

const UIStrings = {
/** Title of an accesibility audit that checks if there are any iframes with focusable content that mistakenly set tabindex = -1. This title is descriptive of the successful state and is shown to users when no user action is required. */
title: '`<iframe>` elements with focusable content do not have a negative `tabindex`',
/** Title of an accesibility audit that checks if there are any iframes with focusable content that mistakently set tabindex = -1. This title is descriptive of the failing state and is shown to users when there is a failure that needs to be addressed. */
failureTitle: '`<iframe>` elements with focusable content have a negative `tabindex`',
/** Description of a Lighthouse audit that tells the user *why* they should try to pass. This is displayed after a user expands the section to see more. No character length limits. 'Learn More' becomes link text to additional documentation. */
description: 'Focusable elements within an `<iframe>` element with a negative `tabindex` cannot be reached via keyboard navigation. [Learn more](https://dequeuniversity.com/rules/axe/4.2/frame-focusable-content).',
};

const str_ = i18n.createMessageInstanceIdFn(__filename, UIStrings);

class FrameFocusableContent extends AxeAudit {
/**
* @return {LH.Audit.Meta}
*/
static get meta() {
return {
id: 'frame-focusable-content',
title: str_(UIStrings.title),
failureTitle: str_(UIStrings.failureTitle),
description: str_(UIStrings.description),
requiredArtifacts: ['Accessibility'],
};
}
}

module.exports = FrameFocusableContent;
module.exports.UIStrings = UIStrings;
4 changes: 4 additions & 0 deletions lighthouse-core/config/default-config.js
Original file line number Diff line number Diff line change
Expand Up @@ -256,7 +256,9 @@ const defaultConfig = {
'accessibility/document-title',
'accessibility/duplicate-id-active',
'accessibility/duplicate-id-aria',
'accessibility/empty-table-header',
'accessibility/form-field-multiple-labels',
'accessibility/frame-focusable-content',
'accessibility/frame-title',
'accessibility/heading-order',
'accessibility/html-has-lang',
Expand Down Expand Up @@ -516,7 +518,9 @@ const defaultConfig = {
{id: 'document-title', weight: 3, group: 'a11y-names-labels'},
{id: 'duplicate-id-active', weight: 3, group: 'a11y-navigation'},
{id: 'duplicate-id-aria', weight: 10, group: 'a11y-aria'},
{id: 'empty-table-header', weight: 2, group: 'a11y-tables-lists'},
{id: 'form-field-multiple-labels', weight: 2, group: 'a11y-names-labels'},
{id: 'frame-focusable-content', weight: 3, group: 'a11y-navigation'},
{id: 'frame-title', weight: 3, group: 'a11y-names-labels'},
{id: 'heading-order', weight: 2, group: 'a11y-navigation'},
{id: 'html-has-lang', weight: 3, group: 'a11y-language'},
Expand Down
2 changes: 1 addition & 1 deletion lighthouse-core/gather/gatherers/accessibility.js
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,9 @@ async function runA11yChecks() {
'svg-img-alt': {enabled: false},
'audio-caption': {enabled: false},
'aria-treeitem-name': {enabled: true},
'empty-table-header': {enabled: true},
// https://github.com/dequelabs/axe-core/issues/2958
'nested-interactive': {enabled: false},
'frame-focusable-content': {enabled: false},
},
});

Expand Down
4 changes: 2 additions & 2 deletions lighthouse-core/test/fraggle-rock/scenarios/api-test-pptr.js
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ describe('Fraggle Rock API', () => {

const {auditResults, erroredAudits, failedAudits} = getAuditsBreakdown(lhr);
// TODO(FR-COMPAT): This assertion can be removed when full compatibility is reached.
expect(auditResults.length).toMatchInlineSnapshot(`73`);
expect(auditResults.length).toMatchInlineSnapshot(`75`);

expect(erroredAudits).toHaveLength(0);
expect(failedAudits.map(audit => audit.id)).toContain('label');
Expand Down Expand Up @@ -147,7 +147,7 @@ describe('Fraggle Rock API', () => {
const {lhr} = result;
const {auditResults, failedAudits, erroredAudits} = getAuditsBreakdown(lhr);
// TODO(FR-COMPAT): This assertion can be removed when full compatibility is reached.
expect(auditResults.length).toMatchInlineSnapshot(`152`);
expect(auditResults.length).toMatchInlineSnapshot(`154`);
expect(erroredAudits).toHaveLength(0);

const failedAuditIds = failedAudits.map(audit => audit.id);
Expand Down
Loading