Skip to content

Commit 01d1e1a

Browse files
WilcoFierstombrunetcarlosapaduarteJym77
authored
Update implementation info (#1837)
* Update implementation info * Update earl-reports.md * update spelling-ignore * Try fixing test failures * Update earl-reports.md * Update pages/implementations/testcases.md * Apply suggestions from code review Co-authored-by: Tom Brunet <thbrunet@us.ibm.com> * Apply suggestions from code review Co-authored-by: Carlos Duarte <carlosapaduarte@gmail.com> * Update pages/implementations/earl-reports.md Co-authored-by: Jean-Yves Moyen <jym@siteimprove.com> Co-authored-by: Tom Brunet <thbrunet@us.ibm.com> Co-authored-by: Carlos Duarte <carlosapaduarte@gmail.com> Co-authored-by: Jean-Yves Moyen <jym@siteimprove.com>
1 parent d38a671 commit 01d1e1a

3 files changed

Lines changed: 68 additions & 25 deletions

File tree

‎__tests__/spelling-ignore.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -245,6 +245,8 @@
245245
- TestSubject
246246
- earl:cantTell
247247
- getComputedPriority
248+
- isPartOf
249+
- scId
248250

249251
# sizes
250252
- 1em

‎pages/implementations/earl-reports.md‎

Lines changed: 39 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,14 @@ title: Reporting Format
44

55
## Contribute An Implementation
66

7-
If you developed an accessibility tool or a testing methodology, and would like to have your implementation included in the ACT-R website, there are two ways you can do so.
7+
If you developed an accessibility tool or a testing methodology and would like to have your implementation included in the WAI website, there are two ways you can do so.
88

9-
1. If you have a tool that can return a data format, you will need to run your tests against the [ACT-R test cases](../testcases/) and [submit a report](../reporting/).
10-
2. If you use manual test methodology, where you fill results into some report template or tool, you can [Use the WCAG-EM Report Tool](../wcag-em-tool/) instead to produce implementation reports.
9+
1. If you have a tool that can return a data format, you will need to run your tests against the [test cases](../testcases/) and [submit a report](../reporting/).
10+
2. If you have a manual test methodology where you fill results into some report template or semi-automated tool, you [can use the ACT Implementor tool](https://act-implementor.netlify.app/#/) instead to produce implementation reports.
1111

1212
## Understanding the Reporting format
1313

14-
To display a accessibility test tool or methodology on the ACT-R Community website, ACT-R use the [Evaluation And Reporting Language](https://www.w3.org/TR/EARL10-Schema/) expressed using [JSON-LD](https://json-ld.org). Tool developers that have an EARL reporter should be able to provide their existing reports. If you don't already have an EARL reporter, we provided a basic data structure that you can use.
14+
To display an accessibility test tool or methodology on the WAI website, use the [Evaluation And Reporting Language](https://www.w3.org/TR/EARL10-Schema/) expressed using [JSON-LD](https://www.w3.org/TR/json-ld/). Tool developers that have an EARL reporter should be able to provide their existing reports. If you don't already have an EARL reporter, we provided a basic data structure that you can use.
1515

1616
### Context And Graph
1717

@@ -52,7 +52,7 @@ Add an `Assertion` object to the `assertions` array for each outcome provided by
5252
The following properties are required for each Assertion:
5353

5454
- `@type`: This must be `Assertion`, to distinguish it from other data types that might exist in an EARL report.
55-
- `test.title`: A title for the rule / test procedure as it is known in the implementation
55+
- `test`: See [Test Criterion](#test-criterion).
5656
- `result.outcome`: One of the following values:
5757

5858
- `earl:passed`: A node in the test case passed the rule
@@ -71,13 +71,44 @@ The following properties are required for each Assertion:
7171
"assertions": [
7272
{
7373
"@type": "Assertion",
74-
"test": { "title": "My Tool's rule title" },
75-
"result": { "outcome": "earl:passed" }
74+
"result": { "outcome": "earl:passed" },
75+
"test": { ... },
7676
}
7777
]
7878
}
7979
]
8080
}
8181
```
8282

83-
**Advanced**: All required properties map to [EARL](http://www.w3.org/ns/earl#), except for `title` and `source` which are properties of [Dublin Core](http://purl.org/dc/terms/). Property names can be anything, as long as they can be expanded to the correct URL.
83+
### Test Criterion
84+
85+
For each result, we'll need to know which rule or procedure in the tool reported it, and which WCAG success criteria were failed. To do so, on the `test` property, add an object. This must include:
86+
87+
- `title`: A string, that uniquely identifies which rule or procedure in the tool found the result.
88+
- `isPartOf`: An array of strings, listing the WCAG success criteria that fail when the rule or procedure fails. These must use the IDs of the WCAG success criteria, prefaced with `WCAG2:`. For example `WCAG2:name-role-value`.
89+
90+
```json
91+
{
92+
"@context": "https://act-rules.github.io/earl-context.json",
93+
"@graph": [
94+
{
95+
"@type": "TestSubject",
96+
"source": "https://act-rules.github.io/testcases/a1b64e/6c3ac31577c3cb2d968fc26c4075dd533b5513fc.html",
97+
"assertions": [
98+
{
99+
"@type": "Assertion",
100+
"result": { "outcome": "earl:passed" },
101+
"test": {
102+
"title": "image-button-has-name",
103+
"isPartOf": ["WCAG2:non-text-content", "WCAG2:name-role-value"]
104+
}
105+
}
106+
]
107+
}
108+
]
109+
}
110+
```
111+
112+
**Note**: The IDs of WCAG success criteria can be found in [sc-urls.json](https://github.com/act-rules/act-tools/blob/main/src/data/sc-urls.json), on the `scId` property. These are the IDs introduced in WCAG 2.1, and continued in WCAG 2.2. Use these IDs, even when reporting for WCAG 2.0.
113+
114+
**Advanced**: All required properties map to [EARL](http://www.w3.org/ns/earl#), except for `isPartOf`, `title`, and `source` which are properties of [Dublin Core](http://purl.org/dc/terms/). Property names can be anything, as long as they can be expanded to the correct URL.

‎pages/implementations/testcases.md‎

Lines changed: 27 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,11 @@ title: Test Cases
44

55
All ACT rules include a number of test cases, which are designed for easy consumption by accessibility test tools and test methodologies. Test cases are updated regularly as part of rule writing. All test cases are described in a JSON file:
66

7-
<a class='btn' href='/testcases.json'>
8-
See the Test Cases JSON
7+
<a class='btn' href='https://www.w3.org/WAI/content-assets/wcag-act-rules/testcases.json'>
8+
See the Test Cases JSON
99
</a>
1010

11-
**Note**: Based on the file extension specified for a test case within the examples code snippet (e.g.: html, svg, xhtml), the corresponding `DOCTYPE`, if applicable, is dynamically added to the generated HTML test case file.
11+
**Note**: HTML test cases are embedded in a small template which adds a `!DOCTYPE`, HTML root node with lang attribute, and head with a title. These are omitted for test cases with an HTML element or a `!DOCTYPE`.
1212

1313
## Test Case Format
1414

@@ -24,19 +24,29 @@ In the `testcases.json` file, test cases are included on the `testcases` array,
2424

2525
```json
2626
{
27-
"name": "ACT-Rules Community test cases",
28-
"license": "https://act-rules.github.io/pages/license/",
29-
"count": 558,
30-
"testcases": [
31-
{
32-
"testcaseId": "55f3ed0ec0f324514a0d223b737bc1e4c81593c7",
33-
"url": "https://act-rules.github.io/testcases/5f99a7/55f3ed0ec0f324514a0d223b737bc1e4c81593c7.html",
34-
"expected": "passed",
35-
"ruleId": "5f99a7",
36-
"ruleName": "ARIA attribute is valid",
37-
"rulePage": "https://act-rules.github.io/rules/5f99a7",
38-
"ruleAccessibilityRequirements": ["wcag20:4.1.2"]
39-
}, ... ]
27+
"name": "ACT Task Force test cases",
28+
"website": "https://www.w3.org/WAI/standards-guidelines/act/rules/",
29+
"license": "https://act-rules.github.io/pages/license/",
30+
"count": 1132,
31+
"testcases": [
32+
{
33+
"ruleId": "97a4e1",
34+
"ruleName": "Button has non-empty accessible name",
35+
"ruleAccessibilityRequirements": {
36+
"wcag20:4.1.2": {
37+
"forConformance": true,
38+
"failed": "not satisfied",
39+
"passed": "further testing needed",
40+
"inapplicable": "further testing needed"
41+
}
42+
},
43+
"expected": "passed",
44+
"testcaseId": "a4cc71b0434f71f4ea0069c409f73e0207dfb403",
45+
"testcaseTitle": "Passed Example 1",
46+
"relativePath": "testcases/97a4e1/a4cc71b0434f71f4ea0069c409f73e0207dfb403.html",
47+
"url": "https://www.w3.org/WAI/content-assets/wcag-act-rules/testcases/97a4e1/a4cc71b0434f71f4ea0069c409f73e0207dfb403.html",
48+
"rulePage": "https://www.w3.org/WAI/standards-guidelines/act/rules/97a4e1/proposed/"
49+
}, ... ]
4050
}
4151
```
4252

@@ -52,7 +62,7 @@ Correctness of an implementation is based on the results for test cases. See [im
5262

5363
## Contribute An Implementation
5464

55-
If you developed an accessibility tool or a testing methodology, and would like to have your implementation included in the ACT-R website, there are two ways you can do so.
65+
If you developed an accessibility tool or a testing methodology, and would like to have your implementation included in the WAI website, there are two ways you can do so.
5666

5767
1. If you have a tool that can return a data format, you will need to run your tests against the ACT rule's test cases and [submit a report](../reporting/).
5868

0 commit comments

Comments
 (0)