Vivienne Conway, Dr Justin Brown
Edith Cowan University, Perth W.A.
OZeWAI 2011, Melbourne, Vic.
3 Stages of Website Evaluation
First Stage
Preliminary Review
Second Stage
Compliance Check
Third Stage
Audit and Accreditation
Preliminary Review of Websites for
Accessibility – either for internal
audit or preparation for
conformance evaluation of
audit/accreditation
Conformance Evaluation of
Websites for Compliance with
Accessibility Guidelines (WCAG
2.0) A, AA, or AAA
Full technical audit with or without
accreditation, which includes user
assessment
copyright V.Conway & J.Brown,
2011
First Stage
•Preliminary Review
Preliminary Review of Websites for
Accessibility – either for internal
audit or preparation for
conformance evaluation of
audit/accreditation
Skills/Experience Required
•Basic HTML knowledge
•Need to know how to ‘view source’ and what they are looking for
•Basic CSS knowledge
•What should be in CSS, what should be in the HTML etc.
•Basic knowledge of online tools
•CMS editing, ability to turn off CSS, Accessibility tool bar etc.
•Familiarity with guidelines (e.g. WCAG 2.0 )
•Basic knowledge of requirements for people with disabilities and how
they use the Internet
•Basic knowledge of what Assistive Technology is used by people with
disabilities and how they use it.
•E.g. JAWS or NVDA
copyright V.Conway & J.Brown,
2011





1. Initial Visual Inspection
tab through the document from the top to see if
you can access all the controls and elements
without using the mouse (e.g. Prime Minister’s
Contact page: http://www.pm.gov.au/contactyour-pm - You can’t enter your State via the
keyboard)
see if any skip links are visible with keyboard
focus and use them to see if they are active
look to see if there are items where it is difficult
to locate the keyboard focus and then try it with
a mouse to see if the focus is different with a
mouse
look to see if the tabbing order is logical
copyright V.Conway & J.Brown,
2011



see if images have alternative text (hover
over) and if the choice of text makes sense
(not dependent upon surrounding text)
check the link text to make sure that it is
unique and that it is not generic such as ‘click
here’. Make sure that if a general term such
as ‘read more’ is used, that it is within the
paragraph and not on a separate line.
use the mouse to see if the items receiving
focus are shown differently with the mouse
than with keyboard controls
copyright V.Conway & J.Brown,
2011

Colour

Look at the use of colour on the page:
◦ does there appear to be sufficient colour
contrast?
◦ are there any occurrences where colour alone is
used to show links or other features? Are
additional cues such as underlining used?
◦ do the borders on any controls such as buttons
have sufficient colour contrast


Text Size
Can you increase the text size either with Ctrl++,
or zoom, to 200% without having to scroll side to
side to see the text
copyright V.Conway & J.Brown,
2011



Forms
If there are forms, can you tab through them
and select items from any drop-down lists
with the down arrow or space bar for radio
buttons and check boxes
If there are forms, enter incorrect information
to ensure appropriate error message is given.
Also, if the information is of a financial or
legal nature, are you given an opportunity to
change the information before submitting?
copyright V.Conway & J.Brown,
2011


Links and Keyboard Traps
Follow the links and see if they take you to a new
window or tab. If they do, can you get back to the
home page by either:
◦
◦
◦
◦
◦





a home page link or breadcrumb
the back arrow
a common approach such as Alt+F to exit a new window
If not, is there an explanation as to how to get out
Does the link tell you it is opening a new window?
Are there any links to PDF Files without alternatives
and/or warnings?
When you click on a link, does it do anything
unexpected without warning the user?
Tables
If there are any tables, can you tab through the table in
a logical or horizontal manner?
Is the table used for tabular information or merely for
layout purposes?
copyright V.Conway & J.Brown,
2011








Graphic Images
Are there any graphic CAPTCHA’s? If so, is a
non-graphic choice provided?
Are there any animations, self-starting slide
shows/banners, audio? If so, are there
provisions to pause, stop or hide the images?
Are there any images of text? If so, if they are
not logo’s or similar, do they have alternative
text?
Multimedia Content
If there are any videos, are they captioned and is
a transcript provided?
Are there transcripts for audio-only content?
If an item is provided in video-only format is it
captioned, and is an alternative version provided?
copyright V.Conway & J.Brown,
2011
Second Stage
•Compliance Check
Conformance Evaluation of Websites
for Compliance with Accessibility
Guidelines (WCAG 2.0) A, AA, or
AAA
Skills/Experience Required
•Intermediate HTML and CSS knowledge
•Need to know how to fix some problems such as associating a label
with a form control
•Intermediate knowledge of online tools
•More advanced use of the CMS, ability to assess the results of
automated accessibility tools,
•Greater experience with guidelines (e.g. WCAG 2.0 )
•Ability to understand W3C documents such as “How to Meet...”,
“Success Criteria”, “Sufficient Techniques”, “Failures”
•Manual assessment skills to check compliance with Guidelines
•Intermediate knowledge of requirements for people with disabilities and
how they use the Internet
•Intermediate knowledge of what Assistive Technology is used by people
with disabilities and how they use it. E.g. JAWS or NVDA, voice-activation
software, etc.
copyright V.Conway & J.Brown,
2011




Only start this step after completing all of the
Preliminary Check items.
Use an automated tool (or preferably 2 of these
tools) that checks every page and can be
configured for the level you require e.g. WCAG
2.0 AA
Check the results of the tools – look for trends.
Whole website scan tool can locate problems you
might miss manually.
◦ E.g. SortSite is good for locating areas where there is an
ONCLICK without an ONKEYPRESS complementary
command – can create a keyboard trap
copyright V.Conway & J.Brown,
2011

Manually assess individual pages that are:
◦ A template
◦ Typical for the website
◦ Include forms or other user interaction such as
‘Contact Us’ forms, registration forms etc.
◦ Contain images and multimedia elements
◦ Form part of ‘Complete Processes’ – a series of
steps that need to be completed in order to
accomplish an activity – such as a registration or
shopping cart activity
◦ Form part of a ‘Key Functionality’ can be generally
regarded as important functionality for users or
visitors
copyright V.Conway & J.Brown,
2011



How do you do this?
Use the WCAG 2.0 Guidelines, removing any
from a level which is not within the scope i.e.
AAA if that’s not part of your requirement.
Use the Guidelines, Sufficient Techniques,
Advisory Techniques and Failures to guide
your assessment
copyright V.Conway & J.Brown,
2011

Use automated tools on these single pages to
assist you, for example:
◦ aDesigner to check for vision problems – good for
locating fixed and too small font sizes, finding
redundant text information, reading order
◦ Web Accessibility Toolbar (WAT) to check for
 code validation, structure, forms/fieldset, colour
contrast, view images for alt text and remove image to
see if what remains meets requirements, list links to
see if text is appropriate, look for language
declaration, titles, skip links etc.
copyright V.Conway & J.Brown,
2011

Screenreader (e.g. JAWS, NVDA)
◦ tab through the document to make sure all
elements are read out
◦ Access the controls (without mouse) and make sure
there are no keyboard traps
◦ Check the captions for video, audio transcriptions,
images for see if all necessary alternatives are
keyboard accessible
copyright V.Conway & J.Brown,
2011

WAVE (Firefox extension or from WAT)


FIREBUG – Firefox extension




Structure, forms/labels, alternative text, lists
An additional tool that you can use to check other results
and locate line number and coding
TotalValidator
T.A.W. (now has a Beta version of the WCAG 2.0 tool
but only for the online feature)
Functional Accessibility Evaluator (FAE) from the
University of Illinois, now checks to WCAG 2.0
copyright V.Conway & J.Brown,
2011





If you claim a website meets WCAG 2.0 AA, you
are saying that it meets all of A and AA for every
page.
You can make a Statement of Partial
Conformance due to third-party content outside
of your control
Need to insert a list of web content technologies
relied on
Conformance statement can be included in the
metadata
Examples:
http://www.w3.org/TR/UNDERSTANDINGWCAG20/conformance.html#uc-conformanceclaims-head
copyright V.Conway & J.Brown,
2011
Third Stage
•Audit and Accreditation
Full technical audit with or without
accreditation, which includes user
assessment
Skills/Experience Required
•Intermediate-advanced HTML and CSS knowledge
•knowledge of why coding is causing a problem
•how to find the error or problem in the code (understanding
site structure and interaction between html, CSS and JavaScript
files)
• knowledge of what needs to be done to correct the problem
•Intermediate knowledge of online tools
•Ability to use both manual and automated assessments along
with use testing to audit/whole website standard
•Expert understanding of guidelines (e.g. WCAG 2.0 )
•Ability to apply guidelines including Sufficient Techniques to
make critical audit decisions on compliance
copyright V.Conway & J.Brown,
2011
•
Expert ability to test and award
compliance/accreditation status
• Includes reputation, skill and ability to justify level
awarded
•
•
•
Manual assessment skills to check compliance
with Guidelines
Intermediate knowledge of requirements for
people with disabilities and how they use the
Internet
Intermediate knowledge of what Assistive
Technology is used by people with disabilities
and how they use it. e.g. JAWS or NVDA, voiceactivation software, etc.
copyright V.Conway & J.Brown,
2011




You are putting your reputation on the line
and saying that this website meets all of the
requirements for the level
Need to use a reputable accreditation
procedure– not just copying and pasting the
W3C Checkmarks
Meet the W3C Conformance Claim
requirements
Sampling Methods – under review by
Evaluation Methodology Task Force – two
choices
copyright V.Conway & J.Brown,
2011

Use of individual or team approach
◦ Paper by Giorgio Brajnik which looked at the
number of novice assessors compared with expert
assessors – team of 3 experts will usually pick up
all errors compared with 14 novices

Involving people with disabilities
◦ Highly recommended by W3C, although not a
requirement
◦ Can pick up items which other methods do not and
also can make some good usability
recommendations
copyright V.Conway & J.Brown,
2011





Items which cause an automatic failure for
any page: (All A level Requirements)
1.4.2.
2.1.2.
2.2.2.
2.3.1.
Audio Control
No keyboard Trap
Pause, Stop, Hide
Three flashes or Below Threshold
copyright V.Conway & J.Brown,
2011





WCAG 2.0 Guidelines
How to Meet WCAG 2.0
Conformance Statements:
http://www.w3.org/WAI/WCAG20/quickref/O
verview.php#conformance-reqs
Statement of Partial compliance:
http://www.w3.org/TR/2008/REC-WCAG2020081211/#conformance-partial
Web Accessibility National Transition
Strategy:
http://www.finance.gov.au/publications/wca
g-2-implementation/index.html
copyright V.Conway & J.Brown,
2011



It is important to understand how the
different tools work
Where you need to use the tool and the
purpose for the evaluation
3 Classes:
◦ Local machine – domestic, client-side
◦ Cloud-based – they do the work and send results
◦ Enterprise e.g. IBM Rational Policy Tester
copyright V.Conway & J.Brown,
2011

Client-side tools: e.g. Total Validator, SortSite
◦ Easy to use
◦ Have difficulty with proxy and firewall settings
◦ Ties up the machine and connection while crawling
the site
◦ Can run in the domain (for internal auditing)
◦ Can’t have a static site – needs to use a resolvable
IP Address
copyright V.Conway & J.Brown,
2011

Cloud-based tools
◦ They do the work and send the results to you
◦ Costly - # pages per month within a flat fee
◦ One said I’d need to pay for additional servers to do
my PhD research
◦ Duty-Cycle – how many pages can they handle per
month – some large organisations in Australia have
sites with more than 500,000 pages and numerous
CMS’s – not possible to crawl it in one go
copyright V.Conway & J.Brown,
2011

Enterprise
◦
◦
◦
◦
◦

E.g. IBM Rational Policy Tester
Large overhead - cost, server etc.
Has to have a dedicated web server set-up
Expensive
Large commitment to training
Black-boxing a website
◦ Necessary if you need to prove the state of the
website when it was audited
◦ Captures and mirrors the web pages for proof and
compliance claims
copyright V.Conway & J.Brown,
2011

Forms:
◦ Does not have a label, but has a title (Passes 2.4.6
via H65)
◦ Doesn’t have either (fails G131: 1.3.1, 2.4.6, and
3.3.2)
◦ Has a label which is not programmatically
determinable (e.g. Not linked with a ‘label for’
instruction) – F68 (fails 1.3.1. and 4.1.2.)
◦ See H44 (1.1.1., 1.3.1., 3.3.2, and 4.1.2): Using
label elements to associate text labels with form
controls
◦ Also need to look at Guideline 1.3.1. (H71): Groups
of form controls using fieldset and legend elements
copyright V.Conway & J.Brown,
2011


Current debate as to whether you can actually
create an accessible CAPTCHA
Prime Minister’s website:
◦ You don’t know there is a CAPTCHA there until you
try to submit a comment to the PM via the Contact
page: http://www.pm.gov.au/contact-your-pm
◦ Turn off CSS and you will see:
copyright V.Conway & J.Brown,
2011



Web Accessibility Tool Bar – Vision Australia
or Paciello Group:
http://www.paciellogroup.com/resources/wa
t-ie-about.html
aDesigner: free download, single page for
blind or low-vision from eclipse (DOM):
http://www.eclipse.org/actf/downloads/tools
/aDesigner/
Total Validator: free download for basic
assessment or paid versions:
http://www.totalvalidator.com /
copyright V.Conway & J.Brown,
2011



WAVE: from WebAim free extension for
Mozilla Firefox, does single page
http://wave.webaim.org/ , link for FireFox
add-on https://addons.mozilla.org/enUS/firefox/addon/wave-toolbar/
FireBug: free extension for Mozilla Firefox,
single page http://getfirebug.com/
SortSite: low cost – does whole website: (free
trial – download, online, basic and
professional versions)
http://www.powermapper.com/
copyright V.Conway & J.Brown,
2011




JAWS: screen-reader: from Freedom Scientific
(free 40 minute-mode)
http://www.freedomscientific.com/download
s/jaws/jaws-downloads.asp
NVDA: screen-reader (free download)
http://www.nvda-project.org/wiki/Download
TAW3online (free online tool for WCAG 2.0
Beta version):
http://www.tawdis.net/ingles.html?lang=en
Functional Accessibility Evaluator (FAE) from
University of Illinois: free online tool:
http://fae.cita.uiuc.edu/
copyright V.Conway & J.Brown,
2011




HTTrack Website Copier for black-boxing a
website available free from
http://www.httrack.com/
YouTube clips worth watching:
Phillip Patston, Director of Diversity New
Zealand:
http://www.youtube.com/watch?v=2F2yiF0i3
Go&feature=youtu.be
Karen Plimmer, Adaptive Technology
Instructor
http://www.youtube.com/watch?v=g6kEWKp
G9bo&feature=related
copyright V.Conway & J.Brown,
2011

Decide why you want to evaluate a website
and that will help you decide what approach
to use
◦ Are you ‘just checking’ prior to having a more
thorough audit completed?
◦ Are you needing to make a compliance statement
for your organisation or to include in your
accessibility statement?
◦ Are you completing an audit and/or accreditation?

Choose tools and approaches that suit your
task.
copyright V.Conway & J.Brown,
2011
Use the
keyboard to
navigate
through the
page
Headphones to
listen to the
page via a
screenreader
A computer with
a fast and
reliable network
connection
Time!!!
This is not a
quick and easy
task
Perspiration
and patience
copyright V.Conway & J.Brown,
2011





Vivienne Conway
V.conway@ecu.edu.au
V.conway@webkeyit.com
Dr. Justin Brown
J.brown@ecu.edu.au
copyright V.Conway & J.Brown,
2011