Online Employment
Applications and
Website Accessibility
September 27, 2011
Dr. Jonathan Lazar
Dept. of Computer and Information
Sciences &
Universal Usability Laboratory
Towson University
2
ADA National Network Centers
1-800-949-4232 [voice/tty]
adata.org
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
Online Employment Applications
 Employment is a core ingredient in self-esteem,
independence, and happiness
 Historically, the unemployment rate for people with
disabilities, especially people who are blind, is high
 Estimated unemployment rate for people who are blind
in US, UK, and Canada: 70-75%
 Most jobs now require an online application
 If online employment applications are inaccessible, this
shuts people with disabilities out of the job before they
can even apply
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
3
Previously published studies
 Erickson (2002) evaluated 10 job boards and 31 E-
recruiting websites for accessibility using an automated
evaluation tool—none were accessible
 Lazar, Wentz et al (2011) performed expert accessibility
evaluations on 8 job aggregator websites (e.g.
CareerBuilder), and found that 7/8 had accessibility
problems
 No previously published evaluations involve users who
are blind attempting to apply for jobs online—that was
our goal!
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
4
5
Participants
 Pilot study took place with 2 users who are blind before
the main data collection
 All 16 participants were:
 Blind
 Screen reader users unable to use magnification
 Currently either part-time employed, or unemployed,
and seeking full-time employment
 Most participants were recruited through a partnership
with the Maryland Division of Rehabilitation Services,
Office of Blindness and Vision Services
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
Participants (cont.)
 16 participants




Average age: 36.5 years, Gender: 11 female, 5 male
Average screen reader experience:12.06 years
Average Internet experience: 10.94 years
Education: 2 had HS degree, 3 had Associates
degree, 9 had Bachelors degree, 2 had Masters
degrees
 No personal information was used—each participant had
a CV and e-mail account prepared for them
 All CVs submitted were marked “not a real application—
submitted for training purposes only”
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
6
Web sites chosen
 2 online job application sites were chosen for each of the
8 states served by the Southeast DBTAC:
 Alabama
 Florida
 Georgia
 Kentucky
 Mississippi
 North Carolina
 South Carolina
 Tennessee
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
7
Web sites chosen (cont.)
 Honda Alabama
 Smithfield Packing Co.
 Pilgrims
 Lowes Inc.
 Orlando Regional
 Sonoco Products





Healthcare
Home Shopping Network
Harland Clarke Holdings
MARTA Corp.
PNC Bank
Ford Motor Company
 Fluor Enterprises
 Beau Rivage Resort and
Casino
 Hinds Community
College
 International Paper Co.
 Dell Computer
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
8
Research methodology
 All participants utilized JAWS in the evaluations
 Participants were given the URL of the homepage of the
company/organization, and told to apply for a job of a
certain category (e.g. help desk manager or software
engineer)
 A CV and e-mail account had previously been prepared
for the participant to match the job category (with
relevant work experience)
 We used a modified usability methodology so that we
could learn as much as possible about the barriers to
online job applications
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
9
Assisting participants
 For many sites, core features (such as the “search jobs”
function) were inaccessible
 When users could not move forward and told us that
they could not move forward, we offered to assist them,
and we took very careful notes of when we were asked
to perform an “intervention”
 We performed a total of 34 interventions
 2 categories of intervention:
 (18) verbal advice
 (16) performing a mouse click (when it was required)
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
10
11
Assistance
 Ideally, a user needs to apply for a job online without any
assistance
 However, we wanted to learn more about what are the
most common challenges in online job application web
sites
 And if we did not assist the users in any way, very often,
the users could not get beyond the homepage or job
search page, and we would learn nothing about the
problems that exist
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
12
RESULTS
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
13
Overall Results
 24/32 applications were submitted,
HOWEVER
 Only 9/32 attempts at applying for a job online could be
completed without assistance from the researchers
 Some participants needed as many as 3 interventions
 In usability testing research, this would be considered a
task success rate of 28.1%
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
Information on time taken to apply
 Quickest submission: 23 minutes - MARTA
 Longest submission (121 minutes) for successfully
submitted individual applications:
• Beau Rivage, no intervention
• Orlando Health, 1 intervention
 User gave up on one submission attempt (Beau Rivage)
after 229 minutes, application was not submitted
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
14
15
Software Notes
 Note that 7 of the 16 sites use either the Taleo software
or the Kenexa software for the application process
 Taleo has over 5,000 customers
 Kenexa is used in 11 languages and 115 countries
(data from their respective web sites)
 Imagine how minor changes to those two applications
could have a major impact!
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
16
Examples of specific
problems that would stop
blind users from applying
for jobs online
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
MARTA web site
Every job available had a link title of “click here to
read more” from JAWS links list
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
17
18
PNC Bank
The “search jobs” page was flash-based, with no
textual equivalents, so unless you could use a
mouse pointer, you could NOT search for jobs
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
Smithfield Foods
To find out more about jobs, you were required to
click on maps to select where you wanted to find
a job, but there were no textual equivalents, so
mouse usage was required
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
19
International Paper
To find out more about careers, you were required
to click on careers in a pull-down menu, but
there were no textual equivalents, and no way to
access it through JAWS, so mouse usage was
required
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
20
21
Sonoco
Feedback on problems with data entry is
inaccessible…you get feedback by hovering
over with the mouse, and the fields that are
incorrect are marked in red. JAWS users only
hear “Ok button. Press space bar to activate.”
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
International Paper
Feedback about the errors in form data, is not
accessible to JAWS.
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
22
Sonoco and Ford
The question is not in any way connected to the
responses in the code. While a visual user can
see that they are next to each other, this isn’t
clear for JAWS
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
23
Progress bars provide no useful
feedback
They need to also provide information non-visually
about how the user is progressing
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
24
25
Some general problems
which relate to all users, not
only users with disabilities…
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
26
Harland Clarke
Site required user to select an end date, even
though they had selected the job as their current
job.
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
Orlando Regional Health
The application process required a look-up of
school attended, rather than allowing free text
entry. However, there were 8 lookup choices
with the same name
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
27
Orlando Regional Health (part 2)
On the form, email and phone number were not
indicated as required but when the user
submitted the form, an error came up stating that
email and phone number are required
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
28
Orlando Regional Health (part 3)
 Even with textual labels of the steps in the
application process, these textual labels are
confusing:
 Preferences?
 Online questionnaire?
 Languages?
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
29
Orlando Regional Health (part 4)
 Unclear guidance on what format should be
used to submit data in a form
 This was actually requesting data be entered in
a currency format, but there was no guidance,
and the error message was unclear
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
30
31
HSN
User was confused about the difference between
“no selection” and “no”
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
32
Suggestions for Priorities in
Improving Accessibility in
Online Job Application Web
sites
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
Forms properly labeled
Make sure that forms are properly labeled
 Textual labels should be read before the entry
field/button, not afterwards
 Textual labels should actually represent the
same label as presented on the screen
This combination, with no labels, is
confusing
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
33
Required fields and feedback
Make sure that it is clear which fields are
required, and provide accessible feedback on
incorrectly filled-out forms
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
34
Option for more time
If you must have a session time limit, provide
users the opportunity to request more time
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
35
36
Label Links
Make sure that links are labeled properly
 Every job available at Beau Rivage had a link title of
“more info,” instead of the job title
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
Progress info accessible
Make sure that information on progress
through the process is available to JAWS
 Progress indicators shouldn’t only be visual, based on
color or shape
 It’s important to provide textual information
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
37
Proper coding of table headers/
table cell
Type 1:
Tables that require data
input – all form controls
(drop-down lists, radio
buttons, combo boxes, etc.)
should be properly labeled.
Type 2:
Tables displaying data –
here, table headers must be
properly coded/labeled so
that each row and cell could
be read by Jaws.
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
38
39
Logical tab order
Make sure that there is a logical tab order if
you are listening to forms
 When tab order is not logical, here’s what users hear
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
40
Suggestions for success
 If you are building software for online application
processes, place accessibility in the design requirements
and specify evaluation
 If you are purchasing a software package for online
applications, ask if it’s accessible, and ask for data on how
they tested for accessibility
 Evaluate your online application process by:
 Usability testing involving people with disabilities
 Expert inspections (but only with real experts)
 Using automated software tools for feedback
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
41
Open Discussion
&
Questions
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
*
42
THANK YOU!
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
43
Contact Information
Please Feel Free to Reach Out to Us
Dr. Jonathan Lazar
Dept. of Computer and Information Sciences & Universal
*
Usability Laboratory, Towson University
JLazar@towson.edu
Jill Houghton
Deputy Director, Southeast TACE
Acting Executive Director, USBLN
Jilldh@bellsouth.net
Pamela Williamson
Director, DBTAC: Southeast ADA Center
sedbtacproject@law.syr.edu
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
44
For more information
ADA National Network by DBTAC
Web: adata.org
1-800-949-4232 [voice/tty]
Section 508 web site
Web: www.section508.gov
Web Content Accessibility Guidelines
Web: www.w3.org/WAI/intro/wcag.php
Web AIM (Accessibility in Mind)
Web: webaim.org
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
45
Sponsors of Webinar Series
ADA National Network by DBTAC
Web: adata.org
American Association of
People with Disabilities (AAPD)
Web: aapd.com
Council of State Administrators of
Vocational Rehabilitation (CSAVR)
- National Employment Team (NET)
Web: rehabnetwork.org
U.S. Business Leadership Network
Web: usbln.org
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
46
Education Credits - DEADLINE: October 7
CEU (0.1)
Approved by the University College at Syracuse University
To earn CEU credit, you must:
1. Participate in the 1.5 hour webinar.
2. Login to your Webinar Account & complete: CEU Request.
3. Follow CEU Credit Requirements
(http://sedbtac.org/webinars/requestCEU.php?eventid=533).
CRCC (1.5)
Approved by Commission on Rehabilitation Counselor
Certification (CRCC)
To earn CRCC credit, you must:
1. Participate in the 1.5 hour webinar.
2. Login to your Account & score 80% or better on Post Test.
3. Complete: CRCC Request and send a copy to the CRCC office.
4. Follow CRCC Credit Requirements
(http://sedbtac.org/webinars/requestCRCC.php?eventid=533).
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved
47
Disclaimer
The DBTAC: Southeast ADA Center (Southeast DBTAC) is
authorized by the National Institute on Disability and Rehabilitation
Research (NIDRR) to provide information, materials, and technical
assistance to individuals and entities that are covered by the
Americans with Disabilities Act and its Amendments (ADA) under
Grant No. H133A060094. However, NIDRR is not responsible for
enforcement of the ADA. For more information or assistance,
contact your regional ADA Center via the ADA National Network by
DBTAC at adata.org or call 1-800-949-4232 [voice/tty].
The information, materials, and/or technical assistance are intended
solely as informal guidance, and are neither a determination of your
legal rights or responsibilities under the Act, nor binding on any
agency with enforcement responsibility under the ADA. The Burton
Blatt Institute (BBI) at Syracuse University does not warrant the
accuracy of any information contained herein. Any links to non-BBI
information are provided as a courtesy. They are not intended to
nor do they constitute an endorsement by the BBI of the linked
materials.
Webinar Series for Business
Hosted by: Southeast ADA Center and SoutheastTACE
Projects of Burton Blatt Institute at Syracuse University
Funded by NIDRR Grant # H133A060094. © 2011 All Rights Reserved