Product Accessibility Assessment
Prioritizing and Making
Recommendations in an Imperfect
World
http://go.ncsu.edu/ahg2012-procurement
Presenters
Greg Kraus
University IT Accessibility Coordinator
North Carolina State University
greg_kraus@ncsu.edu
Mark Turner
Director, Center for Accessible Media
CSU, Office of the Chancellor
mturner@calstate.edu
Accessibility Throughout the
Procurement Process
• Two institutions process for incorporating
accessibility at key points in the procurement
process
– North Carolina State University
• Synchronous Learning Management System
– California State University System
• Digital Rentals
Topics Covered
•
•
•
•
•
•
Prioritizing finite resources on key IT products
Developing contract language
Defining vendor responsibilities
Conducting accessibility evaluations
Factoring accessibility into vendor selection
Implementation planning for inaccessible
products
NC State University
Purchasing – Instructions to Bidders
• Language specifies “Enterprise-Level IT Systems or
Technologies”
• Vendors must:
– Assure all features, fully comply with Section 508 OR
– Detail why and in what way any feature does not fully comply
with Section 508
– If VPAT is submitted it must include compliance checklists for:
Technical Standards, Function and Performance Criteria,
Documentation and Support
– The product is subject to an accessibility evaluation by the
University.
• http://www.fis.ncsu.edu/materialsmgmt/purchasing/vendo
r/bid_instruct.pdf
The Problem Isn’t Policy, It’s Process
Starting the Process
• Three ways to incorporate accessibility into
the procurement process
– Business process
– Personal Relationships
– Listening (being nosy)
Prioritization – Learning About The
Technology
• What does this software do?
• VPAT and other accessibility documentation?
• Is the technology part of a required academic or
institutional purpose?
• How many people are expected to use this technology?
• What is the scope of access to this technology? Does the
technology have any publicly available access or is it limited
to a set of users?
• Is the primary audience going to include people with
disabilities?
• Is the technology likely to be reused in the future or used
for an extended period of time?
Prioritization – Digging Deeper
• List out all of the functions of this technology,
indicating:
– Other Commercial or Free Solutions?
– Comparable Alternatives Already on Campus?
– Essential to a Business Need (Yes or No)?
– Accessibility Problems?
• Examples from SLMS
– Video conferencing
– Application sharing
Defining Your Business Needs
• What educational or business needs are you
trying to meet?
• In other words, what problem are you trying to
solve?
• Rank your needs as “Essential, High, Medium,
and Low”
• Include accessibility as one of your Essential
requirements
• http://oit.ncsu.edu/itaccess/ict-accessibilityregulation-faqs#procurement
Accessibility is not the quintessential
criterion in product assessment
Our Search For a New SLMS
• Had been an Elluminate Live! customer for
years
• Moved to Blackboard Collaborate for the short
term
• Time to reassess the market
• (This presentation is not a detailed overview
of the accessibility of SLMS systems. It is a
detail of the process we went through.)
Software Considerations From Users
and Support
• Cross Platform
• Support (Help Desk)
• Similar features to those currently available/highly used in
Elluminate
• Moodle plug-in
• Hosting
• Pricing/Cost (including $ and staff resources)
• Accessibility
• System wide licensing (unlimited users/access)
• Recording
• Migration of current recorded content
• Ease of transition (cost to us in staff hours)
• Mobile access
Essential Features
•
•
•
•
•
•
Text chat
Audio chat
Present a PowerPoint presentation
Recording
Cross-platform/cross-browser support
Does not require software installation for
participants
• High or no limit on participants per session
• Accessibility
• Adoption at other universities
Initial Products
•
•
•
•
•
•
•
•
•
Blackboard Collaborate
DimDim
Adobe Connect
WebEx Meeting Center
WebEx Training Center
GoToMeeting
Big Blue Button
Saba Centra
Microsoft LYNC
(teleconference)
• iLinc
• Polycom CMA-D
(teleconference)
• Skype (people may ask)
• Google+
• CCC Confer
• LiveOn
• FastViewer
• omNovia
• Merit WebConnect
• Talking Communities
Passed the Initial Evaluation
• Largely using accessibility (brief testing) and
usage at other campuses, we narrowed the list
to
– Blackboard Collaborate
– Adobe Connect
– WebEx Meeting Center
– WebEx Training Center
– Saba Centra
– iLinc
How the Finalists Were Determined
• Survey of Elluminate users – what they use
• Accessibility options for users with disabilities
• Benchmarking from other regional
universities’ SLMS search results
Three Finalists
•
•
•
•
Blackboard Collaborate
WebEx Training Center
Saba Centra
Accessibility was not the overriding factor in
this step
– One candidate with accessibility support was
eliminated for other reasons
The Final Rubric
• 14 meta tasks we tested
– Which was broken down to approximately 80
tasks
• Ranked each task in importance and
frequency of use from 1 to 3 (3 being highest
or most frequent)
Meta Tasks We Tested
•
•
•
•
•
Create a meeting
Join a meeting
Audio
Chat Tool
Personal Response
Tools
• Whiteboard
• Polling
•
•
•
•
•
•
•
Web Tour
Session Management
Breakout Rooms
File Transfer
Video
Application Sharing
Other
Sample of Ranking of Task Importance
• 3 (Most Important)
– Text chat
– Audio chat
• 2 (Moderately important)
– File transfers
– Room timer
• 1 (Least Important)
– Rearrange slides while in session
– Participate in the session anonymously
Sample Detailed Tasks
• Chat Tool
– Send a chat message
– Read previous chat messages
– Determine when a new chat message appears
– Switch between chat rooms
– Send a private chat
– Read a private chat
Technology Used in Accessibility
Testing
•
•
•
•
•
•
Keyboard-only input
JAWS screen reader
VoiceOver screen reader
ZoomText screen magnifier and reader
Built-in OS contrast tools
Dragon Naturally Speaking
At the end of the day…
• …nothing was perfect
• Every product had failures in the “Essential”
category
• Do you just add up the number of successful
functions and go with the product that was
the most successful?
• No, this is where you earn your keep as an
“accessibility expert”
Conclusion
• “Of the three products tested, Blackboard
Collaborate provides the most accessible user
experience. None of the products are perfectly
accessible, and Saba Centra and WebEx Meeting
Center are accessible to a degree, however,
because of the level of accessibility support
present in Collaborate, a strong business case
would need to be made to choose either Centra
or Meeting Center over Collaborate from an
accessibility perspective.”
Elaborating on the Conclusion
• “It is noticeable that significantly more
thought has gone into the user experience of
Blackboard Collaborate than the other
products from an assistive technology user’s
perspective. Rather than finding an easy way
for a user to simply achieve a basic level of
functionality, Blackboard Collaborate seems to
consider the optimal way for an assistive
technology user to use the system.”
Final SLMS Report
• https://wolfware.ncsu.edu/sops/slms_search/
SLMSRecommendationReport.pdf
Planning for Problems
• Optimizations
• Workarounds
• Accommodations
Optimizations And Workarounds?
• Optimizations
– actions the author or institution can take to make
the technology more accessible
• Workarounds
– actions the end-user can take to make the
technology more accessible
Emerging Technology
• “NC State will support the consideration of using emerging
technologies as they become available or prior to purchase, while
also advocating for their accessibility from their inception and as
they are further developed.”
• You can use (potentially inaccessible) emerging technologies if
– The learning objective can only be met by using the emerging
technology, OR
– No one in your group needs an accessible technology, BUT you must
have a plan in place for providing an equally effective and integrated
experience for all users
• http://policies.ncsu.edu/regulation/reg-04-25-05 (1.3)
• http://oit.ncsu.edu/itaccess/ict-accessibility-regulationfaqs#emerging-technologies
Sample Optimization and Workaround
• Google Apps
– Optimization
• Email notifications for new calendar events by default
– Workaround
• Google Doc Download Tool