Building Accessibility Auditing into
Online Course Development
Korey Singleton & Kara Zirkle, Assistive Technology Initiative
2015 Accessing Higher Ground Conference
What we’ll cover
•
•
•
•
•
Our Office
Online Course Development @ Mason
Integrating Accessibility
Learn from our Experiences
Q&A
Mason, ATI Mission, Services, Staffing
ABOUT US
George Mason University
• Enrollment (Fall 2014) – ~34k students
• Faculty
– 1246 FT Instructional faculty
– 169 FT Research faculty
– 383 PT faculty FTE
• Campus locations
– US: Fairfax, Arlington, Prince William (Manassas, VA), and Loudoun (Sterling, VA)
– International: Songdo (Korea)
• More info…http://about.gmu.edu/
ATI’s Mission
…to ensure equivalent access to
electronic and IT resources
for members of the Mason community...
ATI Services
• Accessible Text (e-text conversion/OCR) & Media
– Provision of accessible text to students, faculty, and staff with print-related disabilities (referral only).
– Provision of accessible media: closed captioning and audio description.
• Web Accessibility
– Section 508/Web Accessibility Training and Support for Mason employees and students.
– Web Accessibility Testing for all Mason websites and web-based resources used in the classroom.
– Policy and Procurement Review for Accessibility
• Assistive Technology Assessments, Support, and Training
– Informal assistive technology assessments and trainings for Mason students, staff, and faculty (walk-ins and referrals).
– Maintenance of Assistive Technology Labs on all campuses.
ATI Staff & Reporting Structure
ATI operates under Compliance,
Diversity, and Ethics Office
• Reports up through ADA Coordinator
to VP for CDE, who reports directly to
University President
• More information about us available at
http://ati.gmu.edu
VP of Compliance,
Diversity and Ethics
(CDE) – Julian Williams
Ruth Townsend,
Associate Director of
CDE and ADA
Coordinator
Korey Singleton,
Assistive Technology
Initiative Manager
Kara Zirkle, IT
Accessibility
Coordinator
Courtney Shewak,
Accessible Media
Coordinator
Program Support
Specialist, Stephanie
Robbins
PT Accessible Media
Specialist, Nancy Borck
Student Worker
Student Worker
Critical Moments in E-Learning @Mason
2010
• Participation
on DE Council
• SACS
Accreditation
(2009)
2011
2012
• Faculty
• Bb/Bb
Development
Collaborate
Workshops for • Supplemental
new online
Applications
teaching
faculty (IDs,
Library,
Faculty)
• AccMedia
Pilot
2013
2014
• ITAG
• Implementat
ion of ITAG
• Library Liaison
Recs
(Accessibility)
• Doc
Accessibility
Pilot
2015
• DE Pilot
Background, Stakeholders, Existing Models
UNDERSTANDING ONLINE COURSE
DEVELOPMENT AT MASON
Background
• Mason’s Office of Distance Education was
established in early 2010 with the hiring of
a new Associate Provost for Distance
Education
– Academic Units operated independently for years
– Transition toward new senior leadership
• Office’s early years focused on:
–
–
–
–
Growing staff
SACS Accreditation (Strategic partnerships)
Establishment of policy’s and procedures
Encouraging faculty members and academic
programs
•
Strategic Partnership (ATI and DE Office)
–
–
–
–
Associate Provost was an internal hire
Asked by DE to assist with SACS accreditation
Asked by DE to participate on hiring committees
Asked DE Staff to participate on IT Accessibility
Committees
Stakeholders
Academic
Units
•
University has increased number of
online courses and programs by ~20%
per year over the past 3 years (DE
Director, Personal Communication, May
2015).
•
Key Offices:
Office of
Distance
Education
(DE Office)
Instructional Design
Team (ID Team)
– DE Office (Provost’s Office)
– ID Team (LSS/IT Office)
– Academic Units (Colleges/Schools)
Existing Online Course Development Models
• 4 “P”s – Proposal, Production, Pilot, and Portfolio (1-year)
• OCDI – Online Course Development Institute (6-week)
• Academic unit-specific Initiatives
The 4P Process
•
•
•
•
•
•
Largely driven by DE Office
Each faculty member provided one-on-one
ID support over course of academic year
6-week Readiness Reviews
Course Portfolio Review Process after
completion of pilot.
End result is a fully developed course
Slowly phasing out
Proposal
(Semester 1)
Portfolio (End of
Semester 4)
Production
(Semesters 2 & 3)
Pilot
(Semester 4)
6-week readiness
review
OCDI
• Largely driven by LSS/ID Team
• Faculty/staff participate in a 6week asynchronous cohort
• Viewed as a more scalable going
forward.
• End result is one fully developed
module.
ATI’s Role, How it has Evolved, Next Steps
INTEGRATING ACCESSIBILITY INTO THE ONLINE
COURSE DEVELOPMENT PROCESS
Baseline Design Considerations
for Online Courses
• Visual:
– Provide alternative text descriptions for all meaningful graphics (images, charts, graphs, SmartArt,
objects)
– Provide descriptions for videos where visual content is important to understanding subject matter.
– Use styles in Office documents, headers to mark-up tables or frames (for websites)
– Choose applications that support keyboard navigation and are compatible with screen readers
• Hearing:
– Provide captions for all videos
– For audio, provide transcripts
• Cognitive, Neurological:
– Use consistent navigation, tab order, appropriate language level
ATI’s Role in Online Course Development Process
How we integrate with the 4P Process…
– DE Orientation Trainings
– 6-week Readiness Reviews
– Course Portfolio Review Process
How we integrate with the OCDI…
– One week of the training is dedicated solely to integrating accessible design
practices
Issues with Accessibility in Existing Models
• 4 “P” Process: Out of the 30 items measured on the DE Office’s Course Portfolio
Rating Sheet, “The course employs accessible technologies or strategies…” was
the lowest scoring item in both Fall 2014 and Spring 2015 (Smucny, 2015a;
Smucny, 2015b).
• OCDI: Started in Spring 2015, but only looks assists with development of one
module of an online course. Continuation with ID is voluntary after that.
Accessible design practices are integrated into training process, but there is little
follow-up beyond 6-week module.
Pilot Projects
• Pilots have played a significant role in the integration of
accessibility in the online course development process.
• Today, we will discuss today:
– Accessible Media
– Document Accessibility
– DE Course Portfolio Reviews
Current Process, Reporting, Experiences
ACCESSIBLE MEDIA
Moving Beyond the Pilot Project...
Sept 2011 – Launched the Pilot Project!
Jan 2012 – Service available to entire Mason community
Over the past 3-1/2 years, we experienced a host of issues and have used that to refine this service
in several ways:
–
–
–
–
–
–
Identified our strategic partners
Updated our media platform
Streamlined workflows
Streamlined staffing
Streamlined costs
Using targeted marketing
Updated Accessible Media Workflow (Spring 2015)
UPLOAD FINAL VIDEO AND
SRT FILE TO YOUTUBE or
KALTURA, THEN SEND
LINK(S) OR PUSH VIDEO
Outsource to 3rd party
(automatic for videos
over 15 minutes or
immediate need)
EDIT WITH YOUTUBE
(Primary)
YOUTUBE/KALTURA
(Primary)
ACCMEDIA COORDINATOR
PREPS FILE
EMAIL REQUEST SUBMISSION RECEIVED VIA ATI WEBSITE
Updated ATI Website
Website: http://ati.gmu.edu
– Usability-group tested
– Easier navigation
– 2 important updates…
Streamlined Services Request Process
Captioning and Audio Description
• ATI’s Accessible Media Page - http://ati.gmu.edu/accessible-media
• Do-It-Yourself (Audio Description) – YouDescribe.org
• Request services using either:
• Accessible Media Request Form
• Bulk Request form
Captioning/Audio Description Cont...
• Turnaround time:
•
•
Accommodation – 7 days (faster if needed)
Compliance – may take longer, depends on priority
• Primary: Streaming through Kaltura, YouTube
• Managing expectations...
Accessibility of Media Players
• Bring up Website
Outsourcing Considerations
Vendors we have used
– 3PlayMedia, Ceilo24, eScribeSolutions, DocSoft, Automatic Sync Technologies,
CloudFactory (SpeakerText), Audio Description Associates (AD)
Tips when outsourcing:
– Ask about cheaper per minute costs for longer turnaround times.
– Bulk purchases = Reduced per minute rates
– Prioritize vendors that sync with institution’s video management platform, if
applicable
– Transcription alone is cheaper than transcription + synchronization
Managing Audio Description
1. Outsource
–
–
Very expensive – Can be as much as $2000 per video minute
Is it necessary?? – Depends….few actual requests in 4-year period
2. In-house
–
–
YouDescribe - Audio Description Crowd-sourcing tool for YouTube videos: http://youdescribe.org
Examples:
• Kevin Hart/Jimmy Kimmel - http://youdescribe.ski.org/player.php?prefer_d=clairemahany&v=OPdbdjctx2I
• MJ Thriller - http://youdescribe.ski.org/player.php?prefer_d=Laurae&v=sOnqjkJTMaA
Managing Library Resources:
Library’s Streaming Media Policy
•
•
•
•
•
At the request of the ATI, the library will create a streaming version of an media items that
already incorporate open- or closed-captions.
Access vs. Copyright – The library will defer to the ATI’s request for equivalent access to library
resources.
Library will seek the incorporation of captions, subtitles, etc. during procurement process.
For titles already in library’s collection, library will reach out to vendor to make their resources
accessible.
When requested to do so by the Assistive Technology Initiative or another unit, the library will
check its holdings for the presence of captions. If existing library catalog records do not
accurately reflect the availability of captions, the library will attempt to update these records.
Making Use of Data!
•
•
•
•
•
•
Date of request
Date needed
Material Title
Person making request
Instructor
Reason for Request
– Accommodation, Compliance (DE, F2F, Web)
•
•
•
Name##
Course
College/School##
Length of Media (minutes)*
•
•
•
•
•
Captioner/Vendor
Date given to Captioner
Date Job completed
Costs per video**
Location of media
– Kaltura, YouTube
•
Delivery method
– Transcripts via email, YouTube
•
•
Delivery Date
Comments
** - Very important for strategic planning
purposes/grants/etc. (Avg. video length, avg.
video cost, # of videos, projected costs)
## - Targeted marketing initiatives
Making Sense of the Numbers!
•
Over 200 faculty/staff members!
•
Reasons for Request
–
–
–
–
Compliance for DE Course – 73%
Compliance for F2F Course – <1%
Web Compliance – <3%
Disability Accommodation – 23% (~30% costs)
• Totals
– Avg. video length – 15.85 min
– Avg. cost/video minute – $1.79
– Avg. cost per outsourced job - $34.66
College/School/Unit
Costs (% total)
CEHD
CHHS
CHSS
Compliance, Diversity,
and Ethics
COS
CVPA
Fiscal Services
HR
ITS
3.97%
3.00%
23.89%
Office of the President
0.06%
Office of the Provost
SCAR
School of Business
SPGIA
University Life
Volgenau School of
Engineering
Grand Total
0.21%
7.48%
9.01%
3.20%
1.10%
0.11%
11.31%
1.35%
0.00%
0.31%
0.05%
34.95%
100.00%
DOCUMENT ACCESSIBILITY PILOT
Getting to a Pilot Project…
• Fall 2013/Spring 2014/Fall 2014
– ATI Staff included as reviewers in DE Course
– Reviewing entire courses (i.e., syllabus, course objectives, etc.)
• Fall 2014
– Provided feedback suggesting that DE leverage ATI Office’s expertise!
– Conducted a Document Accessibility Pilot to help determine the ROI
•
March/April 2015
– Approached about participating in pilot of open-call courses
• May 2015
– Launched the Pilot (Accessibility reviewed separately)!
Document Accessibility Pilot
•
Why
-
•
Faculty members still struggling with what we are asking of them
Pilot
-
Working with an 4 IDs and a single faculty member from each who is currently developing a DE course
Looking at what our office can handle and what we cannot
Tools being used…
• ABBYY Recognition Server 4.0 (PDFs and Image)
• CommonLook Office (Word and PPT)
•
Goal
– Establish a scalable process that will support faculty members who have blind students in their courses.
Build toward supporting DE.
Document Accessibility Pilot – Results
• Faculty/Staff Participants:
• Documents Reviewed:
–
–
–
–
–
5
87
ID #1 (1 PDF, 1 Word*) = 2
Fac #1 (6 PDF) = 6
ID #2 (3 PDF, 3 Word, 1 PPT) = 7
Fac #2 (1 Word, 11 PPT) = 12
Fac #3 (17 PDF, 34 Word, 5 PPT, 2 XLS) = 58
• By Document Type:
– PDF (27), Word (39), PPT (16), Excel (2)
Total # of pages reviewed: 1,121
* – Word encompasses .docx, .doc, .rtf, and .txt file types., PPT encompasses .ppt and .pptx.
Excel encompasses .xls and .xlsx.
Document Accessibility Pilot - Takeaways
• What we learned?
– Streamlined internal document accessibility process
• Documents captured via SharePoint Doc Library
• Reviewed primarily with CommonLook Office Professional
• Still have some bugs to work out (e.g., alt text, final outputs, docs vs. images)
– Great deal of variation in types of documents used
– Better positioned to focus on accommodation at this time
• Now offering service to faculty members that have a blind student enrolled in their courses.
• Spring 2015 - 1 blind student, 4 courses along with orientation materials, We’ve remediated 56 documents (over
1900 pages, not including textbooks)
DE COURSE PORTFOLIO REVIEW PILOT
How we got to the DE Course Review Pilot…
• Two ATI staff members had participated in the DE Office’s Standard Course Portfolio
review process (Spring & Fall 2014).
– Reviews involved several stakeholders reviewing all elements of the online course (syllabus,
course objectives, learning objectives, accessibility, etc.).
– Each reviewer was paid $100 for reviewing the course.
• Big Problem:
– Other reviewers who knew little about accessibility were reviewing accessibility.
– Equally, we are not course developers.
• Made suggestion to DE Office to leverage expertise of our office for accessibility and
let everyone else review the areas that fit with their expertise.
“Open-Call” Course Review Pilot
• “Open-Call” Pilot was only for online courses not supported by
the ID Team or DE Office.
• Two goals:
– To introduce faculty teaching “rogue” online courses to the resources
available through the DE Office and/or ID Team.
– To investigate the feasibility of this accessibility review process.
• We reviewed 6 courses across several disciplines (Nursing,
Nutrition, Communications, Business Leadership)
Sample – ATI Course Accessibility Checklist
ATI Course Accessibility Checklist (Internal Use Only)
Updated 5/28/2015
Includes a review of the following
areas:
1.
2.
3.
4.
5.
6.
7.
Syllabus and Course Readings
Bb Learn
Word
PPT
PDF
Multimedia
Supplemental Applications
The following checklist verifies that the instructional documents, audio, and video content used in
Mason’s distance education courses are in accordance with Section 508 and WCAG 2.0 Level AA
accessibility guidelines. While not a comprehensive review of all the areas covered by these
guidelines, this checklist does examine areas that would have the most significant impact on the
ability of assistive technology users to independently access their instructional materials (e.g., al text,
keyboard navigation, captions, transcripts, etc.).
PLEASE NOTE: This is NOT a comprehensive review of the accessibility of the faculty member’s
course. The reviewers will examine snapshots (i.e., Course readings, LMS layout/structure, 2-3
documents of each type – i.e., Word/PDF/PPT, 2-3 videos, and supplemental applications) of the
elements highlighted in the table below and provide feedback/guidance to the instructor on how to
correct any accessibility issues that are identified.
UNDERSTANDING THE REVIEW PROCESS:
Reviewers examined selected examples of the elements highlighted in the attached checklist (i.e.,
course readings, LMS layout/structure, 2-3 documents of each type – i.e., Word/PDF/PPT, 2-3 videos,
and supplemental applications) and provided feedback and resources for the instructor on how best
to remediate any accessibility issues that were identified.
Tools used for testing accessibility:
·
·
Website Accessibility Reviews – WAVE Toolbar
MS Office Accessibility Reviews – Built-in MS Office Accessibility Checker
Term:
Professor:
Course Evaluated:
Reviewer:
0.0 – Syllabus and Textbooks/Course Readings (Required and
Supplemental)
ID
0.1
0.2
Textbooks/Course Readings
Is an electronic equivalent provided for all print
reading materials?
Yes
No
N/A
Yes
No
N/A
Do all web articles/readings have a PDF/Word
version available?
0.3
Syllabus
Course syllabus includes disability statement?
0.4
Instructor offers multiple formats/options for
“Open Call” Pilot Project Results (May 2015) cont.…
Issue Identified
PPT inaccessible
Word inaccessible
Videos not captioned
and/or transcribed
Video platform
inaccessible
Bb Course Structure
Improper hyperlink
text
Supplemental
app/website
Course #1
Course #2
Course #3
Course #4
X
X
X
X
Course #5
X
X
X
X
X
X
X
X
X
X
X
X
X
Course #6
X
X
X
X
X
X
“DE-Supported” Portfolio Review Results (Sept/Oct 2015) cont.…
Issue Identified
Course #1
Course #2
Course #3
Course #4
Course #5
Course #6
Course #7
X
X
X
X
X
X
PPT inaccessible
Word inaccessible
PDF Inaccessible
X
Videos not captioned
and/or transcribed
X
Video platform
inaccessible
X
X
X
X
Bb Course Structure
X
Improper hyperlink
text
X
Supplemental
app/website
X
X
X
X
X
X
X
Sample – ATI Course Evaluation Document
Office of Distance Education Course Portfolio Reviews
Fall 2015
ATI Course Accessibility Evaluation
Includes the following:
•
•
•
Priority Recommendations and
Resources
Understanding the Review Process
(i.e., testing tools used and process)
Findings
As a part of the Office of Distance Education’s Open Call Course Portfolio Review process, the
instructional materials used in your course (i.e., documents, audio, video, websites, and web
applications) were examined to determine if they are accessible and usable by Mason students,
including those with disabilities, in accordance with University Policy 1308.
This is not a comprehensive evaluation of all the areas covered by this policy; rather this review
focuses on those areas that have traditionally had the most significant impact on the ability of
students with disabilities to independently access instructional materials (e.g., alternative text
descriptions, keyboard navigation, captions, transcripts, etc.).
COURSE:
·
·
·
·
Term:
Professor:
Course Evaluated:
Course Reviewer:
Spring 2015
PRIORITY RECOMMENDATIONS and RESOURCES:
Priority Issue
Action Plan
Ensure all course videos have synchronized
captions and/or transcripts.
Go to the ATI’s Accessible Media Request
page for information on requesting this
service.
Ensure all course videos are streamed
through an accessible video playback platform
(i.e., Kaltura, YouTube).
Upload course videos using the “Kaltura My
Media” link in MyMason\Courses Tab. For
assistance, contact the ATI Office and/or
Learning Support Services.
Ensure that PowerPoint Presentations and
Word Documents are accessible.
Visit ATI’s Guide to Creating Accessible
Electronic Materials (PDF – Section 2: Part II
and Part III).
Visit ATI Website: Creating Accessible
Documents
Additional Takeaways
• Web MyLabs, McGraw Hill, Cengage
• Laid groundwork for DE Pilot
• If a professor is using Copyrighted PPTs from a 3rd party
source like McGraw Hill or Pearson, we can’t do anything
about it, right?
Pilot Project Results
•
Results
– Success! Very well-received by faculty and
DE Staff. Asked to participate in the larger
Course Portfolio Review Process
– Revenue generator!
– Developed DE Course Accessibility Checklist
– Reviewed 6 courses
– Prioritized findings
•
Issues Identified!
– Open Call vs. DE supported courses
– Scalability
Evolution since Pilot Project…
• We’ve been integrated into the DE Office’s Standard Course
Portfolio Review Process
• We have reviewed 7 additional courses this fall
– 2 by request (Sum 2015)
– 5 as part of DE Office’s Standard Course Portfolio Review Process (Fall
2015)
Next Steps
•
•
•
•
Refined Course Accessibility Checklist
Design Course Accessibility Guide as reference to checklist
Work with DE Office to solicit faculty feedback (4P)
Work with ID Team to solicit faculty feedback (OCDI)
Important Considerations
LEARN FROM OUR EXPERIENCES…
Things to consider…
Identify your Strategic Partners
– Is your institution investing in online learning?
Do you have policies and procedures
– Accessibility training (i.e., captioning, document accessibility)
Is accessibility already integrated into the process?
– Quality Matters? OLC? Bb Course Rubric?
Build off of the platform…
– Canvas? Bb? D2L? Kaltura? Pinopto?
Program evaluation?
– Do you have a process for determining success/failure?
Questions
Contact Information
Assistive Technology Initiative (ATI)
George Mason University
Aquia Building, Room 238
Office Phone: 703-993-4329
Fax: (703) 993-4743
E-mail: ati@gmu.edu
Web: http://ati.gmu.edu
Twitter: @AccessibleMason