Building Cross-Platform
Accessible Applications Using Qt
Technical Overview
September 2012
1
DataNaut, LLC
http://www.datanaut.com
Introduction

Overview
–
–
–




2
Presentation covers modern architectural patterns such as Model View ViewModel
(MVVM) combined with the latest cross-platform technologies including Qt and Webkit
to create interactive and accessible applications.
Using Qt and Webkit offer significant advantages when targeting multiple platforms
however accessibility is an issue.
This presentation covers what steps you can take to leverage these powerful tools
and technologies and meet accessibility requirements
Review usage example
Accessibility in Qt
Approach
Tools and Technologies
DataNaut, LLC
http://www.datanaut.com
Working Example


Review working example using this technical approach
Learning Ally Audiobook Manager
–
–
–
–
–

Key goals
–
–
–
–
3
Used by Learning Ally subscribers to manage books that they can access
Users download books which are synchronized with iTunes, Windows Media or
DAISY device
Users interact with a bookshelf which presents the books in an accessible manner
Application has HTML User Interface hosted within a cross-platform Qt application
Find out more at http://www.learningally.org
Desktop application that supports Windows and Mac platforms
Support for screen readers – JAWS, Window-Eyes, NVDA, native
Leverage Webkit and Qt
Implement Model View ModelView pattern
DataNaut, LLC
http://www.datanaut.com
4
DataNaut, LLC
http://www.datanaut.com
Application Approach
Overview




Application implemented as a cross-platform thick client with rich user interface
and advanced accessibility support
Cross platform support is provided by Qt framework with UI built using the web
technologies (HTML/CSS/JavaScript) embedded into the thick client using
WebKit control
By using the WebKit control, hosting web assets helps to facilitate building the
UI using the standard web technologies while still providing thick client
advantages such as file system access, advanced networking capabilities,
accessibility and other lower level OS capabilities
Why?
–
–
–
5
Not all applications will work in the web browser. HTML5 goes a long way but some applications
need more
Webkit provides the capability to implement a modern web-based user interface
Qt does not provide complete accessibility support, need approach to support it in a cost effective
manner
DataNaut, LLC
http://www.datanaut.com
Application Approach
Issues with Qt and Accessibility






6
Qt has limited support for accessibility on Windows and Mac OS.
The majority of stock Qt controls do not support accessibility
Qt has it’s own implementation of Webkit called QWebKit which has
it’s own accessibility issues
An approach is to implement custom control with accessibility support
using interfaces provided by Qt, e.g. an accessible wrapper around
QWebKit
Official Qt forums and docs advise to override standard controls with
your own implementation of accessibility
The best approach is that the Qt community adds support for
IAccessible and IAccessible2 but this is not in the pipeline
DataNaut, LLC
http://www.datanaut.com
Application Approach
Adding Accessibility to QWebkit



Implement custom wrapper around standard QWebKit control that supports
accessibility
Wrapper receives messages with active control and its state from HTML (via
JavaScript) in QWebKit and sends this information to the OS
This works on Windows but there are issues on Mac
–
–
–

Benefits
–
–
–
7
By default the OS reads accessibility details once for each control and doesn’t support dynamic
changes of accessibility details
Since this approach changes the accessibility details on the fly it’s a show-stopper
Requires modifications to alter how Qt processes accessibility-related changes to have the ability to
send control changes to the system
End result is an accessibility façade that can be primarily controlled from HTML and JavaScript
It’s very flexible. You can control navigation between HTML controls, control state and accessible
details
Overall the code is easier to maintain, works better with web designers and development team
DataNaut, LLC
http://www.datanaut.com
Application Approach
Using JavaScript

Handle accessibility in two parts –
–



8
Navigation between controls
Manage what is announced when on a control
Most of the work is done in JavaScript and Qt is used as a proxy to
send updated controls state and hints to the OS
Generate navigation map tables in JavaScript for all accessible
controls in the web page
Add a custom attribute to HTML tag for accessibility specifics including
what’s announced to the user
DataNaut, LLC
http://www.datanaut.com
Application Approach
Accessibility Facade

Each web page has a navigation map that defines accessibility
properties in JavaScript:
var navigationMap = {
//
//
//
//
//
9 - tab
39 - right
37 - left
40 - down
38 - up
//SigUp
"SignUpUsernameInput": { 9: "SignUpPasswordInput", 'shift_tab': "SignUpForgotPasswordLink", 40:
"SignUpPasswordInput", 38: "SignUpForgotPasswordLink", 13: function () { $('#signInBtn').trigger('click'); }
},
"SignUpPasswordInput": { 9: "SignUpRememberMeCheckbox", 'shift_tab': "SignUpUsernameInput", 40:
"SignUpRememberMeCheckbox", 38: "SignUpUsernameInput", 13: function () { $('#signInBtn').trigger('click'); }
},
"SignUpRememberMeCheckbox": { 9: "SignUpButton", 'shift_tab': "SignUpPasswordInput", 40: "SignUpButton",
38: "SignUpPasswordInput", 13: function () { $('#signInBtn').trigger('click'); }, 32: function () {
$('#rememberMe').trigger('click'); } },
"SignUpButton": { 9: "SignUpForgotUsernameLink", 'shift_tab': "SignUpRememberMeCheckbox", 40:
"SignUpForgotUsernameLink", 38: "SignUpRememberMeCheckbox", 13: function () {
$('#signInBtn').trigger('click'); }, 32: 'click' },
"SignUpForgotUsernameLink": { 9: "SignUpForgotPasswordLink", 'shift_tab': "SignUpButton", 40:
"SignUpForgotPasswordLink", 38: "SignUpButton", 13: function () { $('#signInBtn').trigger('click'); }, 32:
function () { $('#forgotUsernameId').trigger('click'); } },
"SignUpForgotPasswordLink": { 9: "SignUpUsernameInput", 'shift_tab': "SignUpForgotUsernameLink", 40:
"SignUpUsernameInput", 38: "SignUpForgotUsernameLink", 13: function () { $('#signInBtn').trigger('click'); },
32: function () { $('#forgotPasswordId').trigger('click'); } },
}
9
DataNaut, LLC
http://www.datanaut.com
Application Approach
Accessibility Facade Continued


To support what is announced by the reader there are special html
attributes for controls to describe the type and voice hint
When the user mouses over or navigates to a control, the control
sends a message to Qt with control type, position, and accessible text.
Qt forwards this info to the system to make it available to the screen
reader
this.read = function (element, activeControlText) {
var textToRead = element.attr('toread');
textToRead = textToRead.replace(/<(?:.|\n)*?>/gm, '');
if (activeControlText && activeControlText != '') {
textToRead += ' ' + activeControlText;
}
console.log("READ " + accessibleRole + "
:" + textToRead);
console.log("Element info: name = " + jQuery(element).get(0).tagName + " top = " + elTop + "; left =
" + elLeft + "; height = " + elHeight + "; width = " + elWidth);
this.acc_client.changeAccessibleInfo(textToRead, accessibleRole, elTop, elLeft, elHeight, elWidth);
}
10
DataNaut, LLC
http://www.datanaut.com
Application Approach
Accessibility Facade Continued
Qt support Code in Qt client:
#if defined(Q_WS_WIN)
QAccessible::updateAccessibility(this, 0, QAccessible::Focus);
QAccessible::updateAccessibility(this, 0, QAccessible::ValueChanged);
#elif defined(Q_WS_MAC)
QAccessible::updateAccessibility(this, 0, QAccessible::ValueChanged);
#endif
11
DataNaut, LLC
http://www.datanaut.com
Future Developments
Support for IAccessible2 in Qt





12
Qt accessibility is based on IAccessible interface however it’s not complete
The problem is that Qt does not send enough information to the screen readers
to properly announce the controls
The appropriate course of action is to implement IAccessible2 interface for Qt
controls. However this is a significant effort and not a high priority in the Qt
development pipeline
DataNaut has performed R&D to add IA2 for several controls
Our goal is to finalize IA2 support for these controls and lobby Qt community for
inclusion in the current build
DataNaut, LLC
http://www.datanaut.com
Tools, Technologies, Resources





13
Qt – http://qt.nokia.com/
Webkit – http://www.webkit.org/
Model View ViewModel – http://en.wikipedia.org/wiki/Model_View_ViewModel
Knockout MVVM Library - http://knockoutjs.com/
Open Accessibility Group http://www.linuxfoundation.org/collaborate/workgroups/accessibility
DataNaut, LLC
http://www.datanaut.com
Questions?

14
Contact Mark Snuffin at snuffin@datanaut.com
DataNaut, LLC
http://www.datanaut.com