Logo above: University of Birmingham
Evaluation of alternative exam
presentation methods for pupils
with low vision
Report for RNIB
September 2010
Authors:
Dr Graeme Douglas
Dr Mike McLinden
Annette Weston
Visual Impairment Centre for Teaching and Research (VICTAR)
School of Education
University of Birmingham
Birmingham, B15 2TT
Contents
Acknowledgements .............................................................................. 4
Executive summary .............................................................................. 5
Summary of findings ........................................................................ 5
Recommendations........................................................................... 7
1
Introduction and context ............................................................... 9
2
Student performance and views of alternative examination
papers .......................................................................................... 10
2.1 Introduction ............................................................................ 10
2.2 Method ................................................................................... 10
2.3 Results ................................................................................... 13
3
Case examples: Students who prefer very large hardcopy print
(N24+) ........................................................................................... 23
3.1
3.2
3.3
3.4
4
Introduction ............................................................................ 23
Method ................................................................................... 23
Results ................................................................................... 26
Discussion ............................................................................. 37
In-depth interviews with staff involved in material modification
...................................................................................................... 40
4.1 Introduction ............................................................................ 40
4.2 Method ................................................................................... 40
4.3 Analysis of results .................................................................. 43
5
Group Interviews: Students who are non-print readers ............ 68
5.1
5.2
5.3
5.4
6
Introduction ............................................................................ 68
Method ................................................................................... 68
Results ................................................................................... 71
Discussion ............................................................................. 82
Technical analysis of the RNIB E-formatting software.............. 83
6.1 Modifying the examination paper ........................................... 83
Evaluation of alternative exam presentation methods for pupils with low vision
2
6.2 Using the software to format the examination paper and quality
of the output. ................................................................................. 85
6.3 ‘Accessible’ files ..................................................................... 87
7
Discussion .................................................................................... 89
7.1
7.2
7.3
7.4
7.5
Summary of findings .............................................................. 89
Classroom practice versus public examination system .......... 91
Possible solutions .................................................................. 92
Recommendations ................................................................. 94
Functional specification of future software ............................. 95
8
References ................................................................................. 100
9
Appendix 1 Student Questionnaire........................................... 101
10 Appendix 3 Font size questionnaire ......................................... 104
11 Appendix 3 Professional Interview Schedule .......................... 106
Evaluation of alternative exam presentation methods for pupils with low vision
3
Acknowledgements
We would like to thank all our participants for generously giving us their
time and views. We thank RNIB for funding this research. We thank the
steering group for their helpful advice throughout this project. Thank you
to Rory Cobb for general advice and help with the group interviews with
students. Thank you to Sue Keil and Paul Nisbet for their general advice.
Evaluation of alternative exam presentation methods for pupils with low vision
4
Executive summary
This is a report commissioned by RNIB in relation to examination access
by blind and partially sighted learners. The research particularly
focussed upon the evaluation of a prototype piece of software – ‘RNIB Eformatting software’. RNIB E-formatting software was designed to allow
the teacher/school to modify examination papers in terms of font size
and style, text and background colour, and the line width inside
diagrams. Once it has been modified to meet the individual requirements
of the candidate, a paper can be printed for them to have as a paper
copy or it may be read on-screen.
The research team carried out five related pieces of research:
 Student performance and views of alternative examination papers
 Case examples: Students who prefer very large hardcopy print
(N24+)
 In-depth interviews with staff involved in material modification
 Group Interviews: Students who are non-print readers
 Technical analysis of the RNIB E-formatting software
Summary of findings
1. How do the Modified Large Print (MLP) versions of papers
produced by RNIB E-formatting software compare to traditional
MLP papers? (both process and outcome)
a. Findings from a trial involving 21 students with low vision suggest that
examinations that are formatted using the RNIB E-formatting software
neither significantly advantages or disadvantages student
performance compared with existing MLP approaches. Familiarity
with examination format appears to be a more important variable.
b. Professionals interviewed were broadly positive about the RNIB Eformatting software:
 They were positive about having a greater range of options for
examination formats which the software could offer.
 They particularly identified control over page breaks and formatting
of pictures/diagrams as key areas for further development.
 There were mixed feelings about how the process of formatting
and production could be incorporated into the short one hour
Evaluation of alternative exam presentation methods for pupils with low vision
5
period before the examination. Some had concerns about
technical problems and working under time pressures.
c. There is evidence that appropriate and consistent modification of
examinations is of key concern to professionals irrespective of the
method of delivery. The process of producing the software versions of
the examination papers (from four different examination boards)
appears to have an important associated benefit, namely
standardising of the modification process and the general format of
the examination papers.
d. Case examples of five students who preferred very large print (greater
than 24 point) demonstrated that they appear to benefit from hard
copy large print formats which are different from those available
through existing MLP approaches. An estimated 9% of GCSE-aged
students with a visual impairment in England and Wales use font
sizes greater than 24 point.
2. What methods of accessing examinations are useful for the
future, and can the RNIB E-formatting software (or a developed
version of it) play a useful part? (e.g. electronic copies, interactive
papers)
a. When reflecting on the ‘perfect scenario’, many professionals reported
that students should have access to examinations which had been
modified appropriately and could then be formatted in a flexible way to
meet individual needs. The use of technology was seen as an
important part of the formatting solution by all participants (teachers,
students with low vision and non-print readers). This suggests a
distinction between appropriate modification of a paper ‘at source’ and
the formatting options that could potentially be provided by the
software.
b. Access to on-screen interactive examinations was seen by both
students and professionals as offering a range of opportunities for
improving access. Nevertheless, further work is required to enable
these opportunities to be developed further. Key challenges identified
through discussion with students (blind and low vision) as well as
professionals included:
 technical work is required to ensure computer-based material is
fully accessible;
 appropriate examination modification is carried out ‘at source’;
Evaluation of alternative exam presentation methods for pupils with low vision
6
 training of students in the use of assistive technology to access
exams;
 training of staff to teach students how to use this assistive
technology for exams access.
c. Future development needs to be flexible enough to offer a range of
on-screen formats (e.g. a range of fonts, sizes, colours, and zoom
facilities). They also need to include options for hard copy versions of
questions and/or supporting information such as diagrams and tables
to be produced in a variety of formats.
Recommendations
We present two broad recommendations based upon the research
findings. The recommendations are made in relation to the project
research questions, i.e. particularly linked to the potential of the RNIB EFormatting software (or similar) in the provision of accessible
examination papers. It is important to note that the authors advise that
the recommendations are implemented as part of a more general review
of examination and curriculum access approaches for visually impaired
students in England and Wales, specifically in relation to:
 The procedures and quality control of examination provision (including
the process of modification);
 Curriculum access strategies adopted in the classroom (particularly in
relation to the use of low vision aids and computers).
Recommendation 1:
A software application for supporting the production of an increased
range of examination formats for students with visual impairment should
be developed. The application would be used by teachers and visually
impaired students, and it should have the following functions:
a) a range of formatting options;
b) printing facilities for students who require hardcopy large print
examinations;
c) onscreen presentation facilities for students who require onscreen
access to examinations.
The report provides a draft functional specification for the software
application. The RNIB E-formatting software evaluated in this research
project provides a very useful prototype of such an application.
Evaluation of alternative exam presentation methods for pupils with low vision
7
Recommendation 2:
The future development of mainstream interactive computer-based
examinations presents significant opportunities for improving access to
examinations amongst visually impaired students. To ensure that
visually impaired students can benefit from these opportunities the
following general recommendations are made:
a) Examination boards and designers of computer-based
examinations technology should work with access technology
experts (for example, within RNIB) to ensure that inclusive design
principles are incorporated into the development of this technology.
b) Developments in computer-based examinations will mean that
educators must continue to ensure that visually impaired pupils are
taught appropriate access skills so that they can independently
access these examinations.
Evaluation of alternative exam presentation methods for pupils with low vision
8
1. Introduction and context
This is a report commissioned by RNIB in relation to examination access
by blind and partially sighted learners. The research was designed in
2008 in response to the Research Brief prepared by RNIB Corporate
Research Team: “Passing the test: how accessible are public
examination systems for blind and partially sighted pupils in the UK”.
The research was carried out in two phases of work carried out between
April 2009 and July 2010:
 Phase 1: International comparison of approaches
 Phase 2: Trialling and functional specification of the RNIB ‘Eformatting’ software
Phase 1 report (entitled ‘Summary report on international systems of
exam access for visually impaired pupils’) was presented in September
2009. This report presents the findings from Phase 2. Following some
reconfiguration of the project following the Phase 1 report, Phase 2 had
the following research questions:
1. How do the modified large print (MLP) versions of papers produced by
RNIB software compare to traditional MLP papers? (both process and
outcome)
2. What methods of accessing examinations are useful for the future,
and can the RNIB software (or a developed version of it) play a useful
part? (e.g. electronic copies, interactive papers)
The research team carried out five related pieces of research which are
presented in turn:
 Student performance and views of alternative examination papers
(linked to research questions 1 and 2)
 Case examples: Students who prefer very large hardcopy print
(N24+) (linked to research questions 1 and 2)
 In-depth interviews with staff involved in material modification
(linked to research questions 1 and 2)
 Group Interviews: Students who are non-print readers (linked to
research question 2)
 Technical analysis of the RNIB E-formatting software (linked to
research question 1)
Evaluation of alternative exam presentation methods for pupils with low vision
9
2. Student performance and views of
alternative examination papers
2.1 Introduction
The research team carried out an experiment which compared paperbased formats prepared using existing large print approaches (modified
within the existing large print ‘access arrangement’ policies) to formats
prepared using the E-formatting software. The aim was to examine
whether either format gave any measurable advantage to the visually
impaired pupils (in terms of time taken). The participants also gave their
opinions about the different formats through a short questionnaire and
group interviews.
2.2 Method
Overall design
A repeated measures design was employed so that each participant tried
out two alternative presentations of an examination – modified large print
(‘MLP’) and E-formatting software (‘Software’). Different versions of the
examination papers and order of presentation were balanced. Time
taken to complete the tasks was measured, and opinions about the
different formats were gathered through a questionnaire.
Participants
Twenty one participants took part in the study in three separate sessions.
All were taking part in a study week at Worcester New College during
their Easter holiday in 2010. All had low vision, were print readers and
expected to be using print-based examination papers in the GCSE
examinations. The majority of participants attended mainstream schools
(except one mainstream school with resource base and one ‘private’). A
breakdown of the participants is presented in Table 1.
Evaluation of alternative exam presentation methods for pupils with low vision
10
Table 1. Characteristics of the participants taking part in the trial
(N=21). Note: frequencies based upon available data.
Characteristic
N
Age group
Year 10
9
Year 11
12
Gender
Male
13
Female
8
Font size (provided prior to testing)
12 point
2
18 point
10
24 point
9
Font size (questionnaire response)
16
1
18
8
24
8
36
1
LVA used (questionnaire response)
Yes
8
No
12
Preferred font (questionnaire response)
Arial
9
Impact Bold
1
Comic Sans
3
Any
1
TOTAL
21
Materials and variables
Content for two ‘pretend’ examinations was created (version A and B).
Both examinations had the same structure and length and were matched
for content difficulty. Each paper consisted of ten questions. The
question styles were drawn from existing papers and included:
 Multiple choice with tick box (‘Tick the BEST answer’)
 Sorting statements into categories
 Multiple choice (Put a ring around the correct answer)
 Reference to a picture, single word written answer on answer line
 Reference to a table, multiple choice
 Labelling a diagram
Evaluation of alternative exam presentation methods for pupils with low vision
11
 Reference to a graph, single word written answer on answer line
 Reference to text and a picture, single word or two word answers
 Linking of associated statements
As part of the introduction given in each session, it was emphasised to
participants that we wanted them to complete all questions and that we
were testing the quality of the material and not testing them. For this
reason the questions were very easy (containing relatively simple
general knowledge and logical questions). The key measured variable
was time taken to complete each examination paper.
After completing the two versions of the examinations (as described in
the procedure below), participants completed a questionnaire (see
Appendix). The questionnaire contained questions in relation to:
 Personal working preferences (font style and size, use of LVA)
 Preferences for different formats of examination and explanations
 Ideal examination format and potential of onscreen examinations.
On completion of the questionnaire participants took part in semistructured group discussions. The topics explored were:
 Reflections upon the versions of the examinations just taken
 General experience and expectations in relation to taking
examinations
 Potential of computer-use in examinations.
Procedure
Participants were allocated into three groups and each data collection
session took place over approximately a 1.25 hour period. At the
beginning of the session the broad purposes of the research was
explained and it was emphasised that the material was being tested
rather than the participants themselves. Participants were then asked to
sign a consent form declaring that they were willing to take part in the
research, but it was noted that they could withdraw at any time.
Participants were each given a £10 gift voucher irrespective of whether
they chose to take part in the research or not. It is worth noting
participants were taking part in a study weekend in preparation for GCSE
examinations and taking part in this session was seen as a useful and
relaxed way of practicing and considering different formats of
examination papers.
Evaluation of alternative exam presentation methods for pupils with low vision
12
Participants were asked to carry out the testing in ‘exam-like conditions’,
i.e. in silence and working independently. Each participant took two
versions of the examination – one modified large print (‘MLP’) and the
other E-formatting software (‘Software’). The order of presentation was
balanced. The following procedure took place:
1. Instructions to participants were given
2. Examination Paper 1 was handed out (all pre-prepared and named for
each participant)
3. When the researcher said ‘start’, participants started timers on their
desk and began Examination Paper 1
4. Once Examination Paper 1 was completed, participants would stop
the timer, note the time on the front of the paper, and indicate to the
researcher (who also noted time taken as a check).
5. Participants then sat in silence until everyone had completed the
Examination Paper 1 (a total maximum time of 15 minutes).
6. All the papers were collected, and Examination Paper 2 was
distributed.
7. The process for Examination Paper 1 was repeated for Examination
Paper 2.
8. On completion of Examination Paper 2, the questionnaire was
distributed and completed by participants.
9. On completion of the questionnaire all participants took part in a group
discussion.
To control for effects of order and exam content, the experimental design
was balanced, i.e. participants received different versions of the paper (A
or B), prepared in different ways (MLP or Software), in different orders
(first or second).
2.3 Results
Throughout the report all the available data is used. For example,
analysis of performance (based upon time taken) includes 19 of the 21
participants because two participants were not able to complete the
examination papers in the time available.
Performance – time taken
A key aim of the research was to establish whether there was a
difference in performance when using examinations prepared using the
existing large print approaches (modified within the existing large print
Evaluation of alternative exam presentation methods for pupils with low vision
13
‘access arrangement’ policies, ‘MLP’) compared with formats prepared
using the E-formatting software (‘Software’). In terms of mean time
taken for a sample of 19 students with low vision, very little difference
was found (mean time MLP 422 seconds cf. mean time Software 418
seconds) and unsurprisingly this was not found to be statistically
significant. This suggests that examinations formatted using the Eformatting software neither significantly advantages or disadvantages
student performance compared with existing modified large print
approaches.
Analysis of ‘control variables’ used to balance the experimental design
was also carried out. No significant difference was found between the
times taken across the two versions of the examination paper confirming
that they appear to be matched in terms of difficulty. However, a highly
significant difference was found in terms of order of presentation –
examinations taken first were significantly slower than examinations
taken second (first mean = 486 seconds cf. second mean = 354
seconds). In general terms this is not surprising as participants learned
about the structure of the examinations from the first examination to the
second and this was reflected in faster answering (and to this extent
vindicates balancing the design). Perhaps more helpfully, this strongly
suggests that encouraging students to practise examination papers,
allowing them to get used to question styles and structures, is of
enormous benefit.
Table 2. Summary of repeated measures t-test: (a) MLP vrs
Software, (b) First vrs Second, and (c) Version A vrs Version B.
N=19
Comparison
Mean
Standard Significance
(seconds) Deviation
MLP vrs Software
MLP
422
147
Software
418
162
t (18) = 0.12; p>0.05
First vrs Second
First
486
146
Second
354
132
t (18) = 7.15; p<0.0005
A vrs B
Version A
409
147
Version B
431
162
t (18) = 0.64; p>0.05
Note: 420 seconds = 7 minutes
Another interesting observation is linked to the two participants who did
not complete either examination paper within the available 15 minutes.
Evaluation of alternative exam presentation methods for pupils with low vision
14
Given the examination papers were less than 700 words long (plus
diagrams) these two participants clearly had a particularly slow reading
speed. Again, this general point is discussed within the overall
discussion.
Preferences
Participants were asked which examination format they preferred overall.
Table 3 shows that answers were fairly evenly distributed, although the
largest proportion of participants thought the Software version was the
best. Perhaps of greater interest were the comments and explanations
for what participants liked and disliked about the different formats (Table
4 and Table 5).
Table 3. Which examination format did you prefer? N=21
Format
Frequency
MLP
6
Software
8
Both the same
5
Unsure
2
Table 4. MLP – explanations of what participants liked and disliked.
Answers grouped by authors
Likes (frequency)
Generally positive about font, font size, or use of underlining (7)
Generally positive about the layout (5)
Generally positive about use or graphics (4)
Layout of pages / page breaks (3)
Liked using large A3 paper (1)
Dislikes (frequency)
Disliked using large A3 paper (3)
Wording confusing (2)
Poor multiple choice format (2)
Poor pictures (2)
Poor general layout (2)
Evaluation of alternative exam presentation methods for pupils with low vision
15
Table 5. Software – explanations of what participants liked and
disliked. Answers grouped by authors
Likes (frequency)
Generally positive about the layout (5)
Generally positive about use or tables and pictures (4)
Generally positive about font, font size (4)
Liked using A4 paper with larger font (3)
Good multiple choice format (1)
Dislikes (frequency)
Poor page breaks / split questions (6)
Wording confusing (2)
Poor / lack of pictures (2)
Participants’ likes and dislikes of the two formats overlapped
considerably. This is unsurprising given that many expressed no
preference between either formats. For example, there were positive
views about general formatting, fonts, graphics, and layouts for both MLP
and Software versions. Similarly, there were negative views regarding
some graphics, wording and multiple choice formats. These negative
views (and some of the positive ones) broadly fall within the modification
process rather than the formatting process per se. That is, irrespective
of format (MLP or Software) a modifier will have made a decision about
particular graphics and wording modifications and structure of multiple
choice questions (e.g. tick boxes aligned left may be easier for the
visually impaired pupil to find rather than scanning over white space to a
right aligned box). This distinction between the modification and
formatting (including enlargement process) is raised elsewhere in the
report.
Nevertheless, some of the comments did link to particular features of the
formats (and the process of producing them). The Software version use
of page breaks was identified as a problem by six participants, i.e. having
questions split over pages. This is discussed in section 6 ‘Technical
analysis of the RNIB E-formatting software’ and confirms the points
made there. It was also raised in some detail in the group discussions.
This highlights the importance of navigating documents (examination
papers in this context) and the importance of considering this in the
exam modification process, e.g. as illustrated by the following quotes:
Evaluation of alternative exam presentation methods for pupils with low vision
16
“It gets annoying when you have to turn the pages. And
sometimes the question’s on one side and information is on
another page.” Discussion group 1.
“I preferred the large print one it was easier to read -more
compact, less page turning.” Discussion group 2.
Positively, three participants (who preferred 24 point font) specifically
said they liked having the examination on A4 paper (which is possible
using the E-formatting software, but not within the MLP procedures).
Perhaps in keeping with an emerging theme (and general ‘truism’) of
‘everyone is different’, one participant described preferring having bigger
A3 paper. It’s worth noting that this unexpected preference is achievable
using the E-formatting software (by using an A3 printer or photocopier
enlargement). Again, this was raised in the group discussions:
“I think the software was better because, it's on smaller
paper but with the same size writing.” Discussion group 1.
Page breaks and paper size both link to the process of navigating the
examination paper. Some participants made links between this and the
potential of technology.
Ideal format and use of computers
Participants were asked to ‘explain how you would have liked the
examination papers formatted for YOU’. The purpose of the question
was to get some sense of how they would ideally like examination
papers formatted. Most of the responses made references to specific
formatting strategies (and often referring to the examination papers they
had just used). Some of the responses were quite full, while others
made specific single points:
“Font size to be 18 all way through. Clear pictures. Question
numbers in bold. Information tables on same page as
questions to prevent having to keep flicking back and
forward.” Questionnaire.
“Size 24 one question per page bold, comic sans, simplified
diagrams and maps.” Questionnaire.
“The resources/questions and answers on the same double
page spread.” Questionnaire.
Evaluation of alternative exam presentation methods for pupils with low vision
17
The points raised tend to highlight the wide variety of individual
formatting preferences, and some appreciation of the modification
process. In contrast, one respondent implied that she did want any
modification to the formatting:
“The same as everyone else.” Questionnaire.
Only one participant appeared to consider possibilities beyond options
which are currently available, in this case having the examination paper
accessed using a computer:
“On a computer screen.” Questionnaire.
Linked to this last point, the questionnaire contained a question
“Consider if the examination papers were presented ‘on-screen’. What
would be the advantages and disadvantages for YOU?” Responses are
summarised in Table 6.
Table 6. Advantages and disadvantages of onscreen examination
presentation. Answers grouped by authors
Advantages (frequency)
General positive access (e.g. ‘easier to read’) (7)
Easier navigation (e.g. no page turning, wouldn’t miss out
questions) (5)
Individualised format (4)
Disadvantages (frequency)
General negative access (e.g. ‘difficult to see’) (5)
Visual fatigue (eye would tire over time) (4)
Difficult navigation (e.g. scrolling) (2)
Computer crash (1)
As might be expected, the identified advantages and disadvantages
seem slightly contradictory – some participants identifying advantages in
the accessibility / navigation afforded by the computers, some identifying
disadvantages. It’s difficult to be certain when drawing upon small
written quotes, but perhaps the positives can be summarised as a view
that the computer enables individuals to change the formatting according
to their own preferences and use the computer functions to navigate
through examinations. These points were raised in group discussions:
Evaluation of alternative exam presentation methods for pupils with low vision
18
“Flipping- pages gets annoying, especially when it's stapled
in certain places. It stops you getting to certain bits of the
booklet. So tapping the keys getting up and down to get to
earlier questions later questions would help.” Discussion
group 1.
“It’s brighter so you can see clearer. And you can change
the font size, the background colour, and the font colours...”
Discussion group 2.
“I don't want to sound lazy or like I would take advantage,
but my hands do get tired and my neck gets craned when I
write a lot like in English. [..] In English exams that would
help if I could do it on a laptop instead of just [laughs]
crouching down for 2 1/2 hours!” Discussion group 3.
“[Using a laptop in an exam would be a good thing because]
it would be bigger I would not have to keep flipping pages.
Annoying.” Discussion group 1.
In contrast, others were less convinced by this, feeling that their
preferred format could not be accommodated in this way and navigating
(e.g. scrolling) was difficult. Again, these and other related issues were
raised in the group interviews:
“No, I wouldn't use it in an exam. [..] I just prefer to write in
an exam. [..] Plus if I write, I can write it out in a certain
way... But if you do it on a laptop you have to do in a
document... it's hard to do things like spider diagrams,
when you're planning.” Discussion group 2.
“It depends on what kind of exam paper because it was just
questions it wouldn’t be too bad. But if it was like science,
with diagrams or maps, I think you would be quite tricky to
look at the whole thing at the same time. Because of using
zoom text...” [Researcher: the paper enables you to see
‘the whole’] Discussion group 2.
Perhaps some of these problems could be overcome with training and
practice in the use of tools which allowed participants to use computers
more flexibly and skilfully.
Evaluation of alternative exam presentation methods for pupils with low vision
19
However, more challenging is the issue of ‘visual fatigue’ which was
raised by four participants independently in the questionnaires (all of
whom also recognised advantages to using a computer, so they were not
all together dismissive of such a strategy), e.g.
“My eyes would start to ache from staring at a screen for a
long time. It would set off my nystagmus.” Questionnaire.
“Eyes would get more tired. [..] The half-hour, yes. But, say,
for a two hour exam, no... where you're looking at it
constantly.” Discussion group 2.
Similarly, although only one participant raised concerns about computer
crashes (and therefore loss of work) this is also an important point which
is raised in the interviews with professionals. Some strategies can
overcome some of the dangers (e.g. periodic print outs and backups),
but this adds to time pressures and general worry at a stressful time.
The potential of computer-based examinations and preparing young
people to access information are explored throughout this report as key
themes.
Additional themes raised in the group discussions
During discussions about laptops it was encouraging to note that many
used laptops in class, and most described being able to touch type. The
use of computers was also raised in the interviews with professionals
highlighted elsewhere in the report. Even so, some participants were
anxious about appearing to be treated and looking different (whether in
exams or the school generally), or identified other barriers to using
computers:
“I have a laptop but I don’t use it at School. [Because] it’s a
mainstream school and I don’t feel comfortable using it in
classes. It doesn’t seem right.” Discussion group 2.
“I don’t have [a laptop]. Not allowed. Girls in the school
were jealous of it so they stopped me using it.” Discussion
group 2.
“The head teacher said I could bring [the laptop] in, but most
teachers said they did not want me to use it. Got angry
because I was typing on a laptop. [..] I can look at the board
Evaluation of alternative exam presentation methods for pupils with low vision
20
at the same time as writing. So it’s quicker…” Discussion
group 2.
The vignettes are poignant as they suggest access to information for
these students is being compromised. These points also have
similarities to studies which have found that low uptake of low vision aids
(LVAs) in schools is partly explained by students not wanting to appear
different (e.g. Mason, 1999). In keeping with this over half of the
participants in this study (12) said they did not use an LVA yet all of them
described preferring a large font size.
Discussions suggest that there was a lack of awareness amongst the
participants about the examination modification process (even though
many of them would be taking examinations in the following months, and
had taken examinations in the past). This was most clearly illustrated by
participants sometimes having an expectation of having examination
formats which are not available within the current modification system.
Linked with this, participants had mixed experiences of practicing
examinations in the relevant format:
“I’ve tried mock exams but they were not the right format.”
Discussion group 2.
“In mock exams I never ever have an enlarged paper like
that [referring to test material]. But in exams [I do]. [..] The
resource base makes the decision for me.” Discussion
group 2.
There was also some (limited) discussion about where the modification
system had not worked in previous exams participants had taken:
“Some reason it always seemed to be me who gets the
wrong one... or they only photocopy half the paper ... so
have to stop my exam and everyone else can carry on. It's
rubbish.” Discussion group 2.
[Participant] gave an example of exams not arriving until a
day late and the LSA having to photocopy the normal paper
onto A3 sheets, “The size is quite annoying because it was
covering most of the desk.” Discussion group 3.
As this report develops we identify a tension between normal classroom
practice (in terms of curriculum access) and options available in
Evaluation of alternative exam presentation methods for pupils with low vision
21
examinations. There are clearly multiple reasons for these tensions, but
it suggests that students may be taking examinations in relatively
unfamiliar formats. The experiment described in this section illustrates
the significant impact that familiarity can have on performance.
This general point about difference between normal classroom and
examinations practice exists irrespective of whether the current
examination modification process works smoothly. Even so, some of the
examples given by students suggest that examination modification
process does not always work smoothly. Both these themes are
developed further in the interviews with professionals.
Evaluation of alternative exam presentation methods for pupils with low vision
22
3. Case examples: Students who prefer
very large hardcopy print (N24+)
3.1 Introduction
The research team carried out semi-structured interviews with five young
people who were identified as preferring to use enlarged hardcopy print
in a font greater than 24 point (i.e. greater than the maximum sized print
provided by examination boards through existing access arrangements).
During the interview participants were asked to comment upon different
font sizes in terms of their preference using a ‘font size questionnaire’,
and then carried out matched ‘examination-style’ tasks which compared
paper-based formats prepared using existing large print approaches
(enlarged to 24 point bold Arial, with a LVA if required) to a format based
upon their own preference. The aim was to examine whether either
format gave any reported advantage to the participants. The participants
also commented upon onscreen presentations of the same examination
questions.
3.2 Method
Participants
Five participants took part in the study. All had low vision, were print
readers and were interviewed separately. Three of participants attended
a special school for visual impaired students (and had not yet taken any
GCSE examinations). Two of the participants had attended mainstream
schools in the past and were currently attending mainstream colleges. A
summary of the participants is presented in Table 7.
Evaluation of alternative exam presentation methods for pupils with low vision
23
Table 7. Characteristics of the participants taking part in case
examples (N=5).
Participant
number
Name
Age
Preferred font
(based upon
information given
prior to interview)*
36 point Arial Bold
36 point Arial Bold
36 point Arial
40 point Arial
42 point Verdana
Preferred font
(based upon
information given
at interview)*
1
Keith
12
36 point Arial Bold
2
Ruth
14
36 point Arial Bold
3
Yvonne
12
36 point Arial
4
Danny
19
28-30 point
5
Sarah
19
Unsure/ 36 point
Arial Bold
* Note: this information was not always consistent between prior and
interview information.
Materials and procedure
The researcher asked the participants about the size of print they liked
and used by administering the ‘Font size questionnaire’ (see Appendix
2). Information gathered included: preferred font style and size and also
some details about LVA and computer use for accessing print. This was
followed by asking participants to read through alternative print sizes
(Arial bold 18, 24, 28, 32, 36 and 40 point) and for each providing their
opinions about ease of access (generally and in an examination). At the
end of the questionnaire, students were asked again about the font they
preferred to use overall.
Content for two ‘pretend’ examination-style tasks was created (version A
and B) taken from materials developed for the student performance tasks
(previous section). However, the tasks were much shorter using only
three multiple choice questions taken from those materials. Both
versions A and B had the same structure and length and were matched
for content difficulty. The format of version A was based upon the
students preferred hard copy large print format (see Table 7) which had
been prepared prior to the interview. Version B was based upon Arial
bold 24 point (on A4 paper). It is worth noting that the research team
created the materials using Microsoft Word (the RNIB E-formatting
software proved too difficult for the process as it did not give the control
regarding line spacing and page breaks – this is discussed in a later
section of the report, “Technical analysis of the RNIB E-formatting
software”).
Evaluation of alternative exam presentation methods for pupils with low vision
24
As part of the introduction given in each session, it was emphasised to
participants that we wanted them to complete all questions and that we
were testing the quality of the material and not testing them. For this
reason the questions were very easy (containing relatively simple
general knowledge and logical questions).
The researcher first asked participants to answer question 1 of version A.
The researcher noted observations including the time taken and asked
the participants for their opinion of the task (“How did you find that?”).
Once question 1 was completed, the same took place for question 2 and
3 (together). After completing version A, a similar procedure was
followed for version 2 questions. In most cases, participants could not
access this version without the use of a handheld LVA or CCTV
(participant 2 could read the version 2 text without an LVA).
A final part of the structured interview asked participants about
advantages and disadvantages of each of the format versions, and which
they preferred. In addition, alternative onscreen versions of the
questions were also presented to the participants using the RNIB Eformatting software (which gave control over font and background colour
as well as font style and style). Participants were encouraged to read
from the screen, and comment on the different presentations available.
[Note: The laptop computer used for the demonstration had a relatively
small screen (approx A4 landscape sized). The font sizes selected using
the RNIB e-formatting software were not reproduced exactly to the
chosen font size on the screen and the sizes reported in the report below
are estimates based upon visual inspections of the screen presentation.]
A final question asked participants about the advantages and
disadvantages of onscreen presentation.
Each interview took between 45 minutes and one hour. At the beginning
of the session the broad purposes of the research was explained and it
was emphasised that the material was being tested rather than the
participants themselves. Participants were then asked to sign a consent
form declaring that they were willing to take part in the research, but it
was noted that they could withdraw at any time. Participants were each
given a £10 gift voucher.
Evaluation of alternative exam presentation methods for pupils with low vision
25
3.3 Results
Case Example: Keith
Discussion about fonts
Keith described his preferred font size and style as 36 Arial (bold) without
the use of an LVA. He also said he used this in his day-to-day classroom
work. Even so, he described usually using a computer with JAWS and a
large font in most lessons. His profile using the font size questionnaire is
in the table below. Keith felt he would prefer size 40 for an examination.
Table 8. Keith – Font size questionnaire responses
Size
Can you read?
Could you read for a long piece
of text?
18
OK
No
24
OK
No
28
Easily
No
32
Easily
Don’t know
36
Easily
Yes
40
Easily
Yes
Observation of Keith reading questions
Table 9. Keith – Observations when reading hardcopy print
questions
Version
Time
Notes
Version 1: Arial
120 seconds
Worked very close to material.
(bold) 36 point
Described finding it ‘easy’ and
‘like he does in the classroom’.
Version 2: Arial
(bold) 24 point with
CCTV (white on
black)
130 seconds
The CCTV enlargement was
larger than 36 point and Keith
appeared to have a greater
working distance. He navigated
the questions without obvious
difficulty and skilfully used the
movable reading table.
Keith used a thick black pen to write his answers. Writing was very slow,
and Keith described that he would usually write using a computer.
Evaluation of alternative exam presentation methods for pupils with low vision
26
Overall, for the two multiple choice questions, Keith was marginally
slower when using 24 point with the CCTV. Nevertheless the difference
in working speed for this short multiple choice task did not seem obvious.
Preferences and screen use
Keith found it quite difficult to explain his preferences in the short
interview. However, when asked his preference between 36 point or 24
point and CCTV he clearly chose 36 point, saying he was “used to it”
(referring to classroom practice). Keith also noted that 24 point without
the CCTV would be impossible for him.
When asked, given any choice, what format he would like for
examinations, Keith said 40 point Arial Black (on paper). When
prompted, Keith did say he would like to use a computer (and later
having seen the examination paper onscreen felt that the computer
presentation was his overall preference - again this was prompted).
When reading the examination questions from the computer screen,
Keith worked very close and used approx 48 point1 on screen and felt he
would prefer bigger). Advantages Keith felt the computer gave him were
greater control of font size and style, as well as colour combination (Keith
chose white font on black background). Disadvantages were seen as
screen reflection. When prompted whether he thought onscreen would
be tiring he replied ‘not really’.
Case Example: Ruth
Discussion about fonts
Ruth described her preferred font size and style as 36 Arial (Bold),
without the use of an LVA. However she said her usual font size in her
day to day work was 24 (without an LVA). She did have an LVA, but
usually had large print prepared. She also said she usually used a
computer in class. Her profile using the font size questionnaire is
presented in the table below.
1
Note: the onscreen fonts are approximate because they are a product of the selected font size and
monitor size of the laptop computer used. Estimates were made afterwards based upon comparison
with standardised printed material.
Evaluation of alternative exam presentation methods for pupils with low vision
27
Table 10. Ruth – Font size questionnaire responses
Size
Can you read?
Could you read for a long piece
of text?
18
OK
No (“eyes hurt”)
24
OK
No
28
Easily
No
32
Easily
Yes
36
Easily
Yes
40
Easily
Yes
Observation of Ruth reading questions
Table 11. Ruth – Observations when reading hardcopy print
questions
Version
Time
Notes
Version 1:
50 seconds
Used a ‘tilt-top’ table. Seemed to find it
Arial (bold) 36
easy. Described it as “straightforward”.
point
Good posture, maintaining a relatively
normal reading distance.
Version 2:
35 seconds
Worked quickly, partly linked to
Arial (bold) 24
anticipating the question structure.
point
Worked closer to the point: “I struggled
- had to go closer than the 36…” Ruth
also wondered about getting tired.
Preference and screen use
Ruth preferred the 36 point compared to the 24 point (although she
worked through the 24 point quicker, most likely she was anticipating the
question structure having done it second). She felt the 36 point was big
and easier to read, although it was “too bold”. She felt the 24 point was
too small, although she thought it was “set out nicer - less room”.
Ruth looked at the onscreen version and discussed different sizes of
font. She felt the screen might be easier to access, and of all the
presentations discussed she felt the computer was the best choice for
her. Even so, when prompted about whether it might be tiring, she felt
her eyes did get tired when using a computer.
Case Example: Yvonne
As well as her visual impairment Yvonne also had a hearing impairment.
Evaluation of alternative exam presentation methods for pupils with low vision
28
Discussion about fonts
Yvonne described her preferred font size and style as 36 Arial (Bold)
without an LVA. She used 24 point with a CCTV. She had an LVA
which she used with books, and described using a computer
‘sometimes’. Her profile using the font size questionnaire is presented in
the table below. Following this discussion Yvonne felt that she would
prefer font size 32 for her day to day work, and 28 point for examinations
(the smaller font would mean that there would be “less page-turning”).
Table 12. Yvonne – Font size questionnaire responses
Size
Can you read?
Can you read for a long piece of
text?
18
No
No
24
OK
No
28
Easily
Yes
32
Easily
Yes
36
Easily
Yes
40
Easily (“too big”)
Yes
Evaluation of alternative exam presentation methods for pupils with low vision
29
Observation of Yvonne reading questions
Table 13. Yvonne – Observations when reading hardcopy print
questions
Version
Time
Notes
Version 1: 90 seconds
Used a ‘tilt-top’ table. Worked slowly
Arial (bold)
and deliberately. Described she was
36 point
“happy” with the format. Yvonne
needed help with one of the questions
(‘Who was the first person to walk on
the moon?’).
Version 2: 125 seconds
Seemed to work obviously slower
Arial (Bold)
than when using 36 point, particularly
24 point +
navigating the paper (Yvonne did not
CCTV
use the CCTV movable reading table,
(white on
but moved the paper), although she
black)
skilfully held her place with her finger
when answering a question.
Described the format as “easy”.
Yvonne needed help with one of the
questions (‘Which person is a
footballer’).
Preference and screen use
Yvonne did not express particular preferences for any use of the formats
she used in the activities (although she was very shy and reluctant to
speak very much). When she tried the onscreen format (Arial bold,
approximately 48 point) she worked at a working distance of about 25cm
with a comfortable posture. She felt that working from a computer might
be easier to see because it can be made bigger. When prompted about
eye strain, Yvonne said “maybe” noting that when she worked on her
computer at home for a long period her eyes did water. When reflecting
upon all the formats she had tried (36 point, 24 point with CCTV, an
onscreen computer), Yvonne felt all were the same and she had no
preference.
Evaluation of alternative exam presentation methods for pupils with low vision
30
Case Example: Danny
Discussion about fonts
Danny described his preferred font size and style as 28-30 without the
use of an LVA (different to the Arial 40 point we had anticipated). Ideally
this would be 30 without the LVA but smaller if he was to use a
magnifier. He mentioned Calibri as his preferred font but clarified this by
saying that as long as the font was clear he could read it. He used a
range of font sizes in his day-to-day College work. He reported his vision
had deteriorated recently and he was now learning Braille. He presented
the three LVAs he used at home – a hand magnifier; one with an internal
light and an Amigo portable video magnifier (CCTV). He uses a
computer with Zoomtext in his College and at home. His profile using
the font size questionnaire is presented in the table below. Of all the
fonts presented Danny felt he would prefer 32 point for day to day work
and for examinations – although he thought 30 point would still be
appropriate for him.
Table 14. Danny – Font size questionnaire responses
Size
Can you read?
18
24
28
No
No
OK/Hard
32
36
Yes
Yes
40
Yes
Could you read for a long piece
of text?
No
No
No “uncomfortable for a long piece
of text”
Yes
Yes “Maybe too big for long piece
of text”
“Definitely too big”
Observation of Danny reading questions
Given that there was some ambiguity about Danny’s preferred font, the
researcher showed Danny various sizes of the examination questions
(Arial bold 28, 36, 40, and 24 points). Danny preferred not to write down
answers but just to verbalise them (he was used to doing this with a
‘reader’ in previous exam situations). He tended to use a computer to
write (only using a pen for writing his signature).
Evaluation of alternative exam presentation methods for pupils with low vision
31
Table 15. Danny – Observations when reading hardcopy print
questions
Version
Version 1:
Arial (bold)
28 point
Time
210 seconds
Version 1:
Arial (bold)
36 point
105 seconds
Version 1:
Arial (bold)
40 point
110 seconds
Version 2:
Arial (bold)
24 point (no
magnifier)
N/A
Version 2:
130 seconds (not
Arial (bold)
including pauses
24 point (with to adjust device)
Amigo CCTV)
Notes
Read slowly and quite hesitantly –
tended to lean forward slightly to
view paper on table did not appear to
have a comfortable posture.
Commented that did not find the
Roman numerals easy to read or
locate on the page. Commented that
found “the text too small to read
comfortably”.
Much more fluent reading – more
comfortable body position (Danny
noted this in discussion afterwards).
Commented that he found the paper
“easy” to read and had no problems
in accessing the information. Also
commented that he found this more
comfortable for “longer reading”.
Similar to 36 – more fluent than 28
and comfortable viewing position.
Commented that he found this paper
to be very clear and could see it well.
Could not read – brought paper close
to his eyes but gave up trying “Sorry I
can’t see this”.
Appeared to be competent using the
CCTV independently– he read
fluently once he had made
adjustments. Able to adjust the
magnification independently to alter
the viewing size (started large with
only 1-2 words on the screen then
moved down so could approx 3-4 at
a time). Was comfortable using the
device although he was still getting
used to using it for extended reading
(it was relatively new to Danny).
Evaluation of alternative exam presentation methods for pupils with low vision
32
He had a go at the paper with a hand
magnifier and could read this (almost
word by word) albeit slowly. Danny
felt he would struggle to use this for
extended reading (tended to use this
for seeing instructions on tablet
boxes and similar).
Preferences and screen use
The CCTV allowed Danny to access the paper and he could read this
relatively fluently once he had found the correct magnification for the font
size. He assumed a comfortable sitting position when using the CCTV
although he was not sure whether it would be comfortable when using it
for a long exam. Danny did not use the CCTV much at present as more
reliant on the computer for access – and for hand magnifier for small
amounts of text. On balance he felt he would prefer the larger font (i.e.
36 in this case) rather than 24 plus CCTV (mainly due to comfort in using
the device for extended period and the fact that he was still becoming
familiar with the CCTV).
Danny was not sure about preference of font for exams for him but felt it
had to be larger than 30 and no bigger than 40. As indicated above, the
font style itself was not considered an issue as long as it was clear. On
balance he preferred bold if this was available.
Discussion about onscreen access was interesting in that he seemed
much more comfortable reading from a screen rather than from paper
(both in terms of seating position and in reading fluency, and seemingly
with a slightly smaller font). We explored several options with the
software (Danny selected various Arial font sizes he wanted to try out):
 24 Arial bold – seemed very comfortable with this –good seating
posture and fluent reading. Felt that the tables (which had low
contrast lines) could be improved.
 22 Arial (first not bold then with bold) – without bold leant forward
to get closer to screen – with bold resumed a similar distance to
that used for 24 Arial (bold).
 34 Arial – could do this but felt it was too big and would be too
slow.
Evaluation of alternative exam presentation methods for pupils with low vision
33
On balance, Danny felt that 24 Arial bold “stood out best” and was
“clearest” for him. Alternatively, he felt he could use 22 Arial bold but not
for long periods of reading on screen.
In terms of advantages of onscreen access, Danny felt: that the text was
clearer compared to paper (he made reference to contrast), it’s possible
to do all changes to the exam format before starting in a short period of
time (here Danny was making specific reference to the RNIB Eformatting software, or similar software to it), and he would be more
comfortable sitting the exam using a computer than paper (in terms of
familiarity).
In terms of disadvantages, again Danny referred to the feature of the
RNIB E-formatting software which was used and noted that the
writing/input of answers was problematic. While he felt he could toggle
between documents, he would ideally prefer if he could input directly into
exam document.
On balance he felt that he would prefer to access an exam through this
route rather than a paper version (irrespective of software). Danny was
impressed by features of software (trying out various options in terms of
colour/contrast) and felt it could be “very useful to people like me in the
future”.
Case Example: Sarah
Discussion about fonts
Sarah was not sure about her preferred font size but thought it was Arial
bold 48 point. Her mother made reference to a report that said it was
Verdana 42 without the use of an LVA. With an LVA she thought she
used 36 point. Both noted that Sarah had a deteriorating condition and
she was learning Braille. She has recently started using an Amigo
compact CCTV. She was about to start college so did not yet know how
she would access information – in school text had been blown up for her
and presented on A3 paper (no access to LVA nor did she know what
font had been used). She used Zoomtext software on a desktop
computer at home and was hoping to purchase a laptop prior to starting
college. Her profile using the font size questionnaire is presented in the
table below.
Table 16. Sarah – Font size questionnaire responses
Evaluation of alternative exam presentation methods for pupils with low vision
34
Size
Can you read?
18
24
28
32
36
No
No
No
No
Hard (very slowly and
with paper close to
eyes - <3cm)
OK (a bit easier to
read, but still held
very close and not
comfortable reading
this)
40
Could you read for a long piece
of text?
No
No
No
No
No
No
After the font size questionnaire discussion, in another discussion about
font size Sarah felt she would prefer a font of between 42-48 point
(“anything bigger than 40 would be OK”), and for examinations ideally 48
point.
Observation of Sarah’s reading questions
As with Danny, there was some ambiguity about Sarah’s preferred font.
The researcher showed her the largest printed material which had been
prepared (Verdana 40, with and without bold), followed by Arial bold 24
point. Sarah preferred to read out her answer rather than write them
down (she tended not to write using a pen apart from signing forms).
Evaluation of alternative exam presentation methods for pupils with low vision
35
Table 17. Sarah – Observations when reading hardcopy print
questions
Version
Time
Notes
Version
Approx 240
Brought paper close to her eyes – very slow
1: 40
seconds
(read word by word). Lost track of where
point
question ended and response was required.
Verdana
Did not see response boxes initially – asked
(then
where answer would go. Ring in Q2 not clear
with bold)
– and had problems with Roman numerals.
We tried the same paper in bold Verdana – a
lot clearer and more fluent (though could
have been the result of practice) and much
more confident in tackling the questions.
Commented the ring (‘circle the correct
answer’) in Q2 was not bold. Not timed in
bold as it was same paper but generally
appeared a lot more accessible
Version
2: Arial
(bold) 24
point
(with and
without
Amigo
CCTV)
N/A
Without CCTV: Struggled to read – very slow
– read half of Q1 then gave up.
With CCTV: Could access the paper but had
difficulties in finding correct magnification size
(started on maximum giving only one word on
screen at a time). Once Sarah had found
appropriate settings, she was able to read
through the paper to the end and answer the
questions – seemed to enjoy the experience
(after previous attempts when she had got
quite dispirited). We did not time this as she
was struggling to use the technology (which
was new to her) – but generally much more
confident in her reading once she had found
the appropriate setting.
Preference and screen use
The Arial bold 24 point was a real struggle for Sarah, but with the CCTV
she could access the text and was far more confident in her reading.
This raises issues regarding familiarity with the technology – Sarah
required time to find the appropriate adjustment for the 24 point text.
Evaluation of alternative exam presentation methods for pupils with low vision
36
However, once this had been done she was much more comfortable with
using the technology and we tried using it with other font sizes for
comparison. Even so, Sarah felt she would not want to use the CCTV for
a long paper (though she was happy using it with the short three
question paper that was used in the study).
We explored the onscreen version and discussed different sizes of font.
Sarah tried a format in Verdana 32 point – could read this fluently on
screen – able to independently scroll down the page – and could answer
the question without any prompting. Next, Sarah tried Arial bold 32 point
which was read fluently (but she had difficulties in locating the table lines,
which were not bold). She was happy to explore other fonts – Arial 40
bold was equally accessible. Contrast of some of the pictures may have
been problematic.
Overall, Sarah was very confident and comfortable with onscreen access
and she felt she would prefer her exams to be presented in this way if
possible (CCTV fine for small amounts of text). She felt the advantages
were: more independent; less frustrating; easier to access.
Disadvantages identified were: more time as need to scroll down the
page (in terms of amount of text on screen); contrast in diagrams and
tables “needs to be sorted”; hurts eyes if looking at computer for too
long.
3.4 Discussion
The aim of these additional case examples was to gain an insight into
the print access challenges faced by students who prefer to use
hardcopy large print in font size greater than 24 point. In terms of the
number of students this applies to in the UK, findings of a recent survey
of 21 teaching services carried out in England and Wales gives some
insight. Table 18 illustrates the range of preferred reading formats these
services report they currently provide with over 9% of supported pupils
reported to prefer font size greater than 24 point. This suggests that the
five participants who took part in these case examples are by no means
unusual in their preference for 24+ point hardcopy large print.
Evaluation of alternative exam presentation methods for pupils with low vision
37
Table 18. Visually impaired students (aged 13-16 years) preferred
reading formats as reported by 21 education services in England
and Wales (N=287 pupils).
Preferred
Total (N)
Percentage
reading format
12 point
57
19.86%
14-18 point
113
39.37%
20-24 point
69
24.04%
>24 point
27
9.41%
Braille
18
6.27%
Electronic
3
1.05%
Total
287
100.00%
Although the case examples provide relatively limited qualitative data,
they do allow some useful comparisons and observations which are
relevant to the current study:
A comparison between bespoke hardcopy large print (in the
student’s preferred size, greater than 24 point) and Arial bold 24
point (as is available through existing access arrangements).
In four of the five case examples participants felt they would find it too
hard to access the Arial bold 24 point material without additional
magnification. Ruth was an exception and was able to access the
material with relative ease, although she felt it was too small for
sustained work. This suggests that for all these participants, with the
possible exception of Ruth, the use of bespoke hardcopy large print (of
various sizes greater the 24 point) was needed if they were to access the
print without the use of additional magnification devices. This leads to a
follow-up comparison:
A comparison between bespoke hardcopy large print (in the
student’s preferred size, greater than 24 point) and Arial bold 24
point with additional LVA if available.
All four of the participants who required additional magnification were
able to access the Arial bold 24 point when using an LVA. In all four
cases they used a type of CCTV (Keith and Yvonne used a large deskbased system, while Danny and Sarah used a portable device). It is
encouraging that all four could access the material in this way (and it
may be possible for them to access smaller sized print in this way for
Evaluation of alternative exam presentation methods for pupils with low vision
38
particular tasks). Nevertheless, Keith, Danny and Sarah all said they
would prefer the bespoke large print for an examination (Yvonne did not
express a preference). Although given the structure of the activities it is
difficult to be categorical, participants were probably slower when
working with the CCTV and 24 point compared with the bespoke
hardcopy large print. It is a delicate and speculative point which is
discussed elsewhere, but potentially all participants would have
improved their speed of access using the CCTV with practice. Both
Danny and Sarah were new to using their portable CCTVs and even
within the space of the short interview they improved speed and
confidence as they learnt to use the magnification facilities more
efficiently.
The case examples also provide insight to a third area of interest:
How do participants respond to the on-screen computer
presentation as an alternative option?
Four of the five participants expressed an overall preference for the
potential of onscreen computer presentation (Yvonne expressed no
overall preference). It was the control of presentation the screen gave
them which seemed to be main advantage (and this was expressed in
more detail by the two older participants, Danny and Sarah). In all the
cases the participants were using computers in their studies to some
extent. Interestingly, the computers offered potential for writing answers
as much as reading the questions – an important observation was the
difficulty Keith, Yvonne, Danny and Sarah had writing their answers and,
perhaps unsurprisingly, this seemed as big a problem as reading the
print. For these participants, answering examination questions had
required (or would in the future) the support of an amanuensis (scribe) or
computer. Danny raised this issue directly, suggesting that while he
could toggle between electronic question and answer documents on the
computer, he would ideally prefer if he could input directly into an
examination document.
Given the advantages of the onscreen presentation, and the common
use these participants make computers anyway, the use of onscreen
formats for examinations seems a better option for many of these
students. Even so, some of the students did mention (although generally
following prompts from the interviewer) the potential of visual fatigue
associated with using a computer screen.
Evaluation of alternative exam presentation methods for pupils with low vision
39
4. In-depth interviews with staff involved in
material modification
4.1 Introduction
The research team carried out semi-structured interviews with education
professionals who support young people with visual impairment. The
aim of the interviews was to explore the potential of the E-formatting
software with explicit reference to existing modified large print (‘MLP’)
procedures. The interviews also considered how the use of the Eformatting software could be implemented in different contexts. In
addition the interviews discussed the potential of on-screen presentation
of examinations.
4.2 Method
Participants
Eleven participants took part in the study in nine separate interviews. A
list of the participants is presented in Table 19. Participants were
recruited in a variety of ways: contacts offered by the project advisory
group, contacts sought through head of services in the midlands regions,
and a trainee QTVI who was interested in project work in the area of
examinations access. All the participants had experience of working with
visually impaired students who were taking examinations as well as prior
knowledge of modified large print (‘MLP’) procedures.
Evaluation of alternative exam presentation methods for pupils with low vision
40
Table 19. Characteristics of the professionals taking part in the
semi-structured interviews (N=11).
Interview and format
Job title / setting
A - face-to-face
Senior practitioner for a visiting teaching
service (VTS)
B - face-to-face
QTVI in VTS
C - face-to-face
QTVI in VTS
D - face-to-face
Senior Technician for resource base and
VTS
E - face-to-face
Teaching Assistant (TA) in mainstream
school (supporting one child)
F - telephone
Trainee QTVI in VTS
G - telephone
QTVI in VTS and resource base
H - face-to-face
QTVI in VTS and resource base
H - face-to-face
Teaching assistant in resource base
I - face-to-face
Lecturer in specialist FE college
I - face-to-face
Lecturer in specialist FE college
Interview schedule and procedure
The semi-structured interview schedule was designed around the two
project research questions:
1. How do the Modified Large Print (MLP) versions of papers
produced by RNIB E-formatting software compare to traditional
MLP papers? (both process and outcome)
2. What methods of accessing examinations are useful for the future,
and can the RNIB E-formatting software (or a developed version of
it) play a useful part? (e.g. electronic copies, interactive papers)
The sections of the interview schedule were designed to provide a
framework in which the researcher and interviewee could ‘step through’
the process of converting a standard GCSE paper to a MLP paper, and
consider the possible role the RNIB E-formatting software could have in
this process. The interview drew upon a 2009 GCSE examination paper
(OCR, Twenty First Century Science, Biology A, A221/02). This paper
was selected as it was a complex paper that incorporated a variety of
question types, and a MLP and E-formatting software (‘Software’)
version had been produced for visually impaired pupils in 2009.
Evaluation of alternative exam presentation methods for pupils with low vision
41
The interview schedule (presented in full in Appendix) was structured
around three sections. Section 1 explored hard copy examinations
(focussing upon the primary research question). The interviewer
presented the hardcopy original paper and then discussed this in terms
of student familiarity and ease of access. A similar format was used for
the MLP version of the paper. For the Software version two hardcopy
versions were shown as examples (18 point Comic Sans, and 28 point
Arial) .A demonstration of the pilot software was then provided by the
interviewer on a laptop to illustrate its potential to create versions of the
examination in a variety of font sizes and styles (and colours of paper).
Section 2 explored the potential of onscreen examinations (focussing
upon research question 2). The interviewer presented the original, MLP
and Software examination papers onscreen (the original and MLP were
PDF files and used Adobe Acrobat). It was emphasised that students
would be able to use access tools to format the onscreen presentations.
The focus of section 3 was on the ‘perfect exam scenario’. Within this
section, participants were asked to:
 Reflect on the type of examination provision for visually impaired
pupils that they would you like to see in the future (one of the
prompts included as part of this was a distinction between
‘students controlling the access’ versus ‘materials being prepared
to his/her specification’).
 Discuss the specific quality of the formatting and modifications of
the example examination paper.
 Discuss the ‘principles’ under discussion, i.e. ‘the proof of concept’
of the RNIB E-formatting software (and potential of the onscreen
examinations).
Procedure
Seven of the interviews were carried out in person in a quiet room. The
interviewer brought hardcopies of the examination papers and
demonstrated the RNIB E-formatting software and onscreen papers
using a laptop. In the case of the two interviews carried out over the
telephone, materials were sent to participants in advance and the
participants had a computer available during the interview.
At the beginning of the interview the broad purpose of the research was
explained and the structure of the interview described. Participants were
Evaluation of alternative exam presentation methods for pupils with low vision
42
then asked to sign a consent form declaring that they were willing to take
part in the research and that they could withdraw at any time.
With permission from the participants, all the interviews (with one
exception) were audio recorded. The researcher worked though the
interview schedule and took key notes. Each interview took between 1
and 1.5 hours. When the interview was complete, the researcher wrote
up the notes (drawing upon the audio recordings as an aide memoire
and the source of verbatim quotes when helpful). These interview
transcripts were analysed as described below.
Analysis of interview transcripts
The qualitative data analysis tool NVivo 8 was selected to help the
research team make ‘sense’ of the data. The transcripts were coded in
order to draw out themes. The coding structure was constructed by:
(1)
Ensuring it was grounded in the responses provided by
participants as captured in the transcripts of the interviews.
(2)
Drawing upon the project aims and research questions (and
how these were reflected in the interview schedule).
(3)
Drawing upon discussions within the research team following
the interviews.
(4)
Drawing upon broader literature in relation to curriculum access
and visual impairment.
(5)
Drawing upon feedback from the advisory group.
NVivo software was used by author 1 to scroll through the transcripts,
highlight text and to ‘code’. These codes were then described and
illustrated with quotes within a first draft of the report. This draft was
shared with other authors and modified and refined further. Such a
process of analysis is necessarily subjective but seeks to:
 provide an authentic representation of the interviews and professional
views;
 connect with a broader literature and policy context;
 develop practical and useful recommendations.
4.3 Analysis of results
Unsurprisingly the identified themes partly reflect the interview structure.
Further, as described in the method, some participants placed more
attention on a detailed analysis of the quality of the modified papers
Evaluation of alternative exam presentation methods for pupils with low vision
43
while others emphasised the principles and potential of different
approaches. The themes are arranged under the following headings:
1. ‘The devil is in the detail’: Technical analysis of MLP and Software
papers
2. The potential of RNIB E-formatting software
3. Managing the examination process using the software
4. Tensions and contrasts: classroom practice, examination options and
the ‘real world’
5. Access to the curriculum: student skills and service policies
6. The potential of onscreen examination presentation
7. Different views of a ‘perfect world’
An emerging ‘story’ running through the analysis is an interesting
distinction between two higher level approaches to exam modification
and access, namely ‘a student-centred approach to curriculum material
preparation’ verses ‘an independence-skills approach to curriculum
access’. This distinction was not always explicitly articulated by
participants but provides a useful means of understanding the challenges
of examination access faced by young people with visual impairments,
and hopefully ways of providing solutions.
1. ‘The devil is in the detail’: Technical analysis of MLP and
Software papers
The interviews generated detailed technical analyses of the quality of the
examination papers. Key features of included:
 Precise opinions about how things should be done in relation to exam
modification. For some participants this overlapped with a feeling of
dissatisfaction with the existing MLP procedures and the quality and
consistency of some of the examinations they had received.
 A number of participants made reference to the JCQ Best Practice
Guidance (JCQ, 2008), and had detailed understanding of its
contents.
 Commonly (though not exclusively), participants who were not
teachers raised these issues in greatest detail. This perhaps reflects
their close work with preparation of curriculum materials for visually
impaired students.
 Each analysis tended to be child-centred in nature (often making
reference to particular children with whom participants supported).
Evaluation of alternative exam presentation methods for pupils with low vision
44
A selection of responses for the MLP and the Software paper
respectively is analysed below.
Modified Large Print (MLP)
All participants offered feedback on the quality of the MLP paper with a
number providing a detailed page by page analysis. Three examples are
included below to illustrate the nature of this detailed feedback:
“Too much on a page – [child] would not cope with it all. [..]
If pictures are not relevant I would miss them out
completely.” Participant 5.
“Weight of lines may be a bit thicker - this is a common
problem. Why capitalisation and underline? ….seems over
the top and unnecessary, and possibly confusing. But it is
part of the guidelines...” Participant 7.
All questions could be on one page (Q2); Better clarity on
box needed (Q3); Diagram (Q5) still problematic – lines are
thicker and numbers better but line merges into shading.
Overall an improvement in terms of access and navigation
“but there could be some more improvements.” Participant
1
Participant 4 also identified aspects of the modification which was
described as being ambiguous. She also passed on a precise
documented list of errors from the previous year’s exams that made
reference to JCQ Best Practice Guidance (JCQ, 2008).
The level of detail and knowledge displayed by the participants was
impressive. They reflected upon the difficulties individual students they
work with might have. This detail is illustrated with an example identified
by at least three participants when discussing the modification and
formatting of a ‘sorting’ question (in particular the capitalisation and
underlining). Figure 1 presents a scanned image of the question. This
example is particularly interesting as a similar format was used in the
performance task in our work with visually impaired young people (see 0.
2. Student performance and views of alternative examination papers).
Figure 2 presents a scanned image of the answered question in which
the participant made an error.
Evaluation of alternative exam presentation methods for pupils with low vision
45
Figure 1. Example of a MLP question which several participants
identified as being possibly problematic (Source: OCR, Twenty First
Century Science, Biology A, A221/02). Note: original font size 18
point, reduced to fit on page.
Evaluation of alternative exam presentation methods for pupils with low vision
46
Figure 2. Example of MLP question used in the trial in which a
participant made an error. Note: original font size 24 point (18 point
A4 paper, photocopy enlarged to A3), reduced to fit on page.
Evaluation of alternative exam presentation methods for pupils with low vision
47
It is useful to consider likely factors which led to the participant making
this error. Primarily, the underlined capital F (F) appeared to have been
read as a capital E. Many people without a visual impairment may also
have made such a visual error, except the context of having a list of
letters in the question would make the error unlikely (so much so that
people with normal vision would probably not even notice the visual
ambiguity). However, because this participant has very poor visual
acuity, he held the paper very close. This in turn meant that his field of
view and therefore his contextual cues were reduced.
The example is helpful as it serves to support the points raised by the
professionals with an actual error made by a visually impaired student.
In the broader policy sense it might be argued that the individual needs
of the student are impossible to accommodate within any centralised
system. Alternatively, it could be argued that the system is adequate but
in this case it was not implemented in accordance with modification
guidelines (or the guidelines need to be improved).
We develop this theme elsewhere in the report, including the discussion.
Within this analysis it is useful to note however that participants raised
many detailed points about this particular MLP paper – one which was
complex, but had been considerably, and presumably expensively,
modified from the original. In that sense it appears not to result from
any lack of ‘investment’ or ‘effort’ on the part of the examination board,
modifier or producer.
Participants in fact raised many general points about dissatisfaction with
the quality of examinations they had received in the past (e.g. Participant
2, 4, 7, and 8). This also links with research by others. Cobb (2008)
inspected 33 MLP examination papers and concluded ‘a lack of
consistency between different awarding bodies in the style and quality of
the modified papers produced’ (p.ii).
RNIB E-formatting software
As with the analysis of the MLP paper, participants provided detailed
points about the Software version of the paper. These are described in
more detail in the next section (Technical analysis of the RNIB Eformatting software) and provide some possible areas for development.
Points raised include:
 Line spacing
 Pagination (page breaks, splitting of questions over page)
 Limited control over diagrams (including sizing and size of text)
 Better choice of colours for onscreen presentation
Evaluation of alternative exam presentation methods for pupils with low vision
48
It should be noted that for many participants these points were raised in
the context of a generally positive reaction to the software discussed in
the next section (particularly in relation font size). There is however an
important relationship between ‘formatting' and 'modification' that needs
to be unpicked carefully. As an example, a number of the positive points
raised by participants about the software were in fact linked to
modifications which had been made independent of the E-formatting
software itself. As observed by some participants, the modifications to
the Software and MLP versions were not consistent, suggesting that they
had been designed by different modifiers. As a result, different
modification choices had been made including for example the position
of the checkbox in multiple-choice questions (left or right aligned), the
inclusion or exclusion of pictures. This meant that in practice the
comparison between software and MLP versions was not ‘like with like’.
A similar point was highlighted in the section on student performance
(Student performance and views of alternative examination papers).
This again highlights the critical role of the modifier and the challenges of
ensuring quality, reliability and consistency in the modification process A
possible strength of the E-formatting software therefore is that it could
serve to ‘force’ a particular style of presenting particular questions. For
example, a modifier preparing papers for the E-formatting software may
always choose an agreed method of presenting multiple choice
questions (left aligned check boxes which do not require visual scanning
over space), numbering, text alignment, etc. Such a system may
encourage greater consistency of work (‘within’ and ‘between’ modifiers).
2. The potential of RNIB E-formatting software
Under this second heading, we briefly consider the potential of the RNIB
E-formatting software. As noted above, participants were broadly very
positive about the software, and in particular to the ease within which a
range of font sizes / ‘bespoke’ formats could be achieved. This is
particularly relevant to students who prefer print size greater than the
standard 18 point, and crucially, do not like exam papers presented in A3
format (who were seen as being excluded from current procedures).
This point is illustrated by the following quotes:
“Arial Point 28 paper – much nearer to [child D’s] preferred
font size. [..] Generally is very similar to what I produce for
D… hours and hours of work for one person.” Participant 5.
Evaluation of alternative exam presentation methods for pupils with low vision
49
“This is much more the bespoke. This looks much more like
the sort of work that our access support assistants would
produce for people. [..] It would be ideal if we could do this.
[..] I think that [the RNIB E-formatting software]... To be
able to do it for yourself would be great. That is pretty near
the perfect world.” Participant 3.
“We always recommend A4 but some examination boards
provide A3 which we, and most students hate!” Participant
6.
Many of the participants (1, 2, 7, 10, 11) referred enthusiastically to the
potential of the software:
“First impressions of it, I think it's much, much better. [..]
Brilliant [ potential]. We could definitely use it.” Participant
2.
“In the ideal world they would get what they want at the
press of a button...[laugh] I don't think we're there yet. But it
would be nice to be able to pick up a file to convert it quickly
into what they like.” Participant 7.
There were relatively few negative comments beyond the technical
points described previously (page breaks, formatting of diagrams).
There was however some anxiety about whether the software really
could cope with the formatting required for specialist topics, which for
example required subscripts and superscripts (science and maths), or a
range of fonts (English and drama):
“If you just have something that just automates print size
and background there are other sorts of things that crop
up… [..] For example with poetry once you start modifying
you start to get issues with line breaks, stanzas etc. For
drama you couldn’t distinguish between stage direction and
words...” Participant 10, 11.
Similarly, one participant (8) noted that the flexibility of the software
might lead to unexpected / additional required modifications, for example
cross references to other pages; “circle the correct answer” may be
inappropriate if the size of font means that examination will most likely be
answered using a word processor.
Evaluation of alternative exam presentation methods for pupils with low vision
50
3. Managing the examination process using software
Participants were asked how they might use the RNIB E-formatting
software in their context and how this process might be managed. This is
particularly relevant because under current arrangements services may
have one hour before the exam starts in which to use the software to
prepare and print the examination paper. There were mixed views on
this. Some participants were optimistic about the prospect believing that
the software would serve to simplify a complex and difficult procedure:
“That would be brilliant.” It would also help the service
because it would give them the flexibility producing
materials on the day. Participant 3.
“The point of you going in on the morning of an exam, and
then potentially in the hour [before] being able to access a
program like this and provide the student with size 36 Arial
bold wouldn't be a problem from that point of view. We
have three [support teachers] [..] The benefits are fantastic.
It wouldn't be a problem arranging to go to school on that
morning. Liaising with the exams officer etc.” Participant 2
The same participant (2) felt that they had to go through this complex
process anyway describing an example of how a paper had not arrived
as requested, and frantic photocopying had taken place i.e. working in
this way was typical. Even so, some participants may have been overly
optimistic about the flexibility of the early opening procedures for GCSEs,
drawing only upon their experience of working with SATS papers:
“When we did SATS papers – had to apply for early
opening. [..] Problem you have, is if you make a mistake on
the paper. We had a copy of the modified SATS papers –
not big enough [font] for child. Now have software to scan
documents and text but then had to type it all up. [..] If using
this software could use this the day or hour before – haven’t
got the worry of having to check if you have made any
mistakes.” Participant 5.
Several other participants highlighted the pressure and stress involved in
preparing papers in the short time before the exam suggesting that the
introduction of technology which could go wrong may serve to heighten
this stress:
Evaluation of alternative exam presentation methods for pupils with low vision
51
“I hate using computers for exams! It adds to the stress
level ... when you have to prepare something under stress
it’s not very good.” Participant 7.
“I cannot begin to tell you the stress involved in the exam
centre. If you put the disc in, and the student is waiting, and
it comes up and the diagrams haven't been enlarged, the
page breaks all over the place. [..] What if it goes wrong,
what if it doesn't work, what if you... it's horrendous!”
Participant 4.
Some nervousness about being at the mercy of equipment
laptops not working, networks going down and that worries
her a little bit. Do you need a backup plan of having a hard
copy as well? Participant 3.
A number of participants who had extensive experience working within a
resource base felt that it may be more of a challenge in mainstream
schools without resource bases. Nevertheless, others argued that with
adequate preparation and practice these problems could be overcome:
Sees no problem with the use of the software one hour
before the exam. It could be done by the subject teacher or
a teaching assistant from the service - felt that probably best
managed by the service. The key thing is being aware of
the process and having training in the use of the software.
Important to have a practice CD. Participant 3
User interface is quite straightforward. You will need to
have had a chance to practice with it beforehand.
Participant 8.
Participant 3 described additional policies the service had in terms of
ensuring exam provision is arranged: they write the examination
concessions into annual reports; they have a form which is sent to school
(SENCO) at key points during the year (which details the recommended
options for exam access); they have a course/workshop at the beginning
of each year for professionals working in mainstream schools with
visually impaired pupils. This participant predicted that discussion about
the use of such software could easily be fitted into this process.
Only one participant reported negative views about the potential of using
the software. In part this was related to a belief that the practical reality of
Evaluation of alternative exam presentation methods for pupils with low vision
52
using the software tool would prove to be very difficult, and was possibly
linked to the participant’s reportedly negative experience of the exams
modification procedure in the past. The participant also expressed a view
that it was the exam boards’ responsibility to ensure that the
modifications were made and that it was unfair that they should pass on
this responsibility to individual schools and professionals. The position is
succinctly summed up through the following quotation:
“If [using this software] is as simple as pressing buttons
here, then it's simple for somebody to press buttons
somewhere else and send it in the post. If it's not as simple
as just pressing buttons, then it's going to cause chaos in
schools.” Participant 4.
4. Tensions and contrasts: Classroom practice, examination
options and the ‘real world’
A fourth theme which emerged from the interviews was an apparent
contrast between normal classroom practice in relation to curriculum
access and formats which are available in examinations. This theme can
be illustrated by the following quotes which suggest that some students
with visual impairments have their examinations presented in a format
that does not mirror their classroom practice:
“It would be helpful if students could use software [in
examinations] that they are familiar with in their everyday
lessons.” Participant 6.
“The problem with the exam boards is they are doing us no
favours if they produce material in A3 [..] it goes against
what we as QTVIs are trying to encourage in schools.”
Participant 3.
“It's pointless producing beautifully adapted papers and
questions if when it comes to exams they've still got one of
the layouts that’s given. [..] I don't think it allows them to
demonstrate their potential. All their schoolwork, all the text
in school, generally through their years is presented to them
in the same format and then to end up with the exam which
is different…” Participant 2.
Cobb (2008) identified a similar phenomenon in his research noting
‘modified large print papers produced by the current system do not
Evaluation of alternative exam presentation methods for pupils with low vision
53
match the classroom experience of many pupils in mainstream schools’
(p.ii). The general point was also extended further in the interviews with
some participants contrasting the educational experiences of visually
impaired students with the challenges of information access they may
face beyond school in the ‘real world’:
“When they leave school they’ve got to try and cope with the
real world and find strategies of dealing with it – magnifiers,
CCTV cameras or whatever. They need to deal with all
sorts of print styles.” Participant 7.
“What we tend to do is get [student who prefers 40 point
print] to achieve a balance because we know that when she
goes out into [pause] the real world, not everything is going
to be modified. So I suppose you’re doing her no favours by
saying here’s everything modified. So you’ve got to use low
vision aids and CCTVs as well.” Participant 3.
Given the tensions reported above, we consider briefly how
professionals and services rationalise these in practice. For some there
seemed to be an acceptance that examinations are ‘exceptions’ or
‘peculiarities’ that must be worked around, i.e. professionals must work
within the system as best they can and, crucially, help students practise.
As illustrated by the following quotes, creating practice exams and
getting hold of practice exams serves as an important strategy:
“Boards are particularly reluctant to let go of practice papers
– which is exactly what VI candidates need. The format is
more a problem to them rather than the content. They need
more practice not less but is much harder to get practice
papers for VI rather than sighted candidates.” Participants
10 and 11.
“They would have them for practice papers so they would
be familiar with it.” Participant 1.
“The students we have they all have very different
requirements. [..] We tailor make... When we do mock
exams we prepare them how the exams are going to come
in which is not what they want but it's what they have to get
used to using...” Participant 4.
Evaluation of alternative exam presentation methods for pupils with low vision
54
Perhaps the most strategic approach was adopted by participant 7 who
described providing curriculum material in either Arial 12, 18 or 24 point.
The choices available were directly linked to the examination
procedures. Assessments of students are carried out to establish which
presentation appeared to give the optimum accuracy / speed / working
distance balance. Students who were not able to access 24 point print
were encouraged to use large print combined with a low vision aid, or
alternatively Braille may be considered as the route to literacy.
Evaluation of alternative exam presentation methods for pupils with low vision
55
5. Access to the curriculum: student skills and service policies
The previous discussion (in relation to theme 4) leads into an analysis of
participants’ views of general policy on curriculum access and how this
relates to access to examinations. We consider first policies and then
discuss the related topic of student access skills (in particular the use of
LVAs and computers).
Policies
Participants discussed their broad approaches to curriculum access for
students with low vision. Sometimes these appeared to reflect service
policies explicitly, while in some cases participants focused exclusively
on their own practice. Views can be usefully categorised as either ‘a
student-centred approach to curriculum material preparation’ or ‘an
independence-skills approach to curriculum access’. The former was
characterised by curriculum material being prepared specifically for
individual young people. For example:
The service has no general policy, decisions are based
‘individual needs’ basis. Participant 6.
“The policy is, ‘if they need it in an enlarged style then they
get it.’ ” Participant 4
Range of sizes prepared to students: standard 14 point - 48
point bold. Majority have around N24. Also described a
student who had material presented to them in 24 but with
1.5 line spacing - i.e. very highly specifically designed for
each student. Participant 2
For many, the quality of the bespoke materials provided for the students
was a matter of professional pride (and to some extent a key function of
their service):
“The [support teachers] are fantastic and they wouldn't let
the students [have standard 12 point]. I can quite
confidently say, hand on heart, that a student would not
have to access this. [referring the 12 point text]” Participant
2.
“The students we have all have very different requirements.
We produce for them to learn, week in week out, what they
need. Which can be anything from size 14 up to 34... They
Evaluation of alternative exam presentation methods for pupils with low vision
56
might want it on white paper or yellow paper, bold or not
bold. Some children find one type of font easier to read
than another. We tailor make.” Participant 4.
A smaller number of participants discussed broader policy decisions
regarding access, that reflected perhaps a more pragmatic (and arguably
a less student-centred) approach:
“If they can't use 18 to 24 we think of other things - CCTV or
magnifiers.” Participant 7.
“At present we say you can have 18 or 24 point that’s it..
you’re going to have a reader if you want something
different [..] It would be reasonable to set boundaries – I
think RNIB has supported the idea that is 18 or 24. We have
to accept this is not a perfect world as we cannot tailor
make exam paper to each individual student.” Participant
10 and 11.
Participant 7 reflected that the policy of providing material in options
which matched those available in examinations was as much about
pragmatism as it was about philosophy. She noted that if the RNIB Eformatting software was available then students could have a broader
choice of curriculum materials (in particular the range of font styles,
noting that she thought Arial font could be problematic):
“If this [RNIB E-formatting software] was available we
wouldn’t need to be so fussy and we could let them use the
font I wanted.” Participant 7.
From the descriptions provided by participants there is evidence that a
significant amount of professional time is currently devoted to the
preparation of curriculum materials:
Huge amounts of team time resource is given up to
preparing materials – “[some staff] seem to spend 99% of
the time doing it.” Participant 2.
“Generally [the MLP paper] is very similar to what I produce
for [student]… hours and hours of work for one person.”
Participant 5.
Evaluation of alternative exam presentation methods for pupils with low vision
57
Student access
For those participants adopting ‘independence-skills’ approaches to
curriculum access, a key strategy is to encourage students to use low
vision aids (LVAs) and computers as a means of accessing print.
Although the use of LVAs was raised in many interviews, they were not
central to the discussions or the interview schedule. Most reference to
LVAs was in passing, although on a number of occasions participants
described how they used them in relatively negative terms. In this regard
LVAs were often characterised as a ‘second best’ in comparison to
specifically prepared printed material. This was argued from the point of
view of the professionals and the students:
“But LVA's and CCTV's are visually more tiring which is why
we often go to be modified large print.” Participant 3.
They don't have a child who uses a magnifier all the time though they have had some who use a magnifier with
normal print, or with large print. Participant 7 also tests
reading speed when using a magnifier and CCTV.
Interestingly although speeds matched (N18 print with
magnifier versus very large print) the large print version still
chosen by the child. Participant 7.
“Anything like this we would move heaven and earth to
make sure it was a format they could read on an A4 piece of
paper. So they can be in their class with paper the same as
everybody else. [..] In other places in the county, where it is
possibly not so easy as they haven't got the backup the
same as they have here, children do choose their vision
aids more than they do here.” Participant 4.
In line with the findings presented earlier report, some participants (e.g.
3, 7, 4, and 2) highlighted that many young people did not want to use
LVAs because it made them look and feel different, e.g.:
“But when you get to this age you have also got to rely on
young people using their low vision aids because it can be a
bit of a problem.” Participant 3.
The interviews discuss the role of computers (and in particular onscreen
access for exams) much more explicitly. The interview schedule asked
participants directly about the use of computers and laptops in the
classroom and discussion around onscreen access to examinations
Evaluation of alternative exam presentation methods for pupils with low vision
58
made the assumption students access to information using computers.
Unsurprisingly then, the use of computers as a method of curriculum
(and exam) access was discussed in more detail. Most participants
reported that laptops were relatively common in classroom practice:
“Students are very used to using laptops in their lessons
and so would be familiar with them by GSCE time. Threequarters of students used computers in lessons on a daily
basis.” The [service] would provide laptops if the student
was statemented, but if not, the supply of equipment such
as laptops is the responsibility of the school. Participant 6
A lot of students use laptops all the time in classrooms. At
the resource base it’s “pretty common”: all 8 have access to
a laptop and 3/8 use a laptop all the time, the remaining five
use laptops often. [..] Many use laptops anyway in the
exam for word processing. Participant 7.
It’s becoming more common. “A lot of our large print users
have their own laptop in school.” If students are using laptop
frequently they will have curriculum material in e-format
“This is happening… It’s the way we are pushing for this to
go.” Participant 1.
In terms of barriers to greater use of computers, participant 4 felt that
many students chose not to use laptops because they did not want to
stand out as different in the classroom (i.e. similar to the argument made
in relation to LVAs). Further, as illustrated by the following quote, a
number of participants argued that it was important to consider the
training needs of classroom teachers:
“Training of teachers is an issue. The mindset of teachers...
we are trying to get the teachers to take responsibility for
producing materials in appropriate font size or to provide eresource for student to modify themselves.” Participant 1.
Further, as reported by participants 5, developing the skills of students
themselves is needed, particularly if computers are to be used in exams:
[Specific child] currently uses Arial 30 point. She is reliant
on laptop for writing. [..] She uses a computer regularly but
would need to practice accessing in this format. It would be
Evaluation of alternative exam presentation methods for pupils with low vision
59
beneficial as it would cut down on all the modification.
Participant 5
Nevertheless, in the interviews participants made links between the use
of computers in the classroom and the potential of using them in an
examination:
“We have secondary-age students using laptops, a couple
of them for 100% of the time. For them to be able to have
an exam paper on a computer and to be able to answer it
on a computer…” Participant 2.
“Could be very powerful if [that way of working] was built in
to all their teaching materials not just for their exam at the
end.” Participant 8.
6. The potential of onscreen examination presentation
All participants were shown on screen versions of an original
examination paper and its MLP equivalent on screen. This involved
presenting the two PDF files on screen, using Adobe Acrobat reader.
Presenting the files in this way provided very flexibility in the presentation
options – the researcher was able to zoom in and out (enlarging the
presented font) and scroll up and down. Importantly, it was emphasised
that this method of presentation was being used to demonstrate the
principle of onscreen examinations and to encourage reflection.
Participants were also shown the RNIB E-formatting software to
demonstrate an alternative version of onscreen presentation.
In line with the interview schedule, the discussion around the potential of
onscreen presentation of exams focused upon:
 an initial general reaction to the idea,
 how students used computers in the day-to-day classroom (reported
earlier), and
 potential advantages and disadvantages of using computers in
examinations in this way.
Many interviews generated broadly positive responses to the notion of
onscreen presentation of examinations. While there were caveats and
caution about the implementation of such an idea, many welcomed the
general principle. For example, the following two quotes capture broad
Evaluation of alternative exam presentation methods for pupils with low vision
60
positive views about student familiarity with computers and the idea of
‘mainstreaming’ assessment (‘everyone uses computers’):
“Brilliant. Absolutely brilliant. Yep, I think it's the way to go.
[..] We have secondary-age students using laptops, a
couple of them for 100% of the time. For them to be able to
have an exam paper on a computer and to be able to
answer it on a computer…” Participant 2.
“It’s a ‘move forward’ for visually impaired [people] –
everyone uses computers. [..] That’s what [student] will
have to do … whatever job he does in the future, it will all be
computer-based.” Participant 5.
Caution was often constructive, particularly focussing upon training, the
importance of developing students’ access technology skills, and the
development of technology that works:
“For some students I can see that an [onscreen] version
would be an option. But you need to train students.”
Participant 4.
“Touch typing becomes far more important.” Participant 3.
“It does depend on how good the technology is…the
answering onscreen is great if the technology works.”
Participant 10 and 11.
Technical development that would enable students to answer on screen
was highlighted by most participants, and to this extent the potential of
onscreen technology depended upon interactive and accessible
software. As illustrated by the following quote, a number reported that
this offered hidden advantages of providing students access to ‘metalevel’ information which would help students navigate (e.g. move from
question to question, answer boxes could be linked to questions and
available marks).
If it was based upon form-filling this has a lot of possibilities
e.g. the use of Tab, space for checkboxes. There also may
be other hidden advantages if this was done well – e.g.
using navigation tools to step from question to question. If
they could answer on the screen the students “would take it
in their stride.” Participant 7.
Evaluation of alternative exam presentation methods for pupils with low vision
61
Table 20 presents a summary of advantages and disadvantages
identified by participants. Many of the disadvantages identified are
reflected in the caution described above. To this extent these cautionary
points are a reminder of theme 1: ‘the devil is in the detail’. Many of the
potential advantages of onscreen presentations of examination have little
meaning if care is not taken in preparing the material appropriately, i.e.
the process of modification.
Table 20. Onscreen examinations – potential advantages and
disadvantages. Answers grouped by authors
Advantages (including positives)
Control of the font.
Control of the size of font.
Control of the colour combinations.
Familiarity, e.g. links to classroom practice; use of existing access
technology; students can create a format which they use to.
Independence.
Quicker preparation time for the teacher.
Assessment reflects the ‘real world’ in which the student will
eventually have to operate.
Less stress and anxiety.
This is the likely direction of future developments for all students
(e.g. moving towards the ‘paperless school’).
Disadvantages (including concerns and challenges)
Navigation, e.g. scrolling, moving from question to question.
Unfamiliarity, e.g. with unusual specific examination software,
unusual students who are not familiar with computers.
Training would be required (students and staff).
Non-text will need consideration, e.g. diagrams.
Specific topic requirements, e.g. specialist characters, drawings
and graphs.
Technical breakdown and lost work.
Anxiety about using computers (particularly staff).
Visual fatigue.
7. Contrasting views of a ‘perfect world’
Evaluation of alternative exam presentation methods for pupils with low vision
62
At the end of the interview participants were asked “what type of
examination provision for visually impaired pupils would you like to see in
the future?” Importantly, the interviewer predicated the question by
reviewing the different formats that being discussed (MLP, RNIB Eformatting software and onscreen presentation). It is important to
recognise that the interviews were not an attempt to ‘survey’ opinion, but
rather a conversation in which ideas were generated and opinions and
views were formed. Therefore, participants’ responses to this question
captured their emerging views at the end of the interview drawing on the
other issues that had been discussed.
Table 21 presents a summary of how different interviews concluded what
would be their ‘perfect world’. For each we have provided quotes to
illustrate some of the key points that were made and attempt to
categorise these into different ‘ideals’:
 ‘Existing system (done well)’ in which print copies of examinations are
delivered to the exam centre.
 ‘Formatting tool’ in which a bespoke printed version could be
produced in the exam centre (similar to the RNIB E-formatting
software).
 ‘E-version’ in which an electronic version of the examination is
presented.
This is, of course, a simplified categorisation. Nevertheless, it offers
some notion of the emerging opinions of the participants and their
general views of how the examination process could potentially function
in the future. Most participants considered that an improved examination
system (a ‘perfect world’) could incorporate a new way of working which
involves the use of either formatting tools (such as a developed version
of the RNIB E-formatting software) or computer-based examinations. No
participants dismissed either idea.
Evaluation of alternative exam presentation methods for pupils with low vision
63
Table 21. ‘In a perfect world’: Views about the future of examination
modification which emerged from nine interviews with
professionals.
Interview Key points and example quote(s)
‘Ideal
category’
A
Access to e-version of examination which
E-version
can be provided to the student on a computer
in the format they require.
“The easiest way for student to
do the exam. [..] For me, I go
with the laptop option [..] I would
be using the technology.”
B
E-versions of examinations.
E-version
“I think electronically presented
and electronically completed. [..]
[Why not say], ‘right, you
manipulate into the format that
you want rather than I give you
something that I think you
should use.’ ”
C
Having a tool to prepare examination in the
students preferred format.
Formatting
tool
“You would like to order your
exam paper from the board and
say ‘right, I want point 24 in
Verdana, bold, double spaced
by next Friday!’ [Laughs] [But
using something like the RNIB
E-formatting], to be able to do it
for yourself would be great. That
is pretty near the perfect world.”
Evaluation of alternative exam presentation methods for pupils with low vision
64
Interview
Key points and example quote(s)
D
The current system improved and working
well. Examinations should b delivered in the
appropriate format.
“I would like an exam paper to
come in prepared in a tailormade way for an individual child.
[..] That would be my starting
point to the students I work with
here. The paper, hardcopy,
suitably prepared in terms of
size, colour, all rest of it. And I
would like it to be properly
quality controlled so we are
confident it.”
‘Ideal
category’
Existing
system done
well.
Possibility of
E-version
Some students may prefer an e-version of
examination.
“For some students I can see
that an [onscreen] version would
be an option. But you need to
train students.”
E
Having a tool to prepare examination in the
students preferred format.
Formatting
tool
“Papers to be supplied in eformat (on disc/memory stick) –
so if [we] need to adapt them we
don’t have to retype them. [..] If
we were supplied with modified
paper but had RNIB software so
we could enlarge it again and
have the ability to print it off.”
F
Greater clarity, consistency and quality in the
examination modifications procedure and
more flexibility about breaks and opening
times.
Existing
system done
well.
Unclear.
Evaluation of alternative exam presentation methods for pupils with low vision
65
More variety of examination layouts:
“Students would have more
choice of format, layout and
paper size.”
Evaluation of alternative exam presentation methods for pupils with low vision
66
Interview
G
‘Ideal
category’
Personal preference for having a tool to
Formatting
prepare examination in the students
tool and Epreferred printed format, but a realisation that version
E-versions of examinations is probably what
the students will want.
Key points and example quote(s)
“Having printed bespoke paper
would probably be my preferred
option. But I doubt if the
children would feel like that...
they seem to rely so much more
on technology. [..] In a perfect
world and have all the children
learning typing skills when they
were little instead of finger
pecking all the way through.”
H
The ideal world would be ‘modify on an
individual basis’. RNIB software takes you a
long way down that route (with additional
features). Accessible e-versions of exams
has possibilities but there are many
unknowns.
Formatting
tool and
possibility of
E-version
I
Having a tool to prepare examination in the
students preferred printed format and
onscreen.
Formatting
tool and Eversion
Still a role for paper copy – in a
perfect world would have pack
which has an e-version which
they could use on the computer
but also includes a paper
version.
Evaluation of alternative exam presentation methods for pupils with low vision
67
5. Group Interviews: Students who are
non-print readers
5.1 Introduction
Although the main remit of the project concerns exam access for
students with low vision who use print as their primary literacy medium,
an opportunity was provided to interview Years 10 and 11 students who
were not print readers (i.e. they use Braille or technology to access
literacy). This is valuable because it explores the possibility of a single
accessible examination solution which could include both print and nonprint readers. Two group interviews discussed: participants’ experiences
of taking exams to date and the preparation involved; the potential use of
screen reading software in examinations; and a reflection upon what the
perfect examination scenario might be.
5.2 Method
Participants
Seven participants took part in the semi-structured group interview. All
participants were taking part in a study weekend at Worcester New
College during their Easter holiday in 2010. All were blind and did not
read print, either accessing literacy through Braille or technology (or in
most cases both), and all described receiving 100% extra time
allowances in examinations. A breakdown of the participants is
presented in Table 22.
Evaluation of alternative exam presentation methods for pupils with low vision
68
Table 22. Description of the participants taking part in the interview
(N=7)
Partic- School type
Primary literacy
Use of ICT
ipant
medium
A
Mainstream
Does not use Braille
Mainly computer
school
or print. Main access with screenreader*
to literacy is through a
laptop with
screenreader.
B
Mainstream
Grade 2 Braille
Braille Notetaker
school with
and Perkins Brailler
resource base
C
Mainstream
Grade 2 Braille
Mainly through
school with
computer with
resource base
screenreader
D
E
F
G
Mainstream
school with
resource base
Mainstream
school with vi
room
Grade 2 Braille
Mainly computer
with screenreader
Grade 2 Braille
Mainstream
school with
enhanced
resource
provision but
not a resource
base
Mainstream
with resource
base
Grade 2 Braille
Braille for Maths
and Science.
Computer with
screenreader for
other subjects
Uses laptop with
Jaws for most
lessons. Music use
‘Braille notetaker’
Grade 2 Braille
Braillenote for all
subjects except for
Maths and Science
(Perkins Brailler
used).
Does not use
laptop in class
* Note: a ‘screenreader’ is access technology that attempts to re-present
information presented on the computer screen in an alternative format
(usually speech).
Evaluation of alternative exam presentation methods for pupils with low vision
69
As shown, six of the participants were Braille readers and were
expecting to be using Braille-based examination papers in their
forthcoming GCSE examinations. One participant (A) did not read Braille
or print and used technology for accessing examinations (i.e. mainly a
laptop with a screen reader). All participants attended mainstream
schools, a number of which were described as having enhanced
resource provision.
Materials and procedure
The seven participants took part in the semi-structured interview in two
separate sessions (five in session 1; two in session 2). The interviews
were recorded and a transcript made to aid analysis. Given the small
numbers of participants in each group the results from both sessions are
discussed together with individual quotes provided to support the key
themes raised in the interviews.
The interview schedule followed the following structure:
1. Introductions
2. Sharing experiences of taking exams to date
3. Preparation of exams
4. Use of a screenreader to access exams (pros and cons)
5. Reflections on screenreading activity
6. Perfect exam scenario.
The screenreading activity involved participants working through
examination style questions presented on the computer in Microsoft
Word.
Each interview took approximately one hour. At the beginning of the
session the broad purposes of the research was explained. Participants
were then asked to sign a consent form declaring that they were willing
to take part in the research, but it was noted that they could withdraw at
any time. Participants were each given a £10 gift voucher.
Evaluation of alternative exam presentation methods for pupils with low vision
70
5.3 Results
Sharing experiences of taking exams to date
Participants were asked to share their experiences of taking public
exams to date, and provide brief information about:
 how exams were accessed,
 the nature of additional support provided,
 how the process has worked from their point of view: evaluative
comments requested
A summary of their responses is presented in Table 23.
Table 23. Participants experiences of taking exams
Part.
A
B
How exams are
Additional Support
accessed
For year 6 SATS paper
Reader (individual
participant typed answers subjects not specified)
using a computer.
Uncertain how exams will
be accessed in the future.
Other comments
Braille with a reader for
certain subjects.
Reader for Maths (layout
of diagrams) and English
(to help with referencing
back to text)
Reported that felt
well supported.
Reported that felt
generally well
supported. Particular
problems had arisen
in accessing bar
charts in a past
GCSE Maths paper.
Reported that was
generally well
supported.
Perkins Brailler used for
Maths and Science.
Braille notetaker used for
RE and English
C
Braille with a reader for
Maths and Science
Reader for Maths (reads
the equations) and
Science (if running out of
time)
D
Braille with a reader for
Maths and Science
Reader for Science and
Maths. Worked through
Maths diagrams before
the exam starts.
E
Braille with a reader for
Maths and Science.
Reader for Maths and
Science (to talk through
diagrams). A laptop is
Reported that the
arrangements have
worked well to date
and feels ‘well
supported’
Reported that was
well supported.
Evaluation of alternative exam presentation methods for pupils with low vision
71
used for Science. For
multiple choice questions
in science a scribe
completes the form.
Maths is recorded on a
Perkins Brailler.
F
G
Braille with a reader for
Maths.
Braille with a reader for
Maths
English - separate hard
copy booklets are
produced in Braille
(questions in one booklet
and text in a second
booklet). Answers are
written on either a laptop
or Braillenote
Reported that was
well supported
French- listens to tape
and writes answers using
a laptop. Can ask ‘reader’
to pause the tape.
Types answers on a
No evaluative
Perkins. Diagrams are in comments provided
a separate booklet.
English – all textbooks in
Braille (hard copy).
Writes answers on Braille
notetaker.
As shown in Table 5.2 all participants reported that they had access to a
reader for at least one subject, the most common being Maths and
Science. Whereas participant A was not able to specify which exams a
reader was available for, other participants were very precise about in
which particular parts of the exam support was needed. As an example,
participants B and E reported that specific support was required in Maths
to help in understanding the layout of diagrams. Similarly Participant C
reported that support was required to read equations and tables noting
that in a past GCSE Maths paper she had experienced problems in
accessing bar charts. One participant (B) reported that particular support
was required with ‘referencing back to the text’ in English and Welsh
exams. In elaborating this point she explained:
‘In English you get a passage and sometimes with the
questions I get confused as you have to refer back to the
paragraphs… with Welsh exam you get a long passage with
just a few questions.’ (Participant B)
Evaluation of alternative exam presentation methods for pupils with low vision
72
A similar point was made by participant G in relation to the use of English
exams in Braille. She reported that she found it ‘difficult to flick from text
to question book’ and during her last English literature exam ran out of
time:
‘I knew what I wanted to do but didn’t have time to finish the
exam’. (Participant G)
As an alternative to a reader she wondered instead whether a Braillenote
could be used to allow her to ‘speed read’ through the text.
A number of participants described particular strategies they used with
the reader to access exams. As an example participant C reported that
she would only need the services of a reader for Science if she was
running out of time. A similar point was made by participant G, who
noted that in Maths she begins independently with the reader only
helping her if she is behind schedule:
‘I try to read as much as possible – sometimes if I’m behind
they will start reading it to me’. (Participant G)
One participant spoke of particular ‘advance’ strategies she used to
prepare for interpreting the diagrams in Science and Maths exams:
‘I usually go through the diagrams before the exams start….
my exam board does not modify diagrams. Say there’s a
motor in my physics exam they put it in 3-D rather than 2-D’.
(Participant D).
One participant reported that when she had a test ‘out of the blue’ her
reader would write down the answer for her and for multiple choice
questions in science a ‘scribe’ will complete the form. In comparison
another participant was keen to emphasise her independence in exams
reporting that:
‘I have never required anybody to write down answers’.
(Participant E).
The majority of participants reported that they took their exams in a
separate room from their peer group, usually the resource base.
Evaluation of alternative exam presentation methods for pupils with low vision
73
Preparation of exams
Participants were asked to provide information on who would be involved
in the preparation of exams within their schools. Although none were
directly involved in the process, a number of participants had some
knowledge of the various stages involved in obtaining exam papers in an
appropriate format including the role of the School exams officer. This is
illustrated through the following quotes:
‘If it is a GCSE paper the school orders the paper through
the exam officer. If it is a past paper my teaching / support
assistance orders them’. (Participant D)
‘For internal exams it is up to class teachers to give in paper
to the LSA to adapt in time. For external exams I tell my
assistant what exams I have coming up who then liaises
with the school exams officer who orders from the board’.
(Participant C)
‘My helpers send off paper to exam board. If it is a mock
exam it will be produced in the school…and will be laid out
in the same format as modified papers’. (Participant F)
Other participants spoke in more general terms about the process and
either made no reference to the exams officer or did not know about this
role:
‘The teachers get the paper and give it to the support
worker who embosses it into Braille and make diagrams
themselves. I am not involved at all in this process and am
not aware of an exams officer in the school’. (Participant E)
‘The school orders the Braille paper through the exam
board. My support worker opens the paper on the day but
cannot view them in advance’ For internal exams the
support worker ‘embosses them off for me’. (Participant G)
Other participants were less sure about the procedures involved:
‘The exam preparation is done in advance and generally
works well - but I am not sure who is involved’.
(Participant B)
Evaluation of alternative exam presentation methods for pupils with low vision
74
‘I don’t know how it’s done in my school’. (Participant A)
Participants were also asked to provide information about who they
would have a conversation with in their school about access to an exam
paper (eg whether it should be accessed through Braille or a
screenreader). One participant highlighted the role of the QTVI in this
process:
‘My QTVI would be important…. we’ve done mocks and
then made a decision about the real exam. I did a mock
maths and didn’t finish it so they said that for the maths
exams we are going to read to you’. (Participant E)
The majority of participants reported that either they did not have a
conversation with their teachers at all about access or they did not
consider there to be need to have such conversations with their
teachers. This latter point is illustrated by the following quotes from two
participants who reported that as Braille was their medium of instruction
this would be carried through to the exams:
‘They know me by now so I just receive it in Braille’.
(Participant F)
‘I would only use Braille as I’m not very good at using
screenreaders’. (Participant C)
Although access was not discussed, two of the participants emphasised
the opportunities they were provided with to practise papers and ensure
familiarity with the format of exams. This was considered to be a
particular issue with understanding diagrams in Maths and Science as
illustrated by the following quotes:
‘It can sometimes be an issue in understanding diagrams –
or when asked to draw something onto German
film….you’re spending most of your time looking at the
diagrams all the time and you’re not getting the time to look
back over…not having the time to go back over the
questions you can do’. (Participant F)
‘Also they are getting quite strict in what you can and can’t
do… for a diagram they might ask you do something but say
you have to do it in a certain way. You can’t cut shapes out
to show other people. My helper cannot cut out the diagram
Evaluation of alternative exam presentation methods for pupils with low vision
75
for me to show me a 3-D version of a net. Instead of asking
me to draw sometimes I am asked to describe what I think it
looks like. Sometimes I get 3-D shapes but not often.’
(Participant G)
Use of a screenreader to access exams
Participants were asked to consider the potential advantages and
disadvantages of using a screenreader to access exams. A summary of
the key points is presented in Table 24.
Table 24. Summary of potential advantages and disadvantages of
accessing exam through a screen reader
Advantages
Independence – provides increased independence in comparison
with a reader
Concentration – may require less concentration than reading the
same text in Braille
Speed – could be quicker than reading the same text in Braille
Resources – Frees up support worker/reader to do other tasks
Navigation – Could be helpful for scrolling from line to line
Time – could save time if could type in answer directly to
document as it won’t have to flick from one hardcopy book to the
next
Reference – Can mark your place to make it easier to cut and
paste (e.g. if you wanted to insert a quote into your answer)
Familiarity – more similar to way of working if currently use
screenreaders
Disadvantages
Technical problems – e.g. could lose the file if in electronic format
Accuracy (semantics) – the screen reader might not read the
meaning of words accurately (example given of accessing a
foreign language through a screenreader)
Accuracy (technical) – the screenreader may not read technical
diagrams and graphs accurately
Time – could take more time to listen to the paper all the way
through. Issue raised about how one might use ‘skim listening’ in
the same way as ‘skim reading’ to scan the text. Some make
write Braille faster than they type.
Personalised – Screenreader would need to be set up for
individual requirements (speed, voice etc).
Evaluation of alternative exam presentation methods for pupils with low vision
76
Evaluation of alternative exam presentation methods for pupils with low vision
77
Reflections on screenreading activity
Participants were then invited to complete an onscreen exam accessed
through a screenreader (Dolphin Supernova). The material was based
upon the question styles developed for the student performance tasks (in
a previous section). The material were not designed in great detail,
rather an attempt was made to make the questions accessible to a
screenreader (e.g. removal of pictures) and to use answering styles
suitable for that an electronic format (e.g. for multiple choice questions,
replacing ‘circle the correct answer’ with ‘type the letter of the correct
statement here’). The activity was purposefully informal because of the
exploratory nature of the work, and some of the participants were used to
using other screenreading packages. Following this task, participations
were asked to talk through their experiences. A summary of the key
points raised in the discussion is presented below.
Orientation to technology
Two participants emphasised that it would be important to ensure that
the technology is working prior to the exam starting. It was suggested by
one participant that candidates are provided with a period of ‘orientation
time’ before the exam to ensure the technology is in good working order
(10 minutes was proposed by the participant as a minimum). A similar
point was made by one participant who reported that it would be helpful
to provide time in advance to become familiar with the speed and voice
and alter this if necessary to suit personal preferences:
‘You need orientation time prior to the exam starting – to set
up speech programme. You also need to be familiar with the
particular type of software’. (Participant B)
Accuracy (technical)
All participants reported that tables were found to be the main limitations
of using a screenreader to access the exam. This is illustrated by the
following quotes:
‘The tables really confused me. I found it difficult to make
sense of. I managed to answer question but it was difficult –
I had to keep going up and down to find information’.
(Participant F)
Evaluation of alternative exam presentation methods for pupils with low vision
78
‘I didn’t do anything on the tables….it just babbled at me’.
(Participant B)
‘I liked it. It is less tiring and quicker than Braille – except
for the table. It didn’t give all the information you require re.
percentages’. (Participant C)
Three participants reported that they would like to have referred to hard
copy Braille versions of all tables/diagrams if they were to take an online
public exam.
‘I prefer to have a hard copy in front of me with tables /
diagrams’. (Participant B)
Different possibilities were discussed including the option of having the
exam paper presented as a Braille display on a laptop or Braillenote (and
whether a hard copy of the tables in Braille would be required if available
on Braillenote).
Personalised settings
Four participants reported that the settings of the screenreader provided
them with too much information (which could have been modified with
different settings). As examples it was reported by one participant that
she did not require information about commas and full stops but that it
was useful to know about brackets around the mark indicators at the end
of questions (e.g. [10]). Similarly, another participant reported that she
was ‘confused’ by brackets around certain types of information, and she
would have welcomed modification so she did not receive all the
punctuation:
‘I have [screen reader setup] so it doesn’t read any
punctuation – this becomes really annoying’. (Participant C)
Confidence with technology and Training in appropriate skills
Two participants reported that candidates could be at a disadvantage as
they did not feel confident in using screenreaders. As with discussions in
relation to low vision readers requiring training in the use of access tools
(computers and LVAs), these participants were anxious about the access
technology skills they would require if examinations were presented
electronically.
Evaluation of alternative exam presentation methods for pupils with low vision
79
One participant reported that different types of skills are required for
using a screenreader in comparison with other ICT tasks (e.g. web
browsing). This was supported by another participant who reported that
she was used to using Braille so may find it difficult to adjust to an
onscreen version for exams.
Subject specific use
Two participants reported that screenreaders may be more helpful in
some subjects than others in accessing exams.
‘It depends on which subject. For maths/science I would use
a Perkins. For English I would prefer everything in Braille’.
(Participant C)
In comparison, another participant reported that this type of approach
could be helpful in English based subjects where there is a lot of text to
scroll through, but queried whether a screenreader would be helpful (or
allowed) for a comprehension test in English language.
Perfect exam scenario
As a final question, participants were asked to reflect on what they would
consider to be a perfect exam scenario for them. A key theme raised
was in relation to the range of access technology students wanted. It
was staggering the range of combinations that the students said they
would prefer in a ‘perfect scenario’: different methods of reading the
examination (screen reader, hardcopy Braille, refreshable Braille), and
different methods of writing answers (computer, Braille), and all the
combinations of these.
‘I would rather do it written than spoken – I can get across
what I want to say better.’ (Participant C)
‘I would read in Braille and record using a Braille notetaker’.
(Participant F)
‘Would record on Braillenote and read as a hardcopy.
Technology - no role to play in this scenario’. (Participant E)
‘When I use the computer I have speech on and my Braille
display on as well – so I can listen back to what I have
Evaluation of alternative exam presentation methods for pupils with low vision
80
written – if I’ve got spelling mistakes I can correct it.’
(Participant G)
‘Being read to and I write it down – worked well so far’.
(Participant A)
‘Refreshable Braille as an option? [But it’s] really
temperamental - don’t want to depend on it’. (Participant C)
‘Available on Braillenote – I’m faster at Braille than typing.
Papers to be available on Braillenote – and then could
answer on the Braillenote’ (Participant F)
‘Screenreader on a word document where you can input
text directly onto document’ (Participant C)
‘If I was using screenreader would still like to write answers
in Braille’ (Participant E)
Unsurprisingly given the range of materials participants anticipated using
(including speech software), participants discussed the environment in
which they would want to take the examination. They particularly
highlighted that they needed a quiet room, probably with no other
candidates present:
‘Would prefer to be in a place where there is no background
noise’ (Participant D)
‘I like what I do now. In my VI room which is isolated and
quiet. Can be distracting if support worker is Brailling while
I’m doing my exams. I’d like a reader with me if I need them
– just in case’. (Participant E)
‘On my own – I can think better.’ (Participant F)
‘Environment would include a lot of desk space’ (Participant
G)
Evaluation of alternative exam presentation methods for pupils with low vision
81
5.4 Discussion
The interviews with these students who do not read print proved very
interesting. For all except one participant, Braille was a key route to
literacy. Nevertheless, all recognised the role that screenreading /
computer technology played in their life: all used a screenreader and all
appeared to anticipate using this technology in some way in their
examinations. Perhaps as a result of this, all engaged with interest at
the possibility of screenreading technology having a greater role in
examinations. Some key issues raised in the interviews were:
 Some topic/curriculum areas appear to lend themselves to being
‘screen read’ more than others.
 Particular presentations are problematic for some people when
accessed electronically, e.g. tables were identified by many
participants. It may be that training and practice would help with this,
or that hardcopy Braille in a separate booklet would be a preferable
alternative or addition for some. Refreshable Braille was also
highlighted as an option by some.
 Familiarity with technology was identified as important – plus the
requirement for additional support for certain types of questions (e.g.
those involving tables and/or figures). Training needs were identified
as important.
 This has implications for the people who support – skill in the use of
the technology was also felt to be required by teachers.
 Child voice would be important in process of making decisions – it
was clear that the participants had very different ideas to one another
about the combinations of technology that would suit them.
Selective use of screenreaders may be a way forward for exam access.
Participants identified many potential advantages from their point of view,
and relatively few (non-technical) disadvantages. In principle, some
exams which are available electronically for students with low vision
could also be suitable for students who do not access print, and vice
versa (particularly if answers can be given electronically too). However,
the very different modifications required for some examinations may
mean that this is often not possible.
Evaluation of alternative exam presentation methods for pupils with low vision
82
6. Technical analysis of the RNIB Eformatting software
The invitation to tender described the RNIB E-formatting software as
being designed to allow “the teacher/school to modify the exam paper in
terms of font size and style, text and background colour, and the line
width inside diagrams. Once it has been modified to meet the individual
requirements of the candidate, the paper can be printed for them to have
as a paper copy or it may be read on-screen.” In this section of the
report we review how the current version of the software operates. We
focus our discussion upon two stages of the software's use, as well as a
short discussion about file structure:
1)Modifying the examination paper.
2) Using the software to format the examination paper and quality of the
Output.
3) ‘Accessible’ files
In the proposal to carry out this work, the term ‘proof of concept’ was
used. This term was used because the team were commissioned to
establish whether the concept of the RNIB E-formatting software could,
in principle, usefully improve students’ access to examination papers. In
other sections of this report we report upon interviews with professionals
in which we explored the broad principles of using a tool of this type.
The interviews offered evidence that software of this kind would be
welcomed by many. In this and the next section we argue that the
development of similar software could be part of a broader strategy of
working towards computer-based examinations. How the software
should be developed is dependent upon these broad strategic directions.
6.1 Modifying the examination paper
The RNIB E-formatting software as used by a classroom teacher would
be presented on a CD-ROM. The teacher would run the software on the
CD-ROM, choose an XML ‘exam’ file, select key formatting features
(such as font style and size), and then generate the printable version of
the examination paper. Prior to the teacher receiving the software in this
format, a lot of preparation must go into the creation of the XML file.
Importantly, this preparation must incorporate any modifications which
are required, e.g. the removal or simplification of diagrams, the
Evaluation of alternative exam presentation methods for pupils with low vision
83
implementation of standard formats such as capitalisation, and the
modification of language. The guidelines for this modification process
are presented within the JCQ Best Practice Guidance (JCQ, 2008).
The modifications made in the equivalent MLP and Software examination
papers were different. It appears that the MLP version of the
examination paper was passed onto RNIB and then further changes
were made during the creation of the XML file. In some cases these
differences were quite marked (e.g. the exclusion of pictures). Table 25
presents a comparison of the modifications made for one example
examination paper (the same paper which was used in the professional
interviews). Analysis of other examination papers reveals a similar
difference.
Table 25. A summary of differences in modifications made in the
MLP and software versions of the same examination paper (OCR,
Twenty First Century Science, Biology A, A221/02)
Question
Modification
1(b)
Change of the position of the diagram key.
2
Removal of pictures and speech bubbles.
3(b), 5(e), 8 Matching task changed from connecting two boxes to
choosing a statement from each of the two lists.
5
Graph changed, e.g. different arrowheads, use of the key.
5(c)
Vertical list of options (compared with horizontal).
10, 10(c)
Slightly different schema diagram, e.g. different
arrowheads.
General
Positioning of tick boxes in multiple-choice questions (left
aligned in software, right aligned in MLP).
General
The use of CAPITALISATION (software) and
CAPITALISATION underlined (MLP).
General
The use of matrix in some tables.
General
Available marks following the question (software) and right
justified (MLP).
It is unclear why these different modifications were made. Some of the
changes related to diagrams would have been necessary because the
software required some vector diagrams in a particular format to enable
scaling. Changes involving the removal of diagrams may also be linked
to this. Other general changes (such as multiple-choice and
capitalisation) may have just been alternative interpretations of the JCQ
guidelines.
Evaluation of alternative exam presentation methods for pupils with low vision
84
A positive consequence of this process was that all of the exams
produced using the RNIB E-formatting software had a similar ‘look and
feel’. Part of the reason for this was that the RNIB software used to
produce the XML files did so in a standard way. Part of the reason also is
that a single team was involved in the process of modifying all the
examination papers.
6.2 Using the software to format the
examination paper and quality of the
output.
The software was demonstrated to participants relatively quickly.
Nevertheless they got an idea of the principle of the software and the
basic process and quality of the output (and this was the main focus of
the interviews with the professionals).
In terms of the process of using the software, those who did comment
expressed that it appeared to be relatively straightforward. Participants
were asked directly about whether they thought they would be able to
use the software in the context of preparing a student for an examination.
Many participants felt this was possible, while others were more hesitant
and were particularly nervous about working under pressure one hour
before the examination (within the early opening regulations). It is useful
to consider the task of producing a quality large print paper using the
software within the time limit of one hour. This period of one hour must
also include time taken for printing. Some long papers which are printed
in a large font may take a considerable time. Given this, it may be safer
to think that the task of opening the software and creating a file ready for
printing should take a maximum of 30 minutes.
The simple interface has the potential for this. However, based on trials
carried out by the first author, this is problematic (both in terms of time
taken and getting the material to look as required) because of the clumsy
process of inserting page breaks and modifying diagrams. The issue of
page breaks is perhaps best illustrated by considering the creation of the
materials for tasks used in the ‘case examples of students who prefer
very large hardcopy print’ (greater than 24 point). The material needed
for the participants taking part in those case examples included a variety
Evaluation of alternative exam presentation methods for pupils with low vision
85
of styles font in sizes 24, 28, 36 and 40 point. While the material was
relatively simple in structure (only three multiple choice questions), the
creation of the materials effectively gave an opportunity to experience
how easy the RNIB E-formatting software was to operate in extreme
situations. In practice, it proved very difficult to format the material
exactly as wanted and in the end it was prepared using Microsoft Word.
The key problem was controlling the page breaks as we wanted the
materials to have one question per page. At relatively low point sizes
this proved impossible for us using the RNIB E-formatting software.
Unsurprisingly, the direct interaction with the material using Microsoft
Word was easier because we could make use of subtle techniques such
as margin width changes and variable line spacing, as well as the line
break functions offered. As a result we were able to get the material to
look as required (even at large font sizes).
If these problems could be addressed then the issue of time would
probably be solved. However, providing an interface to control these
functions would be difficult within the current software model which does
not allow the user to directly interact with the document (the print ready
file is presented as ‘read only’ within Internet Explorer). In fact, the only
way of finding out where page breaks will appear is using the print
preview function within Internet Explorer.
The interface for modifying the line widths of the vector diagrams is
technically impressive and complex. However, it may be too difficult for
the software to automatically render diagrams with the precision that
users will require (the result being a large choice of possible diagrams,
but none of them being ‘quite right’). It may be more productive to create
a range of diagrams and simply give the user a choice. In practice, it
may be that users of the software print out diagrams and then make
additional modifications (including enlargements) to the hard copy. The
process of photocopy-enlarging part of MLP examinations appears to be
relatively common practice within the current procedure.
In terms of the quality of the output, participants were largely impressed
with the quality of the printed papers. Unsurprisingly, they particularly
liked the control over font style and size. Nevertheless some features
were identified as possible useful additions. Additional features raised in
the interviews were:
 Control over line spacing (3 participants)
 Control over page breaks (5 participants)
 Control over diagrams (size, positioning and fonts) (6 participants)
 Landscaping diagrams (1 participant)
Evaluation of alternative exam presentation methods for pupils with low vision
86
As already described, several other comments were made in relation to
modification issues (e.g. arrowheads on diagrams, quality of graphs).
One other point raised was related to the colours available for onscreen
presentation. The current choices are all saturated and bright colours
and it would be useful to have options for a greater range of colours,
particularly ones which are off-white shades for reduced glare.
Other points noted by the first author when using the software for the
creation of materials was that it would be useful to be able to control the
line format in tables.
6.3 ‘Accessible’ files
As already described, an XML file structure underpins the RNIB Eformatting software. It is some of the features of XML which enables the
user to be able to finely control the format of the output files. Also
embedded within the XML files created for the RNIB E-formatting
software is information about the structure of the examination paper.
This ‘meta-data’ is similar to a style sheet associated with a word
processor document (for example produced by Microsoft Word).
Therefore meta-data about headings, questions, sub-questions and no
doubt other structural information is (or could be) contained within the
XML file.
Although beyond the remit of this report and the technical knowledge of
the authors, this meta-data could be used further if the software were
developed. The development of interactive onscreen examinations
would require an interface which enables users to be able to navigate the
examination paper with ease (an issue raised in the interviews with
professionals and students). For example, the user might want to move
from question to question with keystrokes, to be able to clarify which
answer box was associated with which question, and to be able to check
how many marks were allocated to a given question. Importantly, XML is
not the only format which contains this meta-data. Future work with
examination boards would usefully involve discussion about the
importance and potential of this meta-data for visually impaired students.
This might be in the relative short-term if PDF versions of examinations
were to be offered to visually impaired students as in Scotland (by
adding form fields and potentially ‘tagging’ PDF documents to make
them more accessible and navigable). Either way, it is very likely to be
Evaluation of alternative exam presentation methods for pupils with low vision
87
relevant in the longer term as examination boards develop onscreen and
interactive examinations.
Evaluation of alternative exam presentation methods for pupils with low vision
88
7. Discussion
In this final section of the report we present a summary of the key
findings before considering what this reveals about access to
examinations by visually impaired pupils in England and Wales. This
leads to a short discussion in which we consider how the current
examination modification and formatting procedures for these pupils
could be improved. The recommendations include reference to the
further development of software for the preparation of hard copy and
onscreen versions of examinations. In the final section a broad
‘functional specification’ for this potential software is presented.
7.1 Summary of findings
Below we present the findings from this phase of the research in relation
to the two research questions.
1. How do the Modified Large Print (MLP) versions of papers
produced by RNIB E-formatting software compare to traditional
MLP papers? (both process and outcome)
a. Findings from a trial involving 21 students with low vision suggest that
examinations that are formatted using the RNIB E-formatting software
neither significantly advantages or disadvantages student
performance compared with existing MLP approaches. Familiarity
with examination format appears to be a more important variable.
b. Professionals interviewed were broadly positive about the RNIB Eformatting software:
 They were positive about having a greater range of options for
examination formats which the software could offer.
 They particularly identified control over page breaks and formatting
of pictures/diagrams as key areas for further development.
 There were mixed feelings about how the process of formatting
and production could be incorporated into the short one hour
period before the examination. Some had concerns about
technical problems and working under time pressures.
c. There is evidence that appropriate and consistent modification of
examinations is of key concern to professionals irrespective of the
method of delivery. The process of producing the software versions of
the examination papers (from four different examination boards)
Evaluation of alternative exam presentation methods for pupils with low vision
89
appears to have an important associated benefit, namely
standardising of the modification process and the general format of
the examination papers.
d. Case examples of five students who preferred very large print (greater
than 24 point) demonstrated that they appear to benefit from hard
copy large print formats which are different from those available
through existing MLP approaches. An estimated 9% of GCSE-aged
students with a visual impairment in England and Wales use font
sizes greater than 24 point.
2. What methods of accessing examinations are useful for the
future, and can the RNIB E-formatting software (or a developed
version of it) play a useful part? (e.g. electronic copies, interactive
papers)
a. When reflecting on the ‘perfect scenario’, many professionals reported
that students should have access to examinations which had been
modified appropriately and could then be formatted in a flexible way to
meet individual needs. The use of technology was seen as an
important part of the formatting solution by all participants (teachers,
students with low vision and non-print readers). This suggests a
distinction between appropriate modification of a paper ‘at source’ and
the formatting options that could potentially be provided by the
software.
b. Access to on-screen interactive examinations was seen by both
students and professionals as offering a range of opportunities for
improving access. Nevertheless, further work is required to enable
these opportunities to be developed further. Key challenges identified
through discussion with students (blind and low vision) as well as
professionals included:
 technical work is required to ensure computer-based material is
fully accessible;
 appropriate examination modification is carried out ‘at source’;
 training of students in the use of assistive technology to access
exams;
 training of staff to teach students how to use this assistive
technology for exams access.
c. Future development needs to be flexible enough to offer a range of
on-screen formats (e.g. a range of fonts, sizes, colours, and zoom
facilities). They also need to include options for hard copy versions of
Evaluation of alternative exam presentation methods for pupils with low vision
90
questions and/or supporting information such as diagrams and tables
to be produced in a variety of formats.
7. 2 Classroom practice versus public
examination system
In formulating project recommendations, the findings listed above can be
considered within a broader national context with a particular focus on:
classroom practice and the public examinations system.
Classroom Practice
In considering classroom practice, a commonly expressed view amongst
those professionals interviewed was that ‘good’ service provision should
provide bespoke hardcopy materials in order to meet the individual
needs of visually impaired pupils. These interviews took place in the
context of their general dissatisfaction with currently available exam
formats, in particular the limited range of options available for students
with visual impairment. In this context therefore it is perhaps not
surprising that such ‘student-centred’ views were espoused.
However, in a separate literature review carried out by some of the
authors (see Douglas et al, 2009), it was argued that these views are not
in keeping with literature which promotes the use of assistive technology
and low vision aids (LVAs) to enable students to access information
independently. For example, a review of studies carried out by
Lussenhop and Corn (2002) concluded that the eight key studies they
identified “point strongly toward a conclusion that reading standard print
with optical devices is as effective a literacy medium as large print—and
perhaps a more effective one” (p. 67). The authors note that LVAs are
not always the appropriate solution, but even so they feel it is important
for teachers and students to re-examine “assumptions and traditional
reliance on large print” (p. 68). In their review Douglas et al (2009) argue
that the dilemmas teachers face perhaps “highlights a tension between
educational approaches and contemporary policy directions in relation to
disability; arguably the former would emphasise teaching students
access skills (i.e. how to use LVAs), while the latter would emphasise
providing materials to optimise access (i.e. the provision of bespoke
large print)” (p. 101).
Evaluation of alternative exam presentation methods for pupils with low vision
91
In practice, the take-up of handheld LVAs by pupils with low vision in the
UK is low (e.g. Mason, 1999), many preferring to use bespoke large
print. As presented in section 3 of the report, this large print is
sometimes, but not always, 18 point Arial Bold. However, the research
also suggests that many pupils with visual impairment use computerbased assistive technology to access print.
Public Examinations System
In considering the public examination system, there is some evidence
that current examination modification and formatting process is
implemented inconsistently and with limited quality assurance
procedures. This is based upon the interview data and analysis of MLP
papers presented in this project as well as findings of Cobb (2008).
Putting aside the quality of the implementation of the current examination
modification process, interviews in this project also suggest that the
current examination process does not appear to reflect classroom
practice for students with visual impairment who read print. The most
obvious examples from the project of this ‘disconnection’ are students
with low vision who do not routinely use 18 point Arial Bold in their dayto-day education must still use it in their examinations.
7.3 Possible solutions
Such a ‘disconnection’ is clearly not helpful to students seeking to gain
qualifications. Students without visual impairment do not experience
such a difference between their way of working in the classroom and
their experience of assessment2. Solutions must seek possible ways of
encouraging greater harmony of practice.
The first report produced as part of this research project (‘Summary
report on international systems of exam access for visually impaired
pupils’) noted that providing visually impaired students with access to
examinations is challenging across all the ten surveyed countries. The
report also noted that approaches used in England and Wales are typical
of approaches used across most of the surveyed countries (albeit to
differing degrees), e.g. hardcopy large print, Braille, content
modifications, extra time allowance, and use of scribes. However, the
2
Although it is interesting to note that the current chief executive of Ofqual, Isabel Nisbet, has recently
speculated that teenagers are more comfortable typing answers on a keyboard than writing them
down and raised concerns, “We need to make sure [the way pupils are tested] isn’t overtaken by the
modern world and doesn’t become a relic of the early 20th century.” See Shepherd (2010).
Evaluation of alternative exam presentation methods for pupils with low vision
92
availability of electronic versions of examinations was relatively common
across the surveyed countries and this is in contrast to the approach in
England and Wales.
It was argued that where electronic versions of examinations do not
exist, visually impaired students’ access to examinations appears to be
threatened because the range of formats they require is difficult to
provide. Therefore, the importance of electronic versions of
examinations seems a critical approach to enable students to gain
access to their preferred format.
This approach to accessing electronic versions of examinations is
adopted by the RNIB E-formatting software which has been evaluated in
this research. The software as currently developed aims to provide a
method for teachers to create a range of hard copy formats of
examinations while maintaining the security of the examination content.
It offers good control over print style and print size which in principle
could provide a wider range of presentation options than currently
available in England and Wales. The process of producing the software
versions of the examination papers also appears to have an important
associated benefit, namely standardising of the modification process and
the general format of the examination papers.
However, evidence suggests that while printed hardcopy examination
papers are important, student access to electronic versions of the
examinations is also important. It offers several potential advantages:
 Some students seem to prefer it, particularly those who require very
large print (greater than 24 point);
 It reflects the way many students with visual impairment work in the
classroom;
 It is a pragmatic solution for those students who require fonts bigger
than 24 point (because it is difficult to produce hardcopy versions of
exam papers in these font sizes);
 It has the potential of offering other computer-based functionality
which is of use to visually impaired students.
Evaluation of alternative exam presentation methods for pupils with low vision
93
7.4 Recommendations
We present two broad recommendations based upon the research
findings. The recommendations are made in relation to the project
research questions, i.e. particularly linked to the potential of the RNIB EFormatting software (or similar) in the provision of accessible
examination papers. It is important to note that the authors advise that
the recommendations are implemented as part of a more general review
of examination and curriculum access approaches for visually impaired
students in England and Wales, specifically in relation to:
 The procedures and quality control of examination provision (including
the process of modification);
 Curriculum access strategies adopted in the classroom (particularly in
relation to the use of low vision aids and computers).
Recommendation 1:
A software application for supporting the production of an increased
range of examination formats for students with visual impairment should
be developed. The application would be used by teachers and visually
impaired students, and it should have the following functions:
a) a range of formatting options;
b) printing facilities for students who require hardcopy large print
examinations;
c) onscreen presentation facilities for students who require onscreen
access to examinations.
The report provides a draft functional specification for the software
application. The RNIB E-formatting software evaluated in this research
project provides a very useful prototype of such an application.
Recommendation 2:
The future development of mainstream interactive computer-based
examinations presents significant opportunities for improving access to
examinations amongst visually impaired students. To ensure that
visually impaired students can benefit from these opportunities the
following general recommendations are made:
a) Examination boards and designers of computer-based
examinations technology should work with access technology
experts (for example, within RNIB) to ensure that inclusive design
principles are incorporated into the development of this technology.
Evaluation of alternative exam presentation methods for pupils with low vision
94
b) Developments in computer-based examinations will mean that
educators must continue to ensure that visually impaired pupils are
taught appropriate access skills so that they can independently
access these examinations.
7.5 Functional specification of future
software
It is difficult to write a functional specification for a piece of software
which is only part of an examination modification process (and this
process is not necessarily fixed). Nevertheless, we suggest the following
functional specification based upon our interpretations of the research
findings. If developed, the software may not incorporate all the proposed
functions presented because it will depend upon stakeholders’ decisions
about how they wish to respond to the project recommendations. The
key stakeholders are the examination boards in England and Wales, and
decisions must be made as to whether the software should be a tool for
preparing hardcopy print, a tool for presenting onscreen examinations, or
both. The proposed software functions are therefore grouped into
several broad areas which are presented in turn.
 General formatting facilities
 Printing facilities
 Onscreen presentation facilities
 Security
 Interactive facilities
 Contextual requirements
General formatting facilities
A central function of the software will be the formatting facilities. This is
true whether the software is designed to prepare material for printing
hardcopy exams or for onscreen presentation. The functions are divided
between formatting facilities related to text (e.g. font style and size),
structure / layout (e.g. line spacing), and tables, pictures and graphs (e.g.
size manipulation). It is also noted that the software must have the
facilities to accommodate the particular formatting requirements of some
curriculum areas (e.g. poetry indents and line breaks, mathematical
equations).
Evaluation of alternative exam presentation methods for pupils with low vision
95
Table 26. Specification in relation to general formatting facilities
Specification
Text
Font size
Font style
Font weight (bold/not bold)
Colour (foreground / background). Large colour range. Note: only
relevant for onscreen.
Structure / layout
Line spacing
Page breaks. Note: only relevant for printing.
Tables / Pictures / diagrams
Line width (diagrams/graphs)
Line width (tables)
Size (diagrams/graphs/pictures)
Specific curriculum requirements
Facilities to accommodate the particular formatting requirements of
some curriculum areas (e.g. poetry indents and line breaks,
mathematical equations).
Printing facilities
Many students and professionals felt that hard copy versions of exams
were still a preferred format. Given this, the preparation of bespoke
printed materials is a key function of the software. The current version of
the RNIB E-formatting software has particular difficulties with some
aspects of print preparation. Three areas of the functional requirements
have been identified in relation to printing. Firstly, easy formatting and
layout is required: most notably in relation to page breaks, line spacing
and sizing of graphics. Secondly, it is useful to be able to view onscreen
what will be printed on paper. This ‘What-you-see-is-what-you-get’ (or
WYSIWYG) facility is extremely useful and efficient when preparing
printed material. Thirdly, teachers were concerned about the time that
would be taken for the production of material when under time pressures
before an examination. For this reason it may be sensible for some
popular formatted versions of examinations to be ‘print ready’, i.e. when
printed they will have appropriate layout without the need to re-format.
Arial bold 18 and 24 point appear to be traditionally popular.
Evaluation of alternative exam presentation methods for pupils with low vision
96
Table 27. Specification in relation to printing facilities
Specification
Easy formatting of paper spacing and layout – page breaks, line spacing
for specific paragraphs, picture / diagram / table sizes.
WYSIWYG - ‘What-you-see-is-what-you-get’
Standard formats prepared as print ready (suggest in Arial bold 18, Arial
bold 24)
Onscreen presentation facilities
Many students and professionals identified onscreen versions of exams
as a preferred format. If this presentation method was developed, then
examinations may need to be designed / modified specifically for this
purpose. For example, some exam question structures assume that the
examination is presented on paper (e.g. ‘label the diagram’, ‘circle the
correct answer’). A detailed analysis of the required modifications for
onscreen presentation would need to be considered. It would also need
to account for the student response mode (e.g. using a word processor).
There are a range of functions which were identified which would support
onscreen presentation. The most important are the general formatting
options described above. However, there may be a variety of ways of
implementing these formatting facilities, e.g. some functions may be
dynamic, enabling users to change them in real time (not offered by the
current version of the RNIB E-formatting software). Whether these
functions are embedded within the design of software, or require the use
of additional operating system / access technology (e.g. a screen
magnifier) would need further consideration.
Additional facilities identified in the research might include some
navigation and search facilities. For example, this might enable users to
use key strokes to move from question to question, or search for text
expressions.
This model of onscreen examination presentation would most likely
involve the user typing the answers to the examination in a word
processing package (and submitting a printed version of the answers to
the examination board at the end of the assessment). In this case, the
software would require that the user could ‘toggle’ between software
applications using standard operating system routines, or view the
applications on the same screen if desired. Making use of other
standard facilities may also be helpful, for example being able to ‘copy’
Evaluation of alternative exam presentation methods for pupils with low vision
97
text from the examination document and ‘paste’ into the answer
document may be very helpful when constructing answers.
Finally, using the software in combination with screen magnification
and/or screen reading access technology may be helpful for some users.
Table 28. Specification in relation to onscreen presentation facilities
Specification
Formatting facilities as described above (general formatting facilities).
Keyboard navigation (e.g. up, down; left, right; page up, page down;
question by question)
Search and cross-referencing facilities
‘Copy and paste’ facilities
Easy toggle between examination software and word processing
(answering) software
Compatible with magnification software, or inbuilt facilities.
Compatible with screenreading software
Security
It is important that the software is able to maintain the security of the
examination content. In practice this means that the software should not
allow accidental or malicious changes to the wording or graphical content
of the examination (e.g. the removal or addition of a word, removal or
changing of graphics).
Interactive facilities
A more speculative facility which was identified by the research was the
potential of ‘interactive examinations’ which allow for answers to be
entered into appropriate ‘answer fields’. Some online assessments are
emerging in the UK and elsewhere which have these qualities, and the
PDF versions of examinations offered in Scotland by the SQA also have
some of these facilities. Specifications are not presented because of the
speculative nature of the idea. Nevertheless, things to consider would
be the availability of extensive keyboard facilities for navigation, and
access for non-print users (including linking question content and answer
boxes and other ‘meta data’ including available marks).
Evaluation of alternative exam presentation methods for pupils with low vision
98
Contextual requirements
The specifications outlined above are linked to the direct functioning of
potential software. An important part of the interviews with professionals
was in relation to contextual issues: i.e. how and when such software
might be used, and the process of requesting access arrangements for
given visually impaired pupils.
Table 29. Specification in relation to contextual requirements
Specification
Software is delivered on time.
The software is in a format which can be routinely run on standard
equipment (the RNIB E-formatting software arrived in the form of a CD
and this serves as a useful example).
Appropriate early opening time allowances are possible.
Past papers are available for practice.
There is a clear protocol (including realistic timescale) for requesting
access arrangements.
Modification and formatting is applied consistently for all examinations.
Evaluation of alternative exam presentation methods for pupils with low vision
99
8. References
Cobb, R. A. (2008). Exploring systems for the provision of modified large
print examination papers to partially sighted candidates in England
and Wales. MPhil, School of Education, University of Birmingham.
Douglas, G., McLinden, M., McCall, S., Pavey, S., Ware, J. Farrell, A. M.
(2009). International review of the literature of evidence of best
practice models and outcomes in the education of blind and visually
impaired children. Report for National Council for Special Education
(NCSE), Ireland. Accessed 17th June 2010:
http://www.ncse.ie/research/researchreports.asp
Joint Council for Qualifications (JCQ) (2008). Best practice guidance for
the modification and production of examination papers for
candidates with a visual impairment. Accessed 13th June 2010:
http://www.rnib.org.uk/professionals/education/schoolbasedlearning/
mainstream
Lussenhop, K. and A. L. Corn (2002). “Comparative studies of reading
performance of students with low vision.” RE:view 34(2): 57–69.
Mason, H. (1999). “Blurred vision: a study of the use of low vision aids by
visually impaired secondary school pupils.” British Journal of Visual
Impairment 17(3): 94–97.
Shepherd, J. (2010). Should computers replace pen and paper in the
exam room? Discuss. The Guardian, Thursday 19th August, 2010,
page 8.
Evaluation of alternative exam presentation methods for pupils with low vision
100
Appendix 1
Student Questionnaire
Name:
_______________________
What type of school do you attend? (Choose one)
Mainstream
Resource base
Special
Other:
For the LARGE PRINT version, list any things that
you liked and disliked:
Liked
Disliked
The SOFTWARE version, list any things that you
liked and disliked:
Liked
Disliked
Evaluation of alternative exam presentation methods for pupils with low vision
101
Which version of the examination do you think was
easier to access? (Choose one)
LARGE PRINT
SOFTWARE
Both the same
Unsure
Please explain your answer:
Given any choice, what is your preferred print size
and font?
Print size: ________
Font:
________
Do you use an LVA when reading? Yes/ No
Explain how you would have liked the examination
papers formatted for YOU?
Evaluation of alternative exam presentation methods for pupils with low vision
102
Consider if the examination papers were presented
‘on-screen’. What would be the advantages and
disadvantages for YOU?
Advantages
Disadvantages
Evaluation of alternative exam presentation methods for pupils with low vision
103
Appendix 2
Font size questionnaire
This is 18. A Ford is a kind of car.
This is 24. A carrot is a kind of
vegetable.
This is 28. An orange is a kind of
fruit.
This is 32. A cat is a kind of
animal.
This is 36. The earth is a
planet
This is 40. The sun is a
star.
Evaluation of alternative exam presentation methods for pupils with low vision
104
Questions about size of print you like and use
Name: _______________
Date: ______________
General
Do you ever use an LVA when reading? Yes/ No
Given any choice, what is your preferred print size and font?
Print size:
________
Font:
________
- Is this with or without and LVA?
What font size do you usually use in the classroom / day-to-day work? __________
- Is this with or without and LVA? (also ask about computer use)
18 point
(answer 1 to 4, 1 is easily, 2 is OK, 3 its hard, 4 is No, or TOO BIG)
Can you read this? ____
(answer yes, no, maybe, don’t know)
Could you read it for a long piece of text (e.g. an examination paper)? ___
24 point
Can you read this? ____
Could you read it for a long piece of text (e.g. an examination paper)? ___
28 point
Can you read this? ____
Could you read it for a long piece of text (e.g. an examination paper)? ___
32 point
Can you read this? ____
Could you read it for a long piece of text (e.g. an examination paper)? ___
36 point
Can you read this? ____
Could you read it for a long piece of text (e.g. an examination paper)? ___
40 point
Can you read this? ____
Could you read it for a long piece of text (e.g. an examination paper)? ___
Overall
For you, which font size would you prefer for classroom/day-to-day work? ___
For you, which font size would you prefer for examinations? ___
Evaluation of alternative exam presentation methods for pupils with low vision
105
Appendix 3
Professional Interview Schedule
Section A - Hard Copy Examinations
1) Original paper
This is an example of standard examination papers (not modified, not
enlarged). Most students without visual impairments would receive this
version of the exam paper. Explore the paper and give your thoughts
under the following headings.
Thinking about students with low vision (who are print readers):
a)
Many students work with a variety of material (including
unmodified, ‘normally sized’ print) – including but not exclusive to
exams. For the students you know, how familiar would they be
accessing information in this format? Prompts: Give examples;
Service policy?
b)
How easily would the students you know access this material?
Prompts: Give examples.
What about navigate this material? That is, being able
to move from question to question, link between
question and source material, etc.
Evaluation of alternative exam presentation methods for pupils with low vision
106
2) Modified Large Print (MLP) paper
This is an example of modified large print (MLP) examination papers
(point size 18, and some of the content modified). Many students with
visual impairments would request and receive this version of the exam
paper. It is acceptable for papers to be photocopy-enlarged in the one
hour before the examination (giving a font size of approximately 24
point). Explore the paper and give your thoughts under the following
headings.
Thinking about students with low vision (who are print readers):
a)
Many students work with a variety of material (including MLP – that
is modified to a ‘standard’). For the students you know, how
familiar would they be accessing information in this format?
Prompts: Give examples; Service policy?
b)
How easily would the students you know access this material?
Prompts: Give examples.
What about navigate this material? That is, being able
to move from question to question, link between
question and source material, etc.
Evaluation of alternative exam presentation methods for pupils with low vision
107
3) Software formatted paper
DEMONSTRATE THE SOFTWARE
This is an example of software formatted examination papers (in the
hardcopy examples given the font is Arial point size 28, and Comic 18).
As demonstrated, other versions of the examination paper can be
prepared using the software (different font styles, different font sizes).
Potentially, students with visual impairments could request and receive a
version of the exam paper which could be formatted using this kind of
software in the hour before the examination. Explore the paper and give
your thoughts under the following headings (remembering that the
software can be used to create the same examination paper in different
formats).
Thinking about students with low vision (who are print readers):
a)
Many students work with a variety of material (including material
prepared in a format of their choosing). For the students you know,
how familiar would they be accessing information in this format?
Prompts: Give examples; Service policy; Software / electronic
formats used?
b)
Using this type of software, consider its potential for giving students
access to examinations? [Note: the example printed is Arial point
28, but any other font / size could have been chosen]
Prompts: How might it be improved?
What about navigate this material? That is, being able to
move from question to question, link between question and
source material, etc.
Evaluation of alternative exam presentation methods for pupils with low vision
108
c) Think about managing the process. Given that the use of this type
of software would need a member of staff preparing material before
the exam started – how do you think you / your service / the school
could manage this?
Evaluation of alternative exam presentation methods for pupils with low vision
109
Section B – Onscreen Examinations
Original and MLP papers onscreen
Although it is currently not an option for examinations, we are interested
in your views about the possibility of onscreen presentation.
In the examples we’ll give the standard and MLP examination papers are
presented onscreen through Adobe Acrobat Reader. This onscreen
presentation of the examination paper enables the user to use software
tools to enlarge or reduce the onscreen display. DEMONSTRATE THIS.
Thinking about students with low vision (who are print readers):
a)
What are your initial reactions to this onscreen method of
presenting examinations?
b)
Thinking more broadly than examinations, students can access
their school work using computers. We’re trying to understand how
common this is for the students you know. How common would
you say it is? Remember, we want you to think about students
who read print.
Prompts:
How many students use laptops in class?
How many students are given curriculum material in electronic format?
Please give examples (of students who do and students who do not)
Does the service have a policy on this way of working?
Evaluation of alternative exam presentation methods for pupils with low vision
110
c)
Consider if examination papers were presented ‘on-screen’. What
would be the advantages and disadvantages for the pupils you
work with?
Advantages
Disadvantages
Section C – In a perfect world
1) Think about what we have discussed, including the exam options
which are currently available (modified large print papers in Arial 18
point) and options which could one day be available (e.g. software for
more bespoke formatting, onscreen examinations). What type of
examination provision for visually impaired pupils would you like to
see in the future?
[Prompt:
distinction between the student controlling the access, or materials being
prepared to his / her specification
What about your role / the role of staff in this ‘perfect scenario’]
Evaluation of alternative exam presentation methods for pupils with low vision
111