The use of Tablet PCs for marking physics assignments
Jonathan Underwood* and Stuart Freake,
Department of Physics and Astronomy, The Open University
(* now at Department of Physics and Astronomy, University College London)
May 2007
Abstract
Marking and commenting on electronic tutor marked assignments (eTMAs) requires
a simple and efficient method for tutors to annotate students’ scripts. We report here
on a pilot project that provided associate lecturers (ALs) for a Level 2 physics course
with Tablet PCs and appropriate software that enabled them to input handwritten
annotation, including equations and diagrams, to pdf files containing students'
assignment solutions. A survey of ALs views indicated that this method of marking
eTMAs is effective and acceptable to both ALs and students. Some ALs expressed
concern about additional time required for marking; this may not be an issue when
ALs become more familiar with this system, but needs to be kept under review.
1
Introduction
The majority of Open University courses are assessed by a combination of tutor
marked assignments (TMAs), which are submitted by students at intervals during the
presentation of a course, and an end-of course examination. The TMAs are marked
by associate lecturers (ALs, also known as tutors), who are based at different
location around the country and are responsible for a group of about 25 students.
The traditional method of producing and marking TMAs is paper based: a student
writes the answer to their assignment on paper (or word-processes it and prints it
out), they use the postal system to mail it to their AL, who marks and comments on
the assignment by annotating the student’s script and completing a cover form (the
PT3 form). The AL sends marked assignments and PT3 forms to the Open University
in Milton Keynes, where the marks are recorded in a database, a sample of scripts
are copied so that each AL’s marking can be monitored, and the scripts are then
posted back to students. This system involves shuffling large quantities of paper
around the country and there are delays and expenses associated with the postal
system and with the transcription of marks and the copying of assignments for
monitoring. Now that OU students and ALs are expected to have access to a
computer and the internet, some of the delays and costs can be reduced by handling
TMAs electronically.
The OU has developed a system for handling electronic tutor marked assignments
(eTMAs), and is encouraging course teams and students to use this system
wherever possible. The eTMA system allows students to submit electronic files
containing their assignments over the internet to a file handling system on a server in
Milton Keynes. ALs download the eTMA files for their students from the server, mark
and comment on the eTMAs offline, and then return the marked scripts over the
internet to the server in Milton Keynes, where marks are automatically recorded in
the appropriate database and a sample of scripts is made available for monitors to
download for monitoring. Students can then download their marked assignment from
the central server. This system has many advantages for students, ALs and the OU
1
over the paper-based assignment system that relies on the post office for moving the
assignments around the country, but it also has some disadvantages.
Gibbs and Simpson (Learning and Teaching in Higher Education, Issue 1, 3 (20042005) established a list of conditions under which assessment effectively supports
learning. A key finding of this work was that it is essential that “feedback is timely in
that it is received by students while it still matters to them and in time for them to pay
attention to further learning or receive further assistance”. This issue was also
highlighted in the ‘seven principles of good practice in undergraduate education’
(Chickering, A.W. & Gamson, Z.F. (1987) Seven principles to good practice in
undergraduate education, Racine, W.: The Johnson Foundation Inc. and Chickering,
A.W. & Gamson, Z.F. (1991) Applying the seven principles to good practice in
undergraduate education, San Francisco: Jossey-Bass). The eTMA system in
principle allows ALs to give more rapid feedback to students following the cut-off date
for the assignment than does the paper-based system.
However, the benefits of rapid feedback could be more than offset if the time required
for students to produce an eTMA were significantly longer than to produce a paper
TMA, so that their time is diverted from learning physics, and if a longer time were
required for ALs to mark eTMAs to the same standard as paper TMAs, so that the
quality and quantity of feedback provided to students is reduced. Assignments for
physics and astronomy courses (and courses in some other disciplines too) generally
involve large amounts of mathematical notation, equations and diagrams. If these are
to be produced and submitted by students electronically, and marked by ALs using
the OU's eTMA system, then students and ALs need to be able to incorporate
equations and diagrams simply and efficiently. However, many OU students do not
have the skills required to incorporate equations and diagrams into a word-processed
file, and for some of those who do have these skills the additional time required
compared with handwriting an assignment is prohibitive. The same factors apply to
the marking of eTMAs by ALs, though in this case the time factor is generally the
major issue. Keyboard and mouse based tools, such as equation editors and line
drawing programs, are much more time consuming to use compared to annotating a
paper script with a pen or pencil.
The pilot project described in this report was set up to investigate one possible way in
which ALs could mark physics eTMAs easily, and could incorporate equations and
diagrams just as they would if they were marking a paper assignment. A key
requirement was that ALs should be able to handle eTMAs submitted by students in
a variety of file formats, since it was regarded as important that students should be
allowed to use whichever software packages they were most comfortable with when
preparing their eTMA.
2
The project
In March 2006 the Physics Innovations Centre for Excellence in Teaching and
Learning (πCETL) at the Open University agreed to fund a pilot project to investigate
the use of Tablet PCs by ALs for marking assignments. All of the ALs who were
tutoring a Level 2 physics course, S207 The physical world, were invited to
participate in the project. Since the project was set up after the start of the course,
participation by both ALs and students had to be optional, and 19 of the 24 ALs
agreed to take part. The participating ALs were provided with a Tablet PC (Toshiba
Tecra M4), with appropriate software installed, to enable them to add handwritten
annotation to a student’s eTMA file.
2
Students were informed about the pilot project at the end of March, and told about
some of the benefits of using the eTMA system. They were told that not all ALs were
participating, that student participation was optional, and that they should discuss
with their AL whether eTMA submission was possible and what formats were
acceptable for their eTMA files. It was left up to individual ALs to decide how much
encouragement they gave to students to submit their assignments electronically.
The timing of the approval of the project and of despatch of the Tablet PCs meant
that electronic submission was not possible for the first assignment (TMA01), but
most of the ALs were prepared to accept eTMAs for the second and subsequent
assignments (TMA02 – TMA07).
2.1
Marking procedures with the Tablet PC
ALs had a number of options for marking and commenting on eTMAs using the
Tablet PC.
2.1.1
Ink annotation of a pdf file using PDF Annotator
This is the option that we anticipated that most of the ALs would use. The Tablet PCs
were sent out with the PDF Creator application installed (software authored by Philip
Chinery and Frank Heindörfer, free download from http://www.pdfforge.org). This
application creates a “virtual printer” that can be used to convert into pdf format any
file that be printed. The AL simply opens a student's eTMA file – which requires that
an appropriate application is installed on the Tablet PC – selects Print and chooses
PDF Creator as the printing device. 'Printing' the file then produces a pdf version of
the file, which is stored in a location specified by the AL, rather than printing out on
paper. It is also possible to combine a number of different files into a single pdf file.
The AL then opens the pdf file from the PDF Annotator application (Grahl Software
Design, http://www.GRAHL-software.com), which was also preinstalled on the Tablet
PC. With the computer in the tablet configuration, the pdf file can be annotated by
writing on the screen with the tablet pen in exactly the same way that a paper
assignment would be annotated with a conventional pen, but with various
advantages:
- it is easy to change pen colour and line width – so it's possible to colour code
different types of comments;
- it is easy to erase or correct comments;
- it is easy to move or resize comments;
- it is possible to save comments as macros that can then be inserted easily
elsewhere in the same document or in other documents;
- it is easy to insert blank pages for additional comments, or to append files (such as
specimen answers).
When the AL has finished marking, they save the file in pdf format and follow the
standard procedures for returning the eTMA to the OU server. The student
downloads the marked assignment from the server and opens the file using Acrobat
Reader.
One facility that was not available in PDF Annotator in 2006 was the input of text
using the keyboard. When this application was running, the keyboard was disabled,
so even if the computer were in the laptop configuration rather than the tablet
configuration, it was only possible to annotate a document using the tablet pen.
However, an upgrade to PDF Annotator in May 2007 (version 1.5.0.108) now allows
3
simple text to be keyed into a text box, and ALs will make use of this facility when
marking the later TMAs for the 2007 presentation of the course.
2.1.2
Ink annotation using Microsoft Word
A second option for eTMA marking is to use the Ink tools available when Word 2003
(which was installed on the computers that were provided) and subsequent releases
of Word are installed on a Tablet PC. With the computer in the tablet configuration, it
is possible to use the Ink Annotation tool to write with the tablet pen anywhere in an
open Word document, and, as with PDF Annotator, it is possible to change pen
colour and line width, to erase ink marks, and to reposition and resize comments. In
contrast to the PDF Annotator approach described above, employing Microsoft Word
for marking requires that the student submits the TMA in a file format compatible with
the version of Word used by the AL.
Alternatively, the Ink Comment tool can be used to insert comments in the margin.
This is similar to using the normal Insert Comment feature in Word, except that the
comments are handwritten in a balloon in the right margin using the tablet pen.
These comments can include equations and diagrams.
When the AL has finished marking the script, the file is saved in .doc format for return
to the student via the eTMA system.
An important advantage of marking and commenting using Word is that the AL is not
restricted to using ink since, unlike PDF Annotator, keyboard entry is always
available while the programme is running. It is possible to switch back and forth
between using the keyboard to enter for simple text in the laptop configuration and
using the tablet pen to enter equations and figures in the tablet configuration.
Switching between the two configurations of the screen takes about 15 seconds,
which is not an unreasonable overhead, but the switchover is somewhat clumsy and
it is not clear whether the hinge mechanism for the screen is designed for such
frequent use. An alternative procedure tried by some ALs is to connect a USB
keyboard to the Tablet PC, so that both keyboard entry and tablet pen entry are
possible in the tablet configuration.
3
Evaluation
After completion of the 2006 presentation of S207, a brief questionnaire was emailed
to the ALs who had participated in the pilot project, and 15 of the 19 participants
responded.
Number of eTMAs submitted The extent to which the ALs had engaged with
eTMA marking during the year was equal to the number of TMAs that their students
submitted electronically, since the participating ALs were required to mark
electronically all of the eTMAs that they received. There was a large variation
between the numbers of eTMAs marked, as shown in Figure 1. Eight ALs received
four or fewer eTMAs during the year, but two ALs received over 30, and one of them
received 44. The total number of eTMAs submitted for each assignment increased
for the first few assignments, as shown in Figure 2, then remained fairly steady for
TMAs 04 – 06, and dropped slightly for the final TMA. However, the percentage of
TMAs that were submitted electronically increased throughout the year, reaching
33% for the final TMA.
4
frequency
3
2
1
0
0
5
10
15
20
25
30
35
40
45
50
number of eTMAs marked by ALs
Figure 1 Number of eTMAs marked by each of the 19 participating associate
lecturers.
60
number or percent of eTMAs
number of eTMAs
50
40
30
20
eTMAs as % of total TMAs
10
0
02
03
04
05
TMA number
06
07
Figure 2 Total number of eTMAs received for TMAs 02-07, and eTMAs as a
percentage of the total number of TMAs submitted to the 19 participating associate
lecturers.
5
Format of the eTMA files submitted ALs reported that most of the eTMAs were
submitted as Word doc files, but some students submitted pdf files and occasionally
there were files produced with Star Office or MS Works. Some students inserted
scanned diagrams into Word files, and a few students scanned a complete
handwritten TMA and submitted it either as a set of jpg files or combined into a single
pdf file.
Marking procedures used by ALs Most ALs converted students' files to pdf (if
they were not already in pdf format) and used the PDF Annotator application to
annotate them. Several commented that they liked this way of working because it
mirrored their usual method of marking paper TMAs. Four ALs used PDF Annotator
for files that were submitted in pdf format, but when a student submitted a Word
readable file they opened the file in Word 2003 and used the standard Word facilities
to insert comments, track changes and insert text boxes, plus the facility provided
with the Tablet PC software to use ink annotations and ink comments for equations
and diagrams. One AL indicated he preferred this method to using PDF Annotator as
it enabled him to use the keyboard for text entry and he could type text much faster
than he could write.
Difficulties in learning how to use the marking system ALs were asked whether
they had problems learning how to use the Tablet PC or the associated software.
Even though the only information that they had received was the standard user
manuals for the computer and the software, there were few reported problems. One
person was averse to the use of an additional computer and did not complete the
setting up of the machine because they were put off by 'problems' discussed by
colleagues on the S207 AL FirstClass forum; however, they received no eTMAs so
had no impetus to engage with the system. A few tutors said they would have
appreciated more guidance on the conversion of files to pdf format and on the use of
PDF Annotator, and one commented that the use of Print File in PDF Creator to
produce a pdf file was counterintuitive. A couple of ALs thought there should be
explicit guidance about how to insert pages into a script for additional comments. The
other comments made by ALs were related to difficulties experienced in learning how
to use the standard eTMA file handling system. One AL handed the training materials
to his (presumably IT-literate) son, who then trained him (and received the training
fee).
Ergonomic considerations Six tutors said that it took them some time to find a
comfortable way to mark using the Tablet PC. Three of them found that it was difficult
to use the Tablet PC while wearing varifocal glasses, and two of them overcame this
problem by using single prescription lenses while marking. Two ALs worked with the
tablet on their laps and another found that they had to use a lower working surface.
In future it would be appropriate to provide some 'ergonomic' advice to ALs as part of
a brief guide to the use of the Tablet PC for eTMA marking.
Ongoing problems and disadvantages of using the Tablet PC for marking The
main ongoing problems and disadvantages that ALs mentioned were:

some ALs found that it took longer to mark eTMAs, particularly those who
only marked a small number of eTMAs and had to relearn how to use the
system each time;

it was difficult to trawl through pages to get an overview to ensure that the
comments were appropriate, or difficult to lay out pages and compare several
scripts to ensure consistency;

it was not as convenient as paper for marking while waiting in the car;

it was not possible to use the keyboard while running PDF Annotator; ALs
wanted the ability to insert both typed comments and handwritten comments
6
on the students’ eTMA script; (this is something that is now possible with the
May 2007 release of version 1.5.0.108 of the application);

it is not possible to cut and paste in PDF Annotator;

students who previously submitted early, perhaps because of uncertainties
about the postal service, now tended to delay submission until just before the
deadline.
Advantages of using the Tablet PC The ALs were asked about the advantages of
using the Tablet PC compared with paper marking, and the points they mentioned
were:

the ability to erase comments, or to revise them; they were pleased that they
did not need to use Tippex or to cross out comments; they thought their
comments were often more considered, since they could be changed if
necessary after reading a later part of the answer;

no longer needing to contend with the piles of paper associated with
conventional TMA marking; also, the pages can't get out of order;

the ability to copy and paste comments between different TMAs;

comments could be more legible – though it may be necessary to write more
slowly, or to rewrite comments;

the ease of inserting relevant parts of the specimen answers;

some ALs said that the system required less time than conventional marking;

automated PT3 addition avoids errors in part marks and totals;

improved turnaround time, particularly for students outside the UK;

avoids use of the paper-based mail delivery system; it is easier to upload
marked eTMAs than to post paper TMAs.
In fact, most of advantages and disadvantages noted by ALs apply equally to the
standard eTMA marking system, and they are not Tablet PC specific. However, since
the S207 ALs had not experienced the standard eTMA system, it is not surprising
that they focused on how the eTMA system that they were using differed from the
familiar paper-based system rather than how it differed from the standard eTMA
marking system. Thus none of the ALs explicitly mentioned in their responses to the
survey the major advantage of the Tablet PC compared with marking using the
standard eTMA system – namely the ease of inserting equations or diagrams –
though this was commented on in the AL forum during the year.
Student feedback The small amount of feedback that ALs received from students
who submitted eTMAs was positive, with most of the comments about the quick
turnaround and the ability to submit right up to the cut off time. However, some
students did also comment on the clarity and legibility of the comments that ALs
made in the returned file.
Overall feedback from ALs Almost all of the ALs were very positive about the pilot
project. There were some reservations from those who found that eTMAs took longer
to mark – and this needs to be reviewed when ALs are more familiar with using this
method of marking – and one monitor thought that ALs may be giving less feedback
in order to keep marking time within reasonable levels – again this needs to be
reviewed after the 2007 presentation.
7
4
Future work
In 2006, less than 20% of S207 assignments were submitted via the eTMA system.
Many students lack the IT skills to incorporate equations and diagrams into a wordprocessed TMA, and/or find that producing an eTMA is unacceptably timeconsuming. During the 2007 presentation, another πCETL project will attempt to
increase the number of students submitting electronically by offering students the
loan of either a scanner or a digital pen. These systems allow the student to write
their assignment answers on paper, but then to convert it to a digital format that can
be submitted via the eTMA system. Full instructions will be provided that should
make this process accessible to less confident students, and the additional time
required for the conversion process should be relatively small.
In 2007 all S207 student have the option to use the eTMA system, so all ALs must be
prepared to mark electronically. The four S207 ALs who did not participate in the
project last year have now been provided with Tablet PCs, as have two new ALs. A
further evaluation of ALs experience will be carried out at the end of the year.
5
Conclusion
This pilot project indicates that the Tablet PC provides an effective and acceptable
method of marking physics eTMAs – both from the viewpoint of the AL and the
student. The time required for marking needs to be investigated more closely when
ALs are more familiar with using the system to ensure that additional burdens are not
being placed on them, and eTMA scripts need to be monitored to ensure that the
quality of marking remains high. However, it must be recognized that although Tablet
PCs provide a good way of marking S207 eTMAs, their capital cost is likely to
prohibit their widespread use.
Acknowledgements
We are grateful to the S207 ALs who participated in this pilot project, and whose
comments on the FirstClass forum during the 2006 presentation and whose
responses to the end of course survey were used as the basis of this report: Carol
Boote (R03), Alan Cayless (R11), Sarah Chyriwsky (R09), Jeff Collins (R10), Andy
Diament (R03), David Dolman (R13), Peter Hancock (R04), David Keen (R02), Craig
McFarlane (R06), Graham McMaster (R11), Rami Mehrem (R08), Harald Molgaard
(R01), Jim Noras (R07), Alan Robinson (R05), Ian Saunders (R08), David Smith
(R01), Colin Williams (R09) and Sheila Wright (R07).
We are grateful to Mike Watkins (S207 Course Manager), Tracey Woodcraft (S207
Course Assistant), Kevin Mayles (πCETL Centre Manager) and Diane Ford (πCETL
Centre Assistant) for their contributions to running this project. We also acknowledge
valuable support from Rukhsana Malik, Una Lawson and Asvin Savjani (Student
Services).
This project would not have been possible without the funding and support provided
by the piCETL, which in turn is funded by the Higher Education Funding Council for
England.
8