ICT Prototyping Phase 8 Evaluation
EXPLORE THE MORE FAMILY PORTRAIT
12th September 2000
Executive Summary
Revised usability of the Explore Terminal is now felt to be satisfactory and the
design/functionality of these devices is felt to be nearly complete. The Index feature will be
changed to work via one-touch operation and Oyster are asked to explain the options for the
functionality of the Scan feature. Once this final main point is complete, Oyster can proceed
with the style guidelines for these devices and the Museum will work towards completion of
all content for the explore terminals.
Introduction
Brief for the Device
The Explore the More Family Portrait terminal is one of three "Explore" devices that will
appear in the galleries. It will be positioned in the ‘Achieving Splendour’ display near the
entrance to gallery 58. The following objectives for the device are those taken from the
original Evaluation Strategy for the ICT.
• To help visitors understand the social and cultural life of the upper classes in the early
Tudor period through exploration of the painting.
• To show how this painting relates to an earlier one.
• To link features in the painting to themes that will extend the visitors understanding of the
painting and period.
• To encourage visitors to discover how features and details in this painting relate to other
objects in the V&A collection.
Brief and Overall Methodology for Evaluation
An initial round of public testing was conducted with the public between the 13th March and
16th June 2000. The analysis and evaluation of this testing yielded a number of changes to
the design and content of the device and these were implemented by Oyster prior to a
second round of testing. The Museum wished to assess whether the changes implemented
had succeeded in tackling the usability issues that had arisen during the first round of
testing. In particular:
• Look again at the GO button on the index screen
Analysis of the evaluation focused on usability, content (audience relevance) and technical
performance.
The second round of testing was conducted with samples of 10-15 people depending on the
range of questions involved. The methodology was the same as for the first round of testing.
Staff were not included as a separate sample of users for these tests.
EXPLORE THE MORE FAMILY PORTRAIT RE-TEST.DOC
FRIDAY, 11 APRIL 2003
1
Description of Prototype Device
The prototype device was constructed according to the ‘Brief for Phase 6 - Rebuild' section
of the ICT Evaluation Report for Style Guide dated 20th June 2000.
Description of Public Testing
Hardware and Software
Testing was conducted using the same prototype rig as for the first round of public testing:
Two 15" screens were connected to PIII 500's under Windows NT with 128MB RAM. Screen
resolution was 1024x768 with 24bit colour. Operation was via touchscreen.
Server architecture utilised a Sun Ultra5 running applications through Apache under Solaris
7. Database architecture was Oracle 8.0.6 with Java served through NewAtalanta
ServletExec 2 and Sun JDK 1.2.1.04
Test conditions match those specified in the original System Architecture Definition Report.
The exceptions to the definition were:
• Recent updates and patches to previously specified version of server architecture (JDK,
ServletExec, Solaris, Oracle)
• Replacement of Netscape Enterprise Server to Apache
• Switch to a UNIX based server to increase the stability of Oracle database.
The software program was run through Internet Explorer 5.5 running in kiosk mode.
Methodology and Environment
The test was carried out in the Best of British Gallery. Each test screen was placed on a
work surface with stools for visitors to sit on.
Testing was conducted by the Device Manager and Multimedia Manager on 3rd August.
A combination of observation and interview was used in order to see how visitors used the
device without direction.
Description of Sample
Number of people
Gender
Age
English as first language
Visitors
11
Male = 6
Female = 5
Under 25 yrs = 2
25-34 yrs = 3
35-44 yrs = 4
45-54 yrs = 2
55 plus yrs = 0
Yes = 4
No = 7
Visitors were from Germany(3), UK (2), Italy (2), Switzerland, Bahamas, Canada, Portugal,
Spain.
EXPLORE THE MORE FAMILY PORTRAIT RE-TEST.DOC
FRIDAY, 11 APRIL 2003
2
All visitors used a computer at home or work/college.
Two visitors were interviewed in a group of two.
Findings from the Public Testing
VISITORS
Usability
•
•
•
Five visitors either had to be prompted to use the GO button or had some problems
with it.
Six visitors used GO without prompting and appeared to have no problems with it.
Of visitors who stated a preference (nine out of 11) five said they would prefer a onetouch process (i.e. without the GO button), although four of these had had no
problems with the GO button. Four of the nine who stated a preference were happy
with the current two-touch process using the GO button. Ironically, these four had all
initially had some problems with the GO button.
Content (Audience Relevance)
•
Ten out of 11 users went straight to the “History of the Painting” first and most appeared
to read the entire text. However, this is at the top of the screen so no firm conclusions
can be drawn about its popularity, as visitors were not asked why they had chosen this
first. Some people had to then be prompted to touch the HOME button to get back. A
BACK button as well as a HOME button might be useful here.
EXPLORE THE MORE FAMILY PORTRAIT RE-TEST.DOC
FRIDAY, 11 APRIL 2003
3
Conclusions from public testing
•
The GO button posed problems for five out of 11 visitors. Of the remaining six who had
no problems with it, four said they would prefer a one-touch process. This suggests that
the abandoning the GO button in favour of a one-touch process would be beneficial for
the majority of visitors.
•
The position of the “History of the Painting” section within the programme should be
reviewed. It is not clear from the evaluation whether visitors went to it first because they
were most interested in it or because it is at the top of the list of options. In the previous
version of testing, there was a weaker preference for SCAN, which was at the top of the
list. A decision needs to be made about whether it is desired that visitors go to this
section first, particularly bearing in mind the current navigation problems, with visitors
needing prompting to return to the home page.
EXPLORE THE MORE FAMILY PORTRAIT RE-TEST.DOC
FRIDAY, 11 APRIL 2003
4
Conclusions for Phase 9 - Evaluation of Prototyping Phase
ACTION 1
• The GO button posed problems for five out of 11 visitors. Of the remaining six who had no
problems with it, four said they would prefer a one-touch process. This suggests that the
abandoning the GO button in favour of a one-touch process would be beneficial for the majority
of visitors.
• The Index should be operated from one-touch. This means removing the GO button and the
white animated circles on the painting.
• Oyster to remove the two-touch functionality for the INDEX routes. This also means
removing the animated white circle that accompanies this functionality. The revised
operation of the INDEX route can be described in the style guidelines for Explore
devices.
ACTION 2
• The position of the “History of the Painting” section within the programme should be reviewed. It
is not clear from the evaluation whether visitors went to it first because they were most
interested in it or because it is at the top of the list of options. In the previous version of testing,
there was a weaker preference for SCAN, which was at the top of the list. A decision needs to
be made about whether it is desired that visitors go to this section first, particularly bearing in
mind the current navigation problems, with visitors needing prompting to return to the home
page.
• The “History of the Painting” screen needs to include a back button at the top right of the screen.
This page also needs to have the ability to toggle between the new and old paintings. If an
animation can be incorporated into this, this is desirable. If we put a back button here, do we
also need a back button at the top level of SCAN, INDEX and ZOOM?
• Oyster should move the HISTORY OF THE PAINTING button and text to the bottom of
the home page.
• Oyster should revised the functionality of history of the painting such that it enables
the visitor to toggle back and forth between the drawing and the painting. It is
suggested that this is implemented as a simple fade animation in Flash with a single
button operation.
• Oyster to include a BACK button at the top of the screen on the History of the
Painting screens.
ACTION 3
• The navigation of SCAN should work as follows:
At the SCAN Hub Page, changing category should wipe the existing scan and scan for items in
the picture for the new category.
Navigation forward to level 3 and 4 screens should work as current design.
Backward navigation to the category originally selected should go to the end of the scan for that
category.
Backward navigation to a different category from the one originally selected should allow visitor
to see the scan for the new category. This should first wipe the scan of the category previously
selected.
• Oyster should specify whether this functionality is possible. If not, Oyster should
advise on how scans will work to inform style guidelines for this device.
ACTION 4
• Oyster should identify technical faults with the device including slow transition from attractor to
home-page, jerky operation of zoom box, missing level 3 and 4 images.
• Oyster to ensure that repeated errors from the attractor or home page are removed.
These appear to be due to the interactives management system and only appear to
be stopped by several reboots of the Sun Server. This is of course not acceptable.
EXPLORE THE MORE FAMILY PORTRAIT RE-TEST.DOC
FRIDAY, 11 APRIL 2003
5
APPENDIX 1 - Quotes from Visitors and Observation Notes
GO button
•
Visitor had to be prompted to use GO button. Didn’t really see it as there is so much
going on (text and image). Need to get used to it. Asked if it would be better to get rid of
GO button. Said no, once you realise how to use it, it is fine.
•
Used GO immediately. Said it was not necessary to have it. Better for screen to come
up. Had not looked at other Explore Tools without the GO button at this stage.
•
Used GO straight away. Yes it works and shows the detail that is highlighted.
•
Used GO straight away. The GO button is easy to use. Did not have any problem with it.
•
Did not notice at first as looking at rest of screen. Expected it to just pop up. Once you
realise it’s fine.
•
Would prefer a one-touch navigation from INDEX. Didn’t notice white circles. Didn’t pick
up GO button instantly. Needed prompting.
•
I would prefer one touch to navigate. No need for GO button. Liked white circles. No
problem with the GO button. (Two visitors interviewed together.)
•
Would prefer one-touch navigation but understands point of white circle. A bit
undecided.
•
A bit complicated. Didn’t know what to touch but prefer it with two touches. Two touches
helps you make sure you’re going to get what you want. (Two visitors interviewed
together.)
Other
•
Did not see HOME button without prompting on History of the Painting.
•
Used History of the Painting first.
•
Goes straight to History of the Painting. (Two visitors interviewed together.)
•
Four other visitors are noted as using the History of the Painting first.
•
Liked zoom. Paused a lot on index.
•
Very good… Spent ages comparing Holbein sketch to the Lockey picture. (Two visitors
interviewed together.)
•
Used lots of SCAN items first. Seemed a bit lost with the white circle in […]. Told to
touch box. (Two visitors interviewed together.)
•
Didn’t understand the letters on the people in ZOOM. (Two visitors interviewed
together.)
EXPLORE THE MORE FAMILY PORTRAIT RE-TEST.DOC
FRIDAY, 11 APRIL 2003
6
Appendix 2: Questionnaire
Interview questions (Q) and observation tasks (OT)
ASK VISITOR TO USE THE PROGRAMME
OT: Does visitor use scan?
YES
NO
OT: Does visitor use zoom?
YES
NO
OT: Does visitor use index?
YES
NO
ASK THEM TO USE INDEX IF
[Only note comments about GO button]
THEY HAVEN’T DONE SO
Q: “How did you find using this part
of the programme?”
Country of residence
English 1st language
Sex
YES
M
NO
F
Age range
u.25
Use computer at
home/work/college
etc.?
25-34
YES
EXPLORE THE MORE FAMILY PORTRAIT RE-TEST.DOC
FRIDAY, 11 APRIL 2003
35-44
45-54
55+
NO
7