Concept Testing at the Riverside Museum
Richard Williams
Glasgow Museums is exploring ways of using digital interpretation to communicate
the narrative of display
In 2011 Glasgow’s Museum of Transport will move from its current home to a new,
purpose built museum on the site of an old shipyard on the banks of the River Clyde.
This new Transport Museum, currently titled the Riverside Museum Project, is
designed by the renowned architect Zaha Hadid. It offers the chance to create a
world-class visitor attraction that will contribute to the regeneration of the river Clyde
and improve the display of the city’s important transport, technology and related
collections. The collections will be re-housed in improved conservation and
environmental conditions, and our visitors will enjoy better facilities and improved
access to the objects. Perhaps most importantly, the new displays are an opportunity
to re-think our methods of interpretation and display of our collections.
My role on the Riverside Museum project is to develop the digital interpretation, to
help define the research and content and to develop the approaches we will take in
interpreting our collections. In this, I work with the curatorial team made up of the
senior curator and ten curatorial staff. The approaches we develop are refined
through evaluation in conjunction with the Visitor Studies officers. As a team, we are
conscious of the possibilities digital resources offer in delivering our goals of
introducing movement and narrative to our displays and are keen to develop this
content. However, what has become apparent is that while there is a deep
understanding in museums of the practicalities of digital displays, there is still
relatively little appreciation of how digital interpretation works for individual audience
groups. During the content development stage for Riverside we have developed a
research approach which has been an attempt to address this shortfall.
Riverside’s story-based approach
A significant influence in the planning for the new Riverside Museum has been the
redisplay of the Kelvingrove Art Gallery and Museum in Glasgow, which opened to
the public in 2006. This proved to be a great success; during its first year it received
over 3 million visitors, more than any other visitor attraction in Britain with the
exception of the British Museum. Since then, it has settled down to a figure of over a
million a year, which still makes it the most visited museum outside London.
The redisplay of Kelvingrove was a move away from the more traditional
chronological or taxonomic displays where pre-defined narratives as specified by the
curator are already in place and the objects selected to go on display are fitted
around that narrative. Instead, Kelvingrove adopted a display philosophy of
‘storytelling’ and it is this approach which has been continued at Riverside. Through
Riverside Concept Testing – Richard Williams05.05.09
the concept of developing ‘stories’, the selection of new displays is based solely on
two criteria: the significance and strengths of the collections and audience interest.
Planning the displays starts with research of the objects themselves and the stories
they can tell, rather than allowing a chronological or taxonomic determinant to dictate
how an object will be displayed. This means that displays are being developed which
are self-contained, focused on the object’s real significance and relevant to the
museum’s target audiences. Once a story has been agreed, a key message and
Generic Learning Outcomes are agreed for the story which will define what a visitor
should pick up from the display in order to understand the story.
Each story display within the new Museum has a specific target audience defined for
it at the start of the development process. There are five target audiences for
Riverside; families, under-five children, schools, sensory impaired and teenagers,
and these were agreed to reflect those groups already coming to the existing
Transport Museum and those groups, such as sensory impaired, for which there
should be additional provision. This stage is deliberately completed after the story
and its key message has been agreed; the implication is that any story should be
capable of being delivered to any of our audiences, though the methodology and the
content of the display would inevitably differ.
For each story display, an assessment was made as to the most effective method of
delivering the key message. This decision was based on the available historical
material we had to tell the story, the appropriateness of delivery for the chosen target
audience and the story we are seeking to tell. Sometimes this has resulted in a
wholly text and graphics approach, while for other stories it means a film, an audio
commentary, or a digital interactive. The result is that about 40% of our displays
feature digital interpretation as the ‘main’ method for delivering the content.
The key element in the displays at Riverside will be the collections themselves. All
interpretation, whether it is a text label or a touchscreen, should encourage the
visitor to look more closely at the objects on display and gain a better understanding
of them. This does not mean that all interpretation should be a mere description of
the object, but instead it should focus on an object’s significance, or the story behind
it. To deliver such narratives, all forms of digital interpretation can be just as effective
as the text panels and in some instances far more so.
The museum experience
Unfortunately it is not always the case that museum displays use digital content to
deliver interpretation effectively. Instead it often seems the digital content in a gallery
(whether an AV screen or IT interactive) will sit uneasily alongside the ‘core’ textbased display and will not be used to communicate the main narrative of the
exhibition. Instead it will deliver supporting material such as images or videos which
illustrate in the most literal way what is being discussed in the exhibit; film footage
created for broadcast in another medium such as television is shown simply edited
Riverside Concept Testing – Richard Williams05.05.09
and largely unfiltered. Excerpts from much longer oral history interviews are
broadcast in the gallery, when the original interview was not conducted with an
exhibition in mind and the content is not edited accordingly. Alternatively, digital
resources are used to provide public access to archives, ostensibly to provide
multiple layers of interpretation within an exhibition, where visitors are left to browse
large archives of images or documents relating to the subject matter on display.
With all of these approaches, I question how much consideration has been given to
the effectiveness of presenting content in this way in the museum setting. We know
that visitors have different learning styles even within generic audience types and
there is much theory developed as to how to write text for these groups. We know
through evaluation and tracking the length of time visitors will spend with a digital
display and how they will spend it, but often little adjustment is made to the type of
content available through digital means. Museums could learn from the many
examples in other fields of how to use digital media effectively.
Film is used everywhere to deliver enjoyment, nuance, emotion and an appreciation
of a deeper context. Film, sound and the networked browser are among the greatest
communication devices we have. Yet too often museums use them for the purpose
of passively passing on information and, in the case of the browser, presenting
archived data, rather than drawing upon the rich potential inherent in these media.
Concept Testing
At Riverside, we want to use technology alongside all other forms of interpretation as
one of the range of options we have to tell our stories and communicate our key
messages. This means that the digital content being developed will need to be as
effective as possible at communicating the narrative of the displays, the ideas behind
our stories and the objects on display. All of the digital display options detailed above
are available for us to use to interpret our collections, and the differing tone, style
and treatment of the content will ensure its effectiveness in delivering its message.
It is fair to say that in recent years museums have become far more successful in
using technology. Things don’t break down, and there are many good practice
guides to ensure that the onscreen content avoids the worst presentation mistakes of
the past. An appreciation of the need for testing and how it should be used has been
fundamental in this improvement. While such work is essential, and will form part of
the development process at Riverside, the focus in this type of testing is on the
hardware itself and on software navigation. Despite the proliferation of digital
material in museums today, there seems to be remarkably little research published
on how to create content focused on a specific audience, and on how to
communicate effectively using sound, film and IT screens.
There is often little focus on the interpretive approach, the content creation, the
editing, the emotions created in the visitor through the content, or on how they
Riverside Concept Testing – Richard Williams05.05.09
respond to that content. It is this gap we are seeking to address in the last twelve
months in the Riverside project.
The concept testing we have undertaken has been an opportunity to explore some of
the ideas underpinning the interactives proposed for the Riverside Museum. As I
stated above, from an early stage each story display currently being designed for the
new museum has an established key message, a defined set of GLOs, and a clear
story narrative. Next, a target audience and a chosen interpretive method were
defined for each story. However, while we had defined what interpretive approach
was being taken, within that there was a great deal of scope to refine the approach
the interactive would take and the manner in which the content was delivered.
Essentially, we know what we are saying and why we are saying it but how we say it
can still be explored with the test audiences.
We have been developing the concept testing with a selection of our stories in mind,
and working with our outside contacts to explore some of the specific concepts
underpinning them. Sometimes this has been to gain a better understanding of an
approach we will take for one specific story, while on other occasions we have
undertaken the testing in order to allow us to feed our findings into a number of
developing stories.
The concept testing has taken two paths. First has been an exploration of
methodologies and philosophies from other disciplines into how they prepare
content, edit and deliver it for particular audiences. This has involved looking to both
education experts for a better understanding of their approaches to delivering similar
types of content within their own fields, and also to documentary, film, TV and radio
producers for how they deliver information and edit for particular audiences.
The second path the concept testing has taken has been to work directly with
representatives of the target audiences themselves, and to explore their reaction to
the type of presentations proposed. We have focussed this audience research on
families, teenagers and schools. We are keen that all of the groups we are working
with offer us ideas from their own frame of reference rather than attempting a
solution they think would work in a museum. The difficulty if they do offer ‘museum’
solutions is that their ideas would be compromised by how they happen to
understand museums and our needs. Instead, by observing solutions from the
environments the groups do understand, we can look at the principles underpinning
the approaches taken and not extraneous and irrelevant detail.
Game Play
One significant set of concept tests we have developed has been as part of our
game-play development. We have established a number of interactives in the new
museum where four players will be able to play a game themed on a story display
against each other and the computer. We are aware of the effectiveness in
museums of game-play, particularly for such audiences as schools and families and
Riverside Concept Testing – Richard Williams05.05.09
under fives. Game play sets up a competitive environment in which players are
motivated to understand the story being presented, and the possibility for success
and failure built into the experience offers a heightened sense of achievement. This
is a developing field in museums, and the interest has been mirrored in the school
environment here in Scotland where game-based activities can create an effective
context for learning.
The first step for us has been to gain a deeper understanding of the pedagogical
theory behind game-play within schools. Experts from the field within Learning and
Teaching Scotland (LTS) have highlighted for us the experiences and results gained
using digital gaming tools within classrooms. This has enabled us to get a better
sense of the ideas behind this development and to see how the games can build a
sense of collaboration and context in the classroom. Within this new context,
learning can become less formal and a greater opportunity is created for personal
development within a peer structure which is much more collegiate than a normal
classroom environment.
The concept testing has helped refine our approaches to the game-based
interpretation within the museum. We have looked in detail at two of our planned
stories, known by their working titles of ‘Clippers’ and the ‘the UCS work-in’. These
will both work as four player games for the visitors. With both games, we have
worked with the representatives of the audience group to develop ideas for how the
games will look inside the museum. We have presented the key messages, GLOs
and the central historical narrative surrounding the story as a set of constraints on
the story and have worked to develop ideas within this context.
Upper Clyde Shipyard
‘The UCS work-in’ is the story of how the Upper Clyde Shipyard workers staged a
work-in in 1971-2 when faced with devastating job cuts. It is one of the most famous
episodes in the history of British industrial relations, where workers from different
yards united in their thousands to protect their livelihoods. This was a radically
different form of protest by the leadership where the expected policy of strike action
was avoided and instead the yards stayed open and producing under their
leadership. Eventually, the government was forced to abandon its plans for yard
closures.
The story of the action offers a wide range of conflicting perspectives, from the union
leadership, to the government, industry leaders, the media, the workers and their
families. It is also an episode of negotiation and compromise, whether it was
between the government and the unions, or the workers and their families working to
support them and ensure the success of the action. It is these stories and the
decisions taken by the protagonists during the course of the dispute that the
museum is seeking to tell.
Riverside Concept Testing – Richard Williams05.05.09
The game-play, set up for four players, and designed in a semi-circle around the
objects, encourages debate and confrontation between the players. Each player will
represent a protagonist of the action; take decisions for them during the course of
the game, and attempt to navigate them and by extension the other players towards
the eventual, historic outcome of the work-in itself. For our testing sessions we
worked with the Scottish Teacher Training students at Strathclyde University, who as
part of their training explore creativity within the classroom across the primary school
curriculum. We provided the details of the story, and the set of constraints the
students would work within, and asked them to develop a classroom activity which
would deliver the learning objectives of our story.
With the student teachers we were keen to see the approaches taken in developing
the classroom activity. We were conscious of the steep learning curve for any gameplayer coming to this game fresh and having to understand the goals and values of
the protagonist they were to represent and how they would represent that to the rest
of the game-players. We were also keen to see how the decision-making process
was approached to move the action forward towards a goal that is known at the
outset. Finally, we were keen to see how the teachers would engage their children in
building empathy with the protagonists and communicating their needs to one
another in the game.
With both Learning Teaching Scotland’s experts and the Strathclyde student
teachers it became clear that building empathy and encouraging progress through
communication and cooperation was a key to driving the game forward. Each talked
of solutions within their own teaching experiences which created an environment in
which the children found their own level of understanding of a protagonist which they
were able to then communicate to the wider group, both verbally and through their
actions and feedback in the context of the classroom activity. This has confirmed our
thinking of the importance of this element, and highlighted new ways to encourage
this communication. We will likely emphasise it more than we had planned, and we
may also change the approach to the game in our choices of protagonists and the
game environment and the interface the visitors will play within. Discussions we have
had during testing about the issues themselves - industrial relations issues raised by
the UCS work-in - have helped us think through ways to present these to the players
in the game environment. In the coming months, the student teachers return to their
classrooms and the intention for some is to run their game as a classroom activity in
which we will be able to view the reaction of the children to the content.
Clippers
Our second game-play tested thus far has been ‘Clippers’. This display looks at the
story of the clippers built on the Clyde which were engaged in the tea trade from
China, racing each other back to Britain with the earliest pick of the tea crop. Within
the game-play in the museum, each player will have to get their clipper back first to
London, taking many decisions before and during sailing that will affect the speed
Riverside Concept Testing – Richard Williams05.05.09
with which they cross the globe and whether they will finish first. Events, some as a
result of the decisions the player has taken, and others unforeseen, will occur during
the voyage back. All of these occurrences and the choices and decisions open to the
game-players are taken from the well-documented historical archives of the clippers’
story.
This testing has followed a similar form to that of the UCS work-in story. We have
presented the constraints for the story and allowed the user group to develop a
game for their peers within that framework. In this case, the audience group has
been our Junior Panel, one of a number of audience consultancy groups we are
working with in developing the museum. The Panel is made up of 16 teenagers from
a variety of backgrounds and interest levels in museums. We split the groups into
two, and while one of the groups worked with the concepts underpinning another of
our stories, the second group designed and built a paper-based and simple digital
game with the intention that it would be played by the first group and their friends.
In creating this game we have established the type of decisions the players will need
to take, such as hull design (speed vs. risk), cargo size, crew size, route taken and
so on, and explained the inter-relationship of all the decisions taken during the game.
The teenagers have created a game in which they will establish how the choices are
presented to the game-players and how they are expressed. Priorities have varied,
as has the way the choices and consequences are represented and explained. The
outcome has been an opportunity for us to gain a better understanding of how the
audience perceives the choices, what values and context they bring to the game,
and what parallels they make with their own context. While some of the game-play is
inevitably impractical for direct comparison with our eventual game within the
museum, seeing the teenagers describe the choices and rationale has been
invaluable. Next month we will run a second session, in which the second group will
play the first group’s game, after which we will explore the game-play and everyone’s
reaction to it.
The concept testing for both the UCS work-in and Clippers has been to explore the
audience’s reaction to a specific piece of interpretation. However, other testing has
allowed us to investigate the audience groups’ reactions to more fundamental
display philosophies. This can be demonstrated through the work we have done with
soundtracks within the museum environment.
Sound
Museums often use sound as part of their displays, either as a third person,
curatorial voice providing a commentary or explanation, or as an ambient
‘soundscape’ with sounds apparently evocative of the display being depicted. Sound
can provide a deeper sense of narrative as a result, and so can be effective in
providing explanation and allowing a visitor to construct a deeper meaning from a
display. Through our concept testing, we were keen to gain a better understanding of
Riverside Concept Testing – Richard Williams05.05.09
how visitors react to sound and what type of approach allows us to deliver most
information to the visitor in a meaningful and effective way.
The concept testing of sound was conducted using a number of volunteer families
over the course of 2008. We found them through advertisements posted within the
existing Museum of Transport and Kelvingrove Museum. Each family worked with us
for two 2 hour sessions each, first in Kelvingrove Museum, and then back at the
Museum of Transport. In the first meeting we showed them a couple of the story
displays at Kelvingrove which used predominantly sound to deliver the content and
then followed this with some formal evaluation through questioning and a discussion.
These focussed on their reaction to the sounds and what information they took from
them. We also found that depending on the content, visitors often underestimated by
as much as two thirds the running length of an audio piece.
The second session used one of our planned displays, known as ‘Rowing Ashore’,
as a test-case. This story is about the rowing boat (which will be on display) from the
steam-ship the SS Dunara Castle, which for many years was the only link to the rest
of the world for the inhabitants of the remote islands of St Kilda. Everything landed
on the island, whether it was people, mail, provisions or livestock had to be ferried
either in the displayed rowing boat or its twin. Rowing Ashore will tell this story and
ask the visitor to reflect on what has been carried in the rowing boat, why it was
travelling and what it tells us of the people and their existence played out so far from
the rest of civilisation.
The planned display will be a soundscape around the rowing boat which will explain
the story through the archive testimonies of the islanders and their visitors, spoken
by actors and supplemented by sound effects. With our volunteer families, we played
three different soundtracks created especially for the testing and then followed this
with questions and a discussion. The three soundtracks each told the story of the
rowing boat, first through purely sound effects, secondly through voiced testimonies
and finally through a curatorial narrative.
In conducting the study we wanted a better understanding of how people react to
sound in museums, what their expectations and understandings are of it, and how
we can improve its success. Secondly, we were interested in exploring the extent to
which the visitor comes pre-conditioned to a museum display, expecting a didactic,
curatorial narrative, and whether they might accept a more open-ended, emotional
performance piece in which the key message and GLOs are implicit and the
meaning is aimed at a deeper level. Finally, we wanted to discuss how families
worked together to make sense of the sounds, and how supporting information in the
display could aid in this dynamic.
The conclusions to the testing provided us with some clear methodologies for
developing our sound-based interpretations. As we had suspected, the length of the
piece was contingent on the content. At the start of the Kelvingrove project, audience
Riverside Concept Testing – Richard Williams05.05.09
and interpretation research gathered evidence from other museums and completed
new research and identified 40 seconds as the average length of time people would
listen to a piece of AV or audio. This was written into our display standards for use
except in certain circumstances. However, we are reviewing this again to further
refine it in light of both Kelvingrove and Riverside evaluation. In both projects we
found that this average went up considerably depending on the style and type of
content. This was an assumption we had made while developing some of the soundbased displays at Kelvingrove, but now we found an explanation and rationale for
why it should be the case. Provided with a firm ‘baseline’ of information about what
they were listening to and a framework of how to explore the exhibit, the visitors
were happy to engage with the sounds and to use their imaginations to discuss ideas
within their own groups.
We found that our families were prepared to use sound intermittently, ‘switching off’
and studying the objects and then coming back to listen to more of the sounds. We
discussed with them including sounds within the larger piece that would facilitate
this, ‘signposts’ that will allow them the opportunity to re-engage with the
soundscape again, and a tighter and more circular narrative in which we can
emphasise and re-emphasise particular points. As we had expected, none of the
sound tracks we tested was entirely suitable in its own right (we had isolated them as
solely sound effects, narrative , or voiced testimony to focus on each one’s attributes
in turn), but each contributed something to the overall picture being built in visitors’
minds. Interestingly, once again the visitors proved unable to accurately estimate the
running length of these pieces or even to estimate the times relative to each other.
Challenges
Central to all of the concept testing we are conducting has been the Riverside team
itself. The eventual content developers – myself and my team of ICT curators, the
senior curator for the Riverside project, and the story curators - are all involved in the
process, alongside the Visitor Studies team. The roles and knowledge the staff holds
is important, but equally so has been the dynamic and flexible approach they have
brought to meetings, and their willingness to think of interpretative approaches in
new ways. All of the team has played its part in facilitating the meetings, building
relationships over time with our volunteers and sometimes teasing out the
discussions when going has been slow.
This personal element has been very important in helping overcome some of the
difficulties we have encountered with the testing. The volunteers have been aware of
the prestigious project of the Riverside Museum with which they are involved, and
this has sometimes meant they have been more reserved and conservative than
normal. Volunteers and visitors often project their notions of what they believe to be
acceptable in museums, or their notions of what museums are, onto the work we are
discussing; their inclination is to say what they think we want to hear. Both parents
and teachers sometimes steer their children in a particular way or towards a
Riverside Concept Testing – Richard Williams05.05.09
particular response. We have filmed many of the sessions and this has helped
mitigate these problems, as it allows for further study of the sessions, particularly of
non-verbal responses which might not be apparent at the time.
It has proved surprisingly difficult to move the concept discussions away from a
museum-based solution. This has been the case particularly with the experts who
want to comment on concrete examples and specific sets of facts delivered in a
specific way, rather than on our central question: ‘How should this content be
presented?’ Most prototyping, even formative evaluation, works with existing content
and is about the refinement of the chosen approach and the draft content. The aim of
our testing has been to firm up the concepts underpinning the approach. It has
proved to be our biggest challenge to establish this shift of thinking in-house and
then get it across to our volunteers.
The testing will continue for the next few months as we develop the content for our
interactives. By the end of 2009 the Riverside team will be working with software
developers to produce the interactives for display and so we will be evaluating the
content as it develops. Preparing the content for the concept testing, pulling together
the materials, running the sessions, and analysing the results are all timeconsuming. This is particularly the case within the constraints of a major museum
project where research, content development and design development all constant
demands on the team. However the deeper understanding we now have of the
interpretative approaches we are taking has already proved to be very beneficial in
the content development of the Riverside interactives.
Conclusion
The Concept testing has been very useful in determining how different visitor groups
react to digital content in museums and how they perceive the concepts and content
presented to them. Our work with experts from other fields has led to a deeper
understanding of how they communicate and teach ideas within their specialism,
which has helped us to develop imaginative ways of presenting the material in a
museum environment.
This type of research is difficult to timetable into a digital interactive’s development.
Prototyping and formative evaluation identifies practical problems with the design,
and summative evaluation focuses on what worked or reasons why the interactive
was unable to deliver the intended messages. Time is rarely available to explore the
methodologies of using digital media and how the public perceive and use such
technology. The concept testing we have undertaken is an attempt to address this
shortfall in the context of a live project.
The opportunity to work with representatives of the target audiences is critical to the
success of the interactive development. This might seem an obvious point, and this
should be an intuitive part of any development process. However, this is not always
the case and our work in part reflects the paucity of research into digital delivery, in
Riverside Concept Testing – Richard Williams05.05.09
contrast to the abundance of research on text. Eventually, all methods of
communication with the visitor, whether object groupings, text panels, manual
interactives or digital technology, should be thought of as parts of the same process
in developing display content. The outcome of such an approach would be that
interactives in museums would be selected for their ability to communicate meaning
about the objects on display, rather than digital technology being used for its own
sake.
Riverside Concept Testing – Richard Williams05.05.09