Northern Ireland Museums Council
Learning in Museums - Evaluation
Introduction
Project evaluation is a constant within all aspects of museum management. In general terms
it usually involves an assessment of whether or not the original objectives and outcomes
were achieved upon a programme’s completion, an assessment of the delivery process,
costs and non-monetary aspects, and an assessment of the risks (and the effectiveness of
any mitigation) and the lessons learnt. Against this generic background, the evaluation of
museum learning programmes requires some especial considerations, and these are the
focus of this paper.
Learning and Education
Learning is at the heart of museum activity, whether this takes the form of an exhibition, talk
or lecture, school programme, adult class or an independent visit by a member of the public.
The process of engagement and interaction with objects, artefacts and interpretative text
produces a reaction, creates an impact, and stimulates thoughts, discussion or activity, all of
which is essentially part of the learning process. Learning goes well beyond the school
group, while pupils are certainly potential learners in the museum environment, so too are
the general visitor, family group or adult community group member. Learning includes but is
also distinct from education; the former is arguably more about the process and the impact
of engagement, while the latter concerns the imparting and absorption of new knowledge
and information. Learning can include the development of skills, and can be viewed as a
process provoking thoughts, actions and reactions. Therefore, a general visitor who visits an
exhibition and expresses an opinion in response to what they have seen or read is directly
involved in the museum learning process. It is an individuated process, so that each person
will take away different experiences and conclusions. By contrast, education tends to focus
on the collective; with providing a more standardised experience by way of provoking a more
common experience for those involved. Museum professionals know, recognise and support
this view of learning and ascribe to the idea of learning being at the heart of museum activity
but the question of evidencing and capturing what types of learning, the extent and range is
often a more complex task. In this regard it is useful to consider the following questions:
1.
2.
3.
4.
What is intended to be the impact or outcome of the activity?
Is what was initially envisaged actually happening and is this the same (or not) of all
users?
Are there any new impacts or surprising outcomes arising from the process?
What can we, in the museum, learn for the future?
What is Evaluation?
Evaluation is undertaken to explore or prove the extent to which what you set out to do has
been achieved and to examine what forms or styles of learning have taken place. Evaluation
measures the degree of achievement in relation to the aim, objectives and intended
outcomes of an activity or action.
Formative Evaluation is used to test how programmes and activities can be modified to
best meet the aims and objectives. Such evaluation is used for pilots and to monitor existing
programmes.
Summative Evaluation is used at the end of a project and findings are used to consider
whether it ‘worked’ and explore the potential of repeating such work.
Formative evaluation may seek to answer questions such as: Is it working as intended? Are
we achieving what we set out to do, and if so how and why/why not? Can we modify the
activity to improve delivery and closer meet the objectives?
Summative evaluation questions focus more on final outcomes: Did it work as intended? Did
we achieve what we set out to do? What lessons or action points have been learnt for the
future?
If the evaluation reveals a gap between intentions and reality then reasons for this can be
explored to inform future planning.
Purposes and Benefits of Evaluation
As indicated above, there are various reasons why a museum might evaluate its
programmes and activities. These will include:
To assist with future planning. Here the feedback received can be used to assess the degree
progress made against the programme outcomes, or to ascertain if the objectives and needs
of visitors and users are being met. Consequently, revisions and alterations can be made
and resources tailored to meet the intention. This approach is concerned with constant
review and adjustment by way of continuous self-improvement.
To meet the requirements of funders. Evaluation is an essential aspect of any funded
project. Sometimes the associated processes are defined within the funding letter of offer,
but on other occasions there is an opportunity to design the process in conjunction with the
funder as part of the development or planning stage.
As a marketing tool. Positive endorsements and feedback from one group can be used
generically; to encourage other groups to participate, for advocacy, for public relations and
such like. The capture of such feedback needs to be incorporated into the process (whether
on a day-to-day, or project-by-project basis), and can best be achieved if a particular staff
member is assigned to collect and collate it.
Learning evaluation
One of the more effective and efficient techniques for evaluating learning and education
programmes remains the Inspiring Learning for All (ILFA) methodology. Details of this
approach are available at http://www.inspiringlearningforall.gov.uk/
This framework provides the basis and tools necessary for achieving best practice in
supporting learning and assisting you to identify how your museum supports learning. It
examines impacts through generic learning and social outcomes, the former being the
particular relevance to the subject of this paper.
Generic Learning Outcomes (GLOs) are seen as comprising five categories, referred to as
 Knowledge and Understanding (K&U)
 Skills (S)



Attitudes and Values (A&V)
Activity, Behaviour and Progression (AB&P)
Enjoyment, Inspiration and Creativity (EI&C)
These categories can be viewed as neatly summarising the main forms of learning and as
such offer a framework for interpreting feedback. Within the framework, taking time to think
through and structure an activity is stressed as this is likely to result in the activity being
more focused and the outcomes are more easily measured. It proposes a sequential
approach to the design, delivery, evaluation and envisaging the ’success’ of any project,
summarised as;
Aim
Objectives
Intended Learning Outcomes
Activities
Evaluation
The findings of the evaluation are then used in a ‘feedback loop’ to inform the aims,
objectives and outcomes of future projects, and for revising or adapting of current activities.
Evaluation, therefore, forms an essential part of an on-going cycle and is a dynamic process
towards service improvement.
The Inspiring Learning for All framework also allows museums to measure the Generic
Social Outcomes of their work, particularly across the three areas of
 Stronger and safer communities
 Health and well-being
 Strengthening public life
While these are not directly relevant to the evaluation of learning, they do have a bearing on
the evaluation of the outcome broader learning activity in museums, especially with people
and communities outside the formal education sector.
Gathering feedback
The is a range techniques used to garner feedback, including e-surveys, focus groups and
comment cards, but questionnaires are the most widely used method for collecting data and
opinions. The design of questionnaires associated with learning programmes flows from the
establishment of the intended learning outcomes, as is demonstrated by the ‘worked
examples’ which follow.
It is recommended that the questions posed should be a mix of both ‘open’ and ‘closed’
questions as such a combination allows a degree of quantitative and qualitative analysis.
For examples ‘closed’ questions might include - Did the workshop assist to develop
participants’ knowledge of local history? (the options being - Yes/ No/ Undecided); My pupils
had the opportunity to develop communication skills. (Yes/ No/ Undecided); Was the
experience was enjoyable? (Yes/ No/ Undecided).
Whereas ‘open’ questions may be along the lines of - In what ways did the workshop assist
in developing your participants’ knowledge of local history?; Did the workshop foster
communication skills? Please provide detail.; Please describe the experience of participating
in the activity.
Experience suggests that respondents feel time-pressured when asked to participate in
evaluation processes and thus there are benefits to keeping questionnaires relatively short,
possibly one side of A4. Consideration should be given to capturing contact details and
gaining agreement for you to follow up the process, say on the telephone, at a later time.
While there is the temptation to use to opportunity to gather an abundance of information,
remember that you will have to process it, which can take up considerable time (and possibly
money), so being exact in knowing what it is you wish to find out and precise in the design of
the questionnaire will save time and effort. Both are linked to the ILFA evaluation loop
outlined above.
Analysing Feedback
The use of ‘closed’ questions, and indeed ‘tick box’ opinion responses (see worked
examples below) enables quantitative information to be generated, so that numbers and %
can be used to illustrate the extent objectives and intended outcomes have been achieved.
The answers to ‘open’ questions allow for some qualitative and arguably deeper analysis. As
in the examples that follow, it is useful to incorporate questions that elicit responses around
progression, behaviour, the quality of the experience, and recommendations on changes
that could be made. While the responses received can be collated as raw data, thus
reflecting the breadth (or otherwise) of opinions, there may be benefit in extending the
analysis by coding responses around the five ILFA areas. So, for example,
5.
How would you describe the experience of the workshop?
It was very good and I made some new friends. (coded to AB&P)
I learnt new skills. (S)
I surprised myself as I didn’t think I could sew. (A&V)
It was fun. (EI&C)
While there is a degree of subjectivity in the ascription of such coding, it does allow
qualitative responses to be quantified for the purposes of reportage.
Timing of evaluation
There are various options around when to gather evaluation feedback, each with appreciable
pros and cons. Collecting information during or straight after an event means that things are
still fresh in the minds of participants and responses will be more immediate and possibly
instinctual. Conversely, asking people their views and opinions later on provides more
reflective responses, and possibly a more considered appraisal. Obviously there are no hard
and fast rules, and the ideal would be a combination of both approaches.
The scale, complexity and period of an event or programme will have a bearing on the timing
of evaluation. For instance, more immediate responses and feedback are required if the
activity is some form of ‘pilot’, designed to test an approach as part of the planning of future
programmes, whereas an ongoing evaluation process may be used for permanent
exhibitions, phased or long-term projects. Consideration needs to be given to repeat
evaluations during the course of a programme in order to take account of seasonal
fluctuations (the prevalence of tourists, for instance, school holidays or the cultural inclination
to embark on self-improvement during the autumn months) and other unique and influential
factors. A continuous approach is perhaps the most comprehensive but it does produce a
large volume of material that needs to be processed, which again raises the issues of
resource and capacity.
Who’s opinion?
When seeking feedback from school groups a combination of both teacher and pupil
feedback possibly provides a more rounded perspective on the session. It is often, however,
easier and more practical to ask for teacher feedback only.
Pupil questions need to be age and ability appropriate and free from any pressure to provide
the ‘right’ or ‘correct’ answer. Simple questions, such as - What did you enjoy? What was
your favourite activity? What word(s) would you use to describe the workshop? What object
was most interesting to you and why? or Draw a picture to describe your visit – can be most
useful in this regard. It may be cumbersome to seek general pupil feedback after every
session and thus a sampling approach could prove more workable. Again the issue of
whether this occurs during the visit or upon return to school has associated pros and cons.
Teacher feedback is useful as it brings their professional judgement to bear, which is
especially useful around curriculum links and how the activities worked for the range of
abilities and needs within their class. Ultimately it is they who are the motivators as to
whether a future visit takes place, which can have an impact on the sustainability or content
of a particular programme. In view of this, consideration should be given to exploring the
Objectives and Intended Learning Outcomes with teachers when planning school
programmes and at the outset of each session.
For other learners, whether individuals or groups, the aim too should be to gain a
comprehensive overview of opinions and perspectives and thus ideally all workshop or
activity participants should be given the opportunity to provide feedback. However, within
this sector consideration may need to be given to amassing feedback that is also
representative of socio-economic or demographic groupings.
Word into Actions
As stated earlier, evaluation is a dynamic process geared towards service improvement. It
should be an ascribed duty within the delivery process with a specific person designated to
see it through and to report on the outcomes. More importantly when reviewing the feedback
received it is worth asking such questions as – What are the practical implications of these
responses (regardless of them being supportive or critical)? What adjustments can/should
we now undertake? Who should we tell about the evaluation outcomes and the course of
action taken?
Adjustments could be deemed immediate and unstrategic, such as the rearrangement of
furniture, changing the environment (heat and light) or the availability of supporting
information.
Other responses may require a more strategic consideration, may involve a cost or some
other significant change in how the museum delivers a service. For instance, changing the
time or location of a learning session, providing more support staff, altering the interpretation
of an exhibition, making adjustments by way of enhancing health and safety and
safeguarding, or producing further learning support materials.
Whatever the practical consequence or outcome of actions taken in response to user
feedback it is important that these are communicated to three particular groups of people;
the individuals who took part in the evaluation process, the governing body of the museum
and the museum staff. In so doing the process is and is seen to be an active, vital and
essential aspect of a museum’s day-to-day operation.
Additional information and guidance
Across the web there are myriad sets of guidance and templates for project evaluation,
many of which provide valuable lessons for museums even though their focus may be on
learning across other disciplines.
Notable museum-based resources include;
http://www.inspiringlearningforall.gov.uk/
http://www.hlf.org.uk/HowToApply/goodpractice/Pages/Evaluation_guidance.aspx#.U_73U6
Qg-po
http://sharemuseumseast.org.uk/shares/resource_34.pdf
http://www.museumsgalleriesscotland.org.uk/research-and-resources/research-andevaluation/how-to-carry-out-evaluations/
http://abcofworkingwithschools.org.uk/getting-started/setting-up-a-service-forschools/meeting-school-needs/evaluating-your-service/how-to-evaluate-with-schools-2/
http://www.ucl.ac.uk/museums/research/touch/museumwellbeingmeasures/wellbeingmeasures/UCL_Museum_Wellbeing_Measures_Toolkit_Sept2013.pdf
Worked Evaluation Examples
Example 1
Developing and evaluating a museum learning workshop to support the curriculum
requirements of foundation stage pupils through the use of the toy collection.
The Planning Stage
Assessment using the five ILFA categories
 Knowledge & Understanding (K&U) - Of the toy collection, the materials the toys
are made from and how the toy works.
 Skills (S) - Listening and communication skills. Thinking skills. Fine motor skills.
 Attitudes and Values (A&V) - Understanding the need to care for and handle old
toys carefully.
 Activity, Behaviour and Progression (AB&P) - Use the workshop information for
follow-on activities back at school.
 Enjoyment, Inspiration and Creativity (EI&C) - Enjoy the experience. Make a peg
doll.
Making Curriculum Links
Areas of learning
World Around Us
 Change Over Time - toys now and then, similarities and differences.
 Movement and energy - how the toys works and move
 Understand the properties of materials – making the object
Language and Literacy
 Participating in whole, paired and group work.
Skills
 Communication (listening and working in groups)
 Creativity (craft activity)
 Thinking skills (questioning, decision-making and games).
Learning Experiences
 Enquiry based active and hands-on, variety to suit learning style in a supportive
environment.
Aim
To introduce toys and games from the past and compare and contrast with toys of today.
Objectives
Through participation in the session, pupils will learn how to;
 categorise a range of toys into old and new,
 consider the materials used to make the toys,
 explore how the toys work,
 learn how the teddy bear got its name.
 consider toys and games in the lives of children in the past and compare with their
own lives today.
 participate in a structured craft activity,
 participate in a structured play-based activity.
Intended Learning Outcomes: Combining Curriculum and ILFA:
Through participation in the session, pupils will have:
 Worked together in groups to sort toys into old and new and explain the reasons for
their choices (K&U and S)
 Participated in a discussion about today’s toys (S)
 Named 3 types of material used to make the traditional toys (K&U)
 Grouped toys into categories based on what the toys are made from (K&U, S)
 Discussed and compared the properties of the 3 materials (K&U, S)
 Considered how the toys in the past worked, comparing the sources of energy used
with toys today (K&U, S)
 Ordered a selection of teddy bears from oldest to newest and explain reasoning for
their choice (S)
 Made a peg doll to take home (IE&C)
 Participated in a play activity in groups to explore how the traditional toys worked and
moved (AB&P and IE&C)
 Enjoyed the session (IE&C)
 Provide suggestions on follow-on activities (AB&P)
Sample Evaluation Form
For completion by the teacher or group facilitator.
1.
To what extent do you agree with the following statements (please tick)?
Your pupils…
Strongly
Agree
Agree
Not Sure
Disagree
Strongly
Disagree
Found out more about the
materials used to make
traditional toys
Discovered how traditional
toys worked or moved
Developed group work skills
Developed
enquiry
thinking skills
Developed
skills
and
communication
Enjoyed the activities
2.
Do you intend to build on this workshop back at school and if so in what ways?
3.
What worked the worked best for your pupils? Please provide details.
4.
In what ways could the workshop be improved for the learning needs of your pupils?
Please provide details.
5.
Any other comments.
Example 2
Drawing inspiration from the Museum’s quilt collection to develop an adult craft class.
The Planning Stage
Assessment using the five ILFA categories
 Knowledge & Understanding (K&U) - of the quilt collection, the history of design
and the various techniques used.
 Skills (S) - the application of needlework skills and the hands-on activity of learning
to patchwork.
 Attitudes and Values (A&V) - may reinforce or extend belief in personal abilities.
Change view of what a Museum is and does.
 Activity, Behaviour and Progression (AB&P) - attendance of a series of classes,
producing a finished product, signing up for additional classes, researching and
practising in own time.
 Enjoyment, Inspiration and Creativity (EI&C) - enjoying the experience, sense of
well-being, producing a product.
Writing aims and objectives
Perhaps unlike curriculum based activity, working with older groups allows the opportunity to
involve participants in the planning process. In this regard it may be it is useful to ask what
has motivated people to participate in the museum activity and what they hope to gain from
the experience. Anticipated outcomes, once established, may be kept as separate individual
outcomes or expressed collectively, and evaluated accordingly. The former work well across
large groups of participants and can be evaluated through tailoring the evaluation sheets to
allow for discrete outcomes to be assessed. The latter, the collective outcomes and
evaluation, may be assessed through a generic methodology, as illustrated below, and may
also be extended to include Generic Social Outcomes.
Aim
To use the quilt collection to inspire adult craft workshops.
Objectives
Through engagement with the collection, participants will;
 Discuss the history of quilts,
 Discuss and structure the practise of quilting and patchwork,
 Enhance their own practice of the craft,
 Produce a piece of quilting or patchwork that could be displayed as part of a future
exhibition at the museum.
Intended Learning Outcomes:
Through participation in the activity, people will;
 Gain a knowledge and understanding of the Museum’s quilt collection and the history
of design and technique. (K&U)
 Practise quilting and patchworking skills. (S)
 Gain confidence in their ability to patchwork. (A&V)
 Develop a positive view of the museum’s Collection and the range of museum
activity. (A&V)
 Produce a finished product. (S, EI&C)
 Enjoy the experience. (EI&C)
 Be inspired to pursue further activity. (EI&C, AB&P)
A Sample Evaluation Form
1.
To what extent do you agree with the following statements?
You…
Strongly
Agree
Agree
Not Sure
Disagree
Strongly
Disagree
Understand more about the
quilt collection.
Understand more about of
the history of design and
patchwork techniques.
Gained hands-on experience
of a range of techniques.
Gained confidence in my
ability to patch work.
Gained the skills to produce
a finished product.
Enjoyed the experience.
Have positively changed your
perception about what a
Museum is and does.
2.
Do you intend to take any further action as a result of attending these workshops and
if so, what will you do?
3.
What worked the best/ what did you enjoy or benefit from the most?
4.
In what ways could the experience be improved?
5.
How would you describe the experience of the workshop?
Example 3
An exhibition ‘The Role of your Museum and its Collections’ with a target audience of
families.
The Planning Stage
Assessment using the five ILFA categories
 Knowledge & Understanding (K&U) - Develop a knowledge and understanding of
the role of a local museum, the key aspects of its collection and how these relate and
reveal the histories of the area.
 Skills (S) – the exhibition visitor will try their hand at identifying and documenting an
object
 Attitudes and Values (A&V) – Visitors will value the work that the museum does,
and be willing to communicate the role and value to others
 Activity, Behaviour and Progression (AB&P) - Use the visit for interaction between
family members and interaction with museum staff. Will plan to return to the museum
at another time. workshop information for follow-on activities back at school.
 Enjoyment, Inspiration and Creativity (EI&C) - Enjoy the experience. Bring an
object to the museum for appraisal.
Aim
To present family orientated activities by way of enhancing people’s understanding of what a
museum is for and what it does.
Objectives
Through engagement with the collection, participants will;
 Discuss what the function of a museum is,
 Gain an insight into mow museum staff interpret the collections,
 Appreciate the ways objects are identified and documented,
 Participate with other family members in the activities associated with the exhibition.
Intended Learning Outcomes:
Through participation in the activity, people will;
 Gain a knowledge and understanding of the museum, its collection and local history.
(K&U)
 Practise identification and documentation skills. (S)
 Place an increased value of the role of the museum. (A&V)
 Develop a positive view of the museum’s Collection and the range of museum
activity. (A&V)
 Complete an identification/documentation task. (S, EI&C)
 Enjoy the experience. (EI&C)
 Be inspired to pursue further activity. (EI&C, AB&P)
Evaluation methods
 Given the age range of participants, consideration may be given to employing diverse
evaluation methods, including;
 Questionnaires
 Comment cards
 The hands-on museum staff reporting reactions after sessions completed
 The invitation to draw pictures
 An informal exit survey with refreshments
Possible Evaluation Questions for feedback forms:
 Did you learn anything new?
 How would you describe this exhibition to others?
 Has your view of museums changed and if so in what ways?
 Would you like to know anything more as a result of visiting this exhibition?
 Do you plan to take any action as a result of visiting this exhibition?
 What was your favourite activity and why?
Some possible responses and suggested coding:
 I had fun with my children! (EI&C)
 It was great - I have lots I want to tell my teacher about. (AB&P, EI&C)
 I know more about the local area. (K&U)
 The museum is a welcoming place.(A&V)
 I liked taking a photograph of my favourite object- my photo looked surprisingly good!
(EI&C, A&V, S)
 I want to start collecting objects! (AB&P, EI&C)
 I am a primary school teacher and I plan to bring my class to visit this exhibition- it is
so good. (AB&P)
 I didn’t realise that the museum had such a diverse range of objects it the
collections.(K&U, A&V)
 Wicked (EI&C)
 I can write about an object (S)
 I guessed all 5 of the mystery objects! (S, K&U)