Evaluating Events
MLLSM01
EVENTS POLICY
LECTURE 8
Lecture content
 This lecture will discuss why monitoring and
evaluation are so important to event policy makers
and practitioners
 It will also review aspects of the evaluation process
 It will then reflect upon the principal techniques of
evaluation utilised within the events arena
 Finally, it will consider why net impacts are the most
important facet of evaluating the success or failure of
an event policy intervention
The politics of evaluation
 Moore & Sykes (2000) argue that “monitoring and
evaluation is closely linked with policy development at a
strategic level and when specific projects are being
designed and implemented” (p203)
 Related to the development of event aims and objectives
– often derived from political aims
 So, the “approach to evaluation, the choice of what is
measured and the judgement as to what has been
achieved cannot be divorced from the wider political or
cultural context” (Moore & Sykes, 2000: p203)
 Political actors ‘set the agenda’ (Lukes’ theory of power)
and monitoring and evaluation legitimates it and
produces transparency and accountability
Evaluating Events
 Despite the predominance of economic logic in the evaluation of
event impacts until recently, this is changing as the socio-cultural
and image impacts rise in importance (Richard and Wilson, 2004)
 This does not mean that the economic is now defunct; rather that it
has been augmented by a wider understanding of social, cultural and
environmental impacts – the economic is not enough
 As politics is about the allocation of scare resources, investment in an
events-led urban strategy needs to be justified/legitimated on the
basis of objective outcome and output measures – does it work (e.g.
Commonwealth Games)
 Especially as we see “significant public sector expenditure to develop,
underwrite and promote event-based tourism development” (Connell
& Page, 2005: p64):

LA’s and LECs provide seed funding to encourage events growth and need to be
satisfied of a ROI – EventScotland also invests in international/regional and local events
on the basis of a return – tourist visits, regional identity, local importance.
The purpose of evaluation
 To judge whether there is a rationale for policy
intervention/are interventions producing the desired
outcomes (Moore & Spires, 2000). But also to:



Monitor progress of projects against target – has hosting the Special
Olympics increased overseas visitation to Glasgow/Has the
investment in EventScotland been worth it
Review and revision of strategy in light of new
information/challenges – policy adjustment (e.g. new approach to
extending visitor stay required)
Overall judgement on success or failure of strategy/policy –
performance related and identifying causes of failure
 Evaluation in both public and private sector event
contexts now ever more connected to KPIs – numbers,
type of visitors, media profile, reduced reliance on public
purse
Internal and External Evaluation
 Internal (organisational):
 Team analysis and performance review
 What
worked, what can be improved upon, which systems
need to be revised, which org structure is most effective

Development of KPIs to quantify objective performance of
team:
 Customer
complaints, resident complaints, deadlines met,
media exposure
 External (environmental/macro factors):
 Reinforcing customer-focus
 Seeking quantitative/qualitative responses on the
customer ‘experience’
 Visitor numbers, impression management
When to evaluate
 Three principal windows of opportunity for
evaluating events:
 Pre-event:

Expectations, economics, ownership, ticket purchases
 During event:
 Reliant on dynamic methodologies and support of
organisers/funding partners
 Post-event:
 Legacy (traffic, housing, pollution, community pride)
 Online presence, building relationship databases
Methods of event evaluation
 Quantitative measures:
 Visitor numbers (how many, where from: pre-booked tickets, pay
as you go)
 Spend per head (how much spent and on what element of the
event) – economic additionalities
 Customer surveys, business surveys (Connell & Page, 2005),
online mechanisms
 Qualitative measures:
 How did customers find out about the event (Advert, flyer, radio,
press, word of mouth)
 Expectations and satisfaction (rate satisfaction with event cost,
value, product, impressions of destination)
 Visual methods (video, camera), interviews, workshops, focus
groups
 New tools: online digital media can now facilitate
customer/visitor interaction and ongoing evaluation
Evaluating the social, cultural & environmental
 Waitts (1999; 2003) one of the first to consider socio-
cultural dimensions of events:

He concluded that Sydney 2000 had brought about a shift from
welfare to entrepreneurial goals; a functional transformation of the
cityscapes to generate cultural capital and, the growing importance of
media images
 However, politically, the economic still garners more
support
 But, social models are being developed and cultural
capital can be as much about image, perception and
impressions as about immediate economic return
 The environmental is increasingly important as
sustainable events become the target of organisers
Evaluating Images
 Hiller (1989;1998) and Ritchie & Smith (1991) the first to seriously
consider the image effects of major events
 Now, as Hall (1992: 155) remarks, “hallmark events may be the
regarded as the image makers of modern tourism”
 Started out with narrow definition of ‘image’ and one-off studies –
failing to consider the multi-dimensional aspects of image and the
benefits of longitudinal studies
 Richard & Wilson (2004) used designative (informational) and
appraisive (feelings, values and meanings) elements of images to
evaluate the impact of European Cultural Capital on Rotterdam:


Considered residents and tourists and cultural sector stakeholders
Focused on images attributes including: modern architecture
(designative), water (designative), multicultural, working city,
international, dynamic, culture and art, lots to discover, events
(designative), shopping (designative), nightlife (designative), cozy, unsafe
From mega to community evaluation
 Evaluation strategies need to be specific to the nature and scope of




the event and to be able to identifying potential spatial inequity
(Connell & Page, 2005)
In justifying the ‘benefits’ of attracting a large scale event to your
city/country, organisers are expected to measure ROI, additional
tourist spend, increased dwell time, infrastructural legacy etc.
In contrast, the success (or failure) of a local community events
strategy will be justified on the basis of building social capital
(networks, sense of belonging), pride in place, sense of community,
reductions in crime, quality of life
The challenge for policy makers is that benefits of the mega/hallmark
event are easier to document (economic impact assessments) than
the longer term and more subjective social and cultural indicators
Policy makers need to develop social impact tools to make the
intangible tangible – only then will the social and cultural be taken
seriously
Gross and net impacts
 Difference between what would have happened had the policy
intervention not taken place and what additional benefits have
accrued because of an intervention
 Links back to Carlsen’s (2004) arguments about multiplier effect –
what added value does winning the rights to host the UEFA Cup
Final have on a city – that wouldn’t have accrued anyway
 To reach net additional impact, value has to be assigned to:



Substitution – policy targeted investment takes the place of other investments
Displacement – additional activity from some sectors counterbalanced by less output
from others (tourism displacement as a result of Olympic Games hosting)
Indirect impacts – additional spend in the local economy as a result of increased
business for some firms
 The political issue is whether public sector investment leverages
additional benefits and provides a healthy ROI for public
stakeholders (e.g. London 2012 regeneration)
Strategic Objectives
ECONOMIC
PHYSICAL
Tourism
employment
Environment/
infrastructure
SOCIAL
QOL, community
cohesion
Inputs (expenditure)
Scot Exec funds
Other public sector
Private sector
Activity measures
Business ass
Training
Neighbourhood
renewal
Participation
in arts
Reduction in
crime
Output and Outcome measures
Start –ups
Event jobs
Ext of cultural
activity
No of
performances
Gross impacts
Adapted from PACEC
(1999)
Gross additionality – displacement plus
linkages/multipliers = Net Impacts
No of
volunteers
References
 Carlsen, J, Getz, D & Soutar, G (2000) ‘Event Evaluation Research’,
Event Management, 6 (4): 247-257
 Richards, G & Wilson, J (2004) The Impact of Cultural Events on
City Image: Rotterdam, Cultural Capital of Europe 2001, Urban
Studies, 41 (10): 1931-1951
 Connell, J. & Page, J.S. (2005) Evaluating the Economic and Spatial
Effects of an Event: The Case of the World Medical and Health
Games. Tourism Geographies, Vol 7 (1), pp 63-85.
 Moore, B & Spires, R (2000) Monitoring and Evaluation. In:
Roberts, P & Sykes, H (eds) Urban Regeneration: A Handbook,
London. Sage