Evaluation - Did it really work?
Effective public relations
measurement and evaluation
Presented by Dr Tom Watson, Associate Professor
in Communication, Charles Sturt University
Flying on one instrument?
Agenda
The communications process
 What is evaluation?
 Models of evaluation
 Brief case studies

How PR is practiced

Grunig’s four models

One-way
• Press agentry
• Informational

Two-way
• Asymmetrical
• Symmetrical
Communications process
Presentation

Attention

Comprehension

Acceptance

Retention

Action - Outcome
McGuire (1984)
Communication and evaluation
Presentation
Attention
Comprehension
Acceptance
Retention
Action
Output (L1)
Out-growth (L2)
Outcome (L3)
Lindenmann 1993
Types of evaluation
Summative – final impact
 Formative – “in progress” assessment


Should be integrated into all stages of
programs
What can be measured?
Success or failure of strategy
 Message distribution and reception
 Understanding, favourability and
acceptance
 Trends

What else?

Internal
Staff attitudes; value of newsletters,
video, team briefings
 SWOT


External
Sponsorship impact
 Issues management and prediction
 Stakeholder attitudes

Case studies
PricewaterhouseCoopers (PwC)
accountants and biz advisers
 DIMIA – Harmony Day
 Polestar – major printing group
 Volvo - XC90 launch

PwC
Why aren’t we getting coverage? –
asked the bosses
 Merger of PW with C&L not seen as
equal – a “takeover”
 Change in approach

PwC lessons
Charts and graphics improve story
acceptance and favourable write-up
 CEO in interviews increases
favourability – “voice of the
organisation”
 Benchmark coverage to find “who’s
writing about what”

DIMIA
Monitored media
coverage of events
 Moved from celebrity
to community basis
 More outside metros
 Increased
participation

Polestar
Europe’s biggest print group
 Going through major changes
 Monitored media coverage
 Able to see strategy and tactics
working

Polestar – mid-2001
6
Number of items carrying message
4
2
0
-2
-4
Negatively scoring items
Positively scoring items
-6
-8
-10
Investment
Innovation
Employer
Forwardthinking
Financial
security
CTP
Message
Demographics
Customer
service
Value
Environment
Polestar – early 2003
9
Negatively scoring items
Positively scoring items
Number of items carrying message
8
7
6
5
4
3
2
1
0
Investment
Innovation
Employer
Forwardthinking
Financial
security
CTP
Message
Demographics
Customer
service
Value
Environment
Volvo – new car launch
UK Launch program
Competing against BMW and
Mercedes
 Strategy was to have continuing
campaign instead of launch “blip”
 Closely targeted – 25 to 44, mid to
high income, degree educated,
adventurous
 Total of 609,000 people (1% of pop.)

PR-led campaign
Media relations drove enquiries to
internet and dealers
 More interest than cars to sell; All
ordered before sale date announced
 Advertising ran for one month and
cancelled – saving of c.$6m
 “PR helped sell the required number
of cars”

PR works!
Volvo
XC90
enquiries
vs PR impact
Volvo
XC90
combined
metrics
PR Impact
600
No. of enquiries
800
4500
700
4000
500
3500
600
400
3000
500
2500
300
400
2000
300
1500
200
200
1000
100
100
500
00
J a n u aJan-02
ry
F Feb-02
e b ru a ry
Mar-02
Apr-02 May-02
Jun-02
M a rc h
A p ril
M a y Jul-02
JAug-02
une
0
Sep-02
J u ly Oct-02A u Nov-02
gu st
SDec-02
e p te m b eJan-03
r
O c toFeb-03
ber
NMar-03
o v e m b e r Apr-03
D ecem ber
Month
- PR impact - Total enquiries
PR impact
¦
¦
Total enquiries (personal and web)
Orders
Advertising spend
Evaluation

Three methods of evaluation used:
Responses to internet and then
dealers
 Reach of coverage to target audience

96% of target group saw it once; 90% twice

Sales
Evaluation – your future
Forget the “Holy Grail”, it’s about
relationships (Hon & Grunig 1999)
 Most PR campaigns don’t influence
behaviour. (McCoy & Hargie 2003)
 Avoid dead-ends like management
language for evaluation, e.g. ROI

(Watson 2005)
References and Reading








Hon LC & Grunig JE (1999). Guidelines for measuring relationships in
public relations. www.instituteforpr.com
Lindenmann, W.K. (1993) An “effectiveness yardstick” to measure public
relations success. Public Relations Quarterly, 38 (1)
McCoy M & Hargie (2003) Implications of mass communication theory for
asymmetric public relations evaluation. Journal of Communication
Management 7 (4)
McGuire, WJ 1984 Attitudes and Attitude Change in Lindzey, G. &
Aronson, E. Ed. The Handbook of Social Psychology Vol.2 3rd Edn,
Random House
Walker, G (1997) Public relations practitioners’ use of research,
measurement and evaluation, Australian Journal of Communication 24 (2)
Watson T (2005) ROI or evidence-based PR: the language of public
relations evaluation. PRism 3, http://praxis.massey.ac.nz/issue_3.html
(forthcoming)
Watson T & Noble P (2005) Evaluating PR, Kogan Page, London
Watson T & Simmons P (2004) Public Relations Evaluation – Survey of
Australian Practitioners, ANZCA04, July 2004
Web sources
www.instituteforpr.com
 www.carma.com.au

Up and away!
Key thinking
“PR is concerned with ill-defined
problems. It cannot be evaluated by a
single method or metric because it
does not deal with simple issues …”
“Likewise, business in general
recognises that a simple, single
financial measure does not give an
accurate reflection of a company’s
true worth.”
Gregory & Watson 2006: 1
Policy and resources
Research in the UK sponsored by
CIPR
 Produced by Henley Management
College
 Looked at practitioner and academic
literature.

CIPR policy 1

PR is part of the management task
and is subject to the same disciplines,
such as the need to set direction,
allocate and manage resources, and
monitor progress.
CIPR policy 2

Measurement and evaluation are
problematic in all areas of
management. Complexity is a key
factor and in … business, it is difficult
to separate out the effect of one area
of management such as public
relations.
CIPR policy 3

However, the situation can be eased
by better planning and objective
setting where precise, measurable
objectives allow for enlightened
judgements of progress in campaigns
and their management.
CIPR policy 4

PR can be measured and evaluated in
terms of:
Contribution to social and economic
development
 Contribution to management,
leadership and organisational
performance by aiding better decisionmaking and avoiding mistakes
 Being a process and part of
programme development and
implementation

CIPR policy 5
Available methods, research based,
provide information that is good
enough for decision making for PR
programmes
 Existing methods … are adequate for
measuring the contribution of PR.
 Each method has limitations, so a raft
of measures appropriate to particular
situations needs to be employed.

The CIPR resources

Go to www.cipr.co.uk/research
Most Admired Companies
A range of evaluation metrics were
used – between four and eight.
 Most frequent were informal and/or
qualitative such as journalist feedback
and discussions with stakeholders.
 “ ‘Just checking’ everything is on
track”.

Gregory, Morgan & Kelly 2005