Measurement & Evaluation in PR:
2007 to 2014
Professor Tom Watson, Bournemouth
University
Agenda
•
•
•
•
•
•
From 2nd to 3rd edition (2007 to 2014)
Principles
Research priorities
Barcelona Principles
ROI debate
Communication Performance Management
Evaluating Public Relations, 3rd edition
Published in June; Available in Australia in August
London: Kogan Page ISBN: 9780749468897
Aircraft instruments
Defining evaluation
• “…any and all research designed to determine the
relative effectiveness of a public relations program,
strategy, or activity, by measuring the outputs and/or
outcomes of that PR program against a predetermined
set of objectives” (Lindenmann)
• Management-by-Objectives (MBO) framework
Measurement concepts
• Input: expenditures on communication-related services; use
of financial and human resources
• Output: Presentation and dissemination
• Out-take: Awareness; processing of messages
• Outcome: The desired result
• Arise from Lindenmann’s typology; Widely applied
Outflow -1
• Outflow is the economic impact that results from influence on
stakeholders exerted by corporate/organizational
communications
• “It becomes visible what communications have actually
contributed to achieving the financial and strategic goals of
the organization” (DPRG 2011:14)
Outflow -2
• Corporate communication can add value by supporting
service provision processes of other corporate functions or by
creating intangible assets
• Both contributions depend on specific organisational goals
and strategy
• Performance indicators include business-related metrics such
as sales, innovations, productivity, etc. or intangible capital
such as monetary brand value or reputational capital
‘Value’ as a PR research priority:
Delphi study (Watson (2008)
1. PR’s role in contributing to strategic decision-making,
strategy development and realization and organizational
functioning
2. The value that PR creates for organizations through building
social capital; managing key relationships and realizing
organizational advantage
3. The measurement and evaluation of public relations both
offline and online
Barcelona Principles
• Seven principles of measurement
• Set by AMEC – adopted worldwide by PR professional bodies
• Changed thinking away from AVE
Barcelona Principles
ROI – misapplied language
• PR sought to use business language
• Return on Investment widely expressed, inappropriately
• Related to value created over time by investment in capital
equipment
• Not to immediate results of a PR activity
• Many PR actions can’t express a financial result (Gov’t; NfP)
• “loose and fuzzy” (Watson & Zerfass, 2011, 2012)
Jim Grunig speaks
“I talk more about the value of public relations than about ROI.
As I said, you can explain the value of relationships; but you
really can’t measure a financial return to compare with the
money invested in it. I tend to use the term ROI because PR
people want to hear it used. I will now cease and desist from
using it” (Likely & Watson, 2013: 153)
‘Communication Controlling’
• Controlling = auditing, performance management
• Central European approach to link corporate objectives with
communication objectives
• Each level can be monitored: Input > Output (2 stages) >
Outcome (2 stages) > Outflow
• Matrix of measurements, appropriate to each level
• Used by BASF, Siemens, Henkel, Commerzbank, etc
Outflow
Outflow
Levels of Impact and Evaluation
of Communications
Levels of
Impact
Output
Input
Ressources
MeasureEmployee Assignment
ment Range Financial Expenses
Indicators
(e.g.)
MEASURED
OBJECT
Personnel Costs
Outsourcing Costs
…
Internal Output
External Output
Process Efficiency
Quality of Works
Product
Media Coverage
Content
Budget Compliance
Throughput Times
No. of Shortcomings
Readability/Fogg-Index
Satisfaction of Internal
Clients
…
ORGANISATION
Initiation of Communication Processes
Low Impact on Value Creation
Strong Influence of Communications Management
Clippings
Visits
Downloads
Impact Ratio
Share of Voice
…
MEDIA/CHANNELS
Value Creation
Outcome
Direct Outcome
Indirect Outcome
Perception
Utilization
Knowledge
Opinion
Attitudes
Emotion
Behavioral Disposition
Behavior
Awareness
Unique Visitors
Session Length
Reader per Issue
Reputation Index
Brand Image
Strategic Awareness of
Employees
Purchase Intention
Leads
Innovative Ideas
Project Participation
…
Recall
Recognition
…
STAKEHOLDERS
Communication Processes
Impact on
Strategic and/or
Financial Targets
(Value Chain)
Impact on
Tangible and/or
Intangible Assets
(Capital Accumulation)
Sales
No. of Project
Agreements
Cost Reduction
Reputation Capital
Brand Value
Employee Performance
…
ORGANISATION
Results of
Communication Processes
High Impact on Value Creation
Weak Influence of Communications Management
Implementation
• Communication Controlling has four phases
–
–
–
–
Analysis
Conception
Operationalization
Steering / Reporting
Implementation steps of Communication Controlling
Analysis
Conception
1
2
3
Corporate
Goals
Analysis
Key Questions
Key Questions
• Which data already •
exist in the company?
• Which data can be
obtained with
reasonable cost and •
time expenditures?
• What is the current
status of corporate
•
communications?
Which are the goals
of the company for
the current planning
period?
Are these goals
clearly defined and
prioritized?
What are the strategic
goals and what can
the company do to
achieve those?
• The achievement of
which of these goals
requires the support
of the
communications
function?
4
Communication
Objectives
Key Questions
• What do
communications need
to contribute in terms
of stakeholder impact
and economic
outflow?
• Which
communications
objectives are aimed
at which stakeholder
groups?
• Which new potential
assets need to be
developed to address
these objectives?
• What do we have to
do for that and what
kind of resources will
we have to invest?
Metrics &
Indicators
Key Questions
• How shall the
success of
communications
be measured?
• Which indicators
and KPIs can be
defined for these
metrics?
Operationalization
Steering/Reporting
5
6
Data
Collection
Key Questions
Steering
and
Reporting
Key Questions
• Which data will be
• Who is to be informed at
collected at which
which point in time or
point in time?
event?
• Which evaluation
• In which form and way
methods and
are the findings to be
instruments are to be reported?
applied?
• What kind of
recommendations are
expected to support
decision-making?
Making it work
•
•
•
•
•
•
Has organization defined its corporate objectives?
Are “scorecards” used?
Is measurement and evaluation part of corporate culture?
How is the communications function organized?
“Strong and visible” top management commitment is needed
Close cooperation with management accountants
Summary
• Multiple measurement needed for evaluation
• Value can’t always be measured but can be evaluated
• Communication Performance Management links PR with
objectives
• Read all about it!