The Evolution of Evaluation
What Gets Measured is What Counts
1
Chris Foster
Vice President
Strategy & Organization
Booz I Allen I Hamilton
2
Communicating with the C-suite
Communications Measurement is Seen as an Add-On
“You have to move beyond the obvious
metrics such as media impressions,
Facebook "likes" and Twitter followers.
Those can provide a clue to your reach. If
you really want to show your effectiveness,
you need to decide what needle you want
to move—sales, fundraising activity or
advocacy reach, for example—and then
measure your media activity against that.”
- Laura Howe, VP, public relations at the
American Red Cross
• The C-suite executives expect metrics and prefers those of
outcome-oriented persuasion
• Although PR practitioners note that engagement is a best
practice, most still rely on impressions as evidence of impact
• Measures must move beyond mere impressions (outputs) to
understanding comprehension of message (outtakes), and
ultimately, to understanding how the message impacted the
target audience’s attitudes and emotions (outcomes).
3
Similarities Across Markets
(Global Survey Results)
• We need metrics that support business decisions
• Web metrics such as sentiments and perceptions are most
effective measures of digital media impact
• Economy has had a negative effect on communications and
measurement budgets
• We need much deeper understanding of communications
measurement best practices
• Greatest need is for a common ROI definition
• We need standard metrics for ROI and communications
effectiveness
October 2012 Research Results
5
Tracking challenges
Difficult to show
bottom line impact
Intangible value
Hard to link cause and effect
Cost
Red tape
Poor skill sets
What are the biggest challenges to measuring
the value of public relations?
Changing market dynamics
PR measurement and
media relations measurement
are not synonymous
Communication barriers
Lack of standards
Not black and white
Measuring intangible value Lack of support
Tough to correlate Defining/measuring ROI
with impact Lack of time Changing visions
Measuring outcomes over outputs
6
Poor methodology
Leadership buy-in
Lack of consensus
Not persuasive enough
Seen as hassle
Lack of resources
No one size fits all
Lack knowledge
What do you believe are the greatest barriers to the
adoption of standardized research techniques for the
measurement of public relations?
Lack of standard benchmarks
Lack of standards
Institution-wide inertia
Communication barriers
Varied business models
7
Importance of PR Measures
On a scale of 0 to 10, where “0” is unimportant and “10” is important, how important is it the
following are well understood across the public relations profession in the next five years?
Incidence of “Very” and “Somewhat Important” Rating
Setting measureable objectives
91%
How to attribute the impact from a PR program on business goals / objectives
88%
What should be measured beyond media clips
84%
The benefits of measuring a PR campaign
84%
Ensuring standard approaches to measurement throughout an organization
82%
Measuring the value of building relationships with influencers
79%
How to measure the ROI of my PR campaign / spend
77%
How to measure social media
77%
How to measure the impact of an event
75%
60%
70%
80%
90%
100%
Base: All respondents
n=57
8
Importance of PR Measures
On a scale of 0 to 10, where “0” is unimportant and “10” is important, how important is it the
following are well understood across the public relations profession in the next five years?
Incidence of “Very” and “Somewhat Important” Rating
Measuring the effects of sponsorships
75%
How PR activities can be incorporated into a company's Marketing Mix Modeling
74%
How often to measure an on-going campaign or program
72%
How soon after a campaign to measure
70%
Isolating the effect of PR from other campaign activities
70%
How to measure the value of corporate reputation programs
70%
How to measure the value of employee engagement
60%
Percent of PR campaign that should be spent on measurement
Base: All respondents
n=57
53%
0%
25%
50%
75%
100%
9
Attribute Assessment
How much do you agree or disagree with the following statements, where “5” is “strongly agree”
and “1” is “strongly disagree”?
Incidence of “Strongly” and “Somewhat Agree” Rating
Common definitions for the ROI of public relations are needed
74%
The language of PR measurement needs to change to connect better with the C-suite and Board table
72%
Global PR organizations should make educating organizations about the proper measurement of PR a top priority
67%
The Barcelona Principles were a good start, but now we must get much more specific
55%
Common terms and definitions for the measurement of PR do not exist at all
23%
A common set of standards are not applied in the PR measurement profession
19%
A common set of standards for PR measurement is unnecessary
16%
The Barcelona Principles are the common set of standards the industry had been lacking
11%
0%
25%
50%
75%
100%
Base: All respondents
n=57
10
Challenges / Barriers to Measurement
What are the challenges/barriers to measurement that you encounter? Below are statements about
barriers to measuring communications that you may or may not agree with or hear most commonly
from clients. Using the scale where “0” means you completely disagree with that statement and a
“10” means you completely agree with that statement, please rate each of the following.
Incidence of “Very” and “Somewhat Agree” Rating
I don't have the internal staff to conduct research on communication effectiveness
68%
There isn't an effective tool to measure the impact of communications
46%
Senior management doesn't value communications measurement and research
42%
It's more important to spend budget on communications programs than measurement and research
39%
It's not possible to measure the impact of social media on my audiences
12%
It's not possible to measure the return on investment for public relations activities
12%
Communications measurement does not improve my job performance
Base: All respondents
n=57
9%
0%
25%
50%
75%
100%
11
Digital Media Impact Metrics
Below is a list of metrics that may be used to measure the impact of digital media. Based on your
experience or by what you read and hear, which one do you consider most effective?
Survey research to measure audience awareness, sentiments and perceptions
44%
Website metrics (volume, unique hits, time spent on site, page views)
21%
Volume of online mentions (Twitter, Facebook, etc.)
9%
Key word search metrics
4%
Measure of online sentiment through social media monitoring tool
2%
Other
16%
There are no metrics or methods to effectively measure the effect of digital media
5%
0%
25%
50%
75%
100%
Base: All respondents
n=57
12
Importance of Competencies
Public Relations Professionals
On a scale of 0 to 10, where “0” is unimportant and “10” is important, how important are each of
the following competencies for a public relations professional?
Incidence of “Very” and “Somewhat Important” Rating
Has knowledge of different outcome measures
92%
Understands the benefits of different media measurement techniques
92%
Able to advise clients how measurement drives business performance
89%
Measuring quality and does not rely on simple quantitative measure such as clips and impressions
89%
0%
25%
50%
75%
100%
Base: Public Relations Professionals
n=47
13
Importance of Competencies
Public Relations Professionals
On a scale of 0 to 10, where “0” is unimportant and “10” is important, how important are each of
the following competencies for a public relations professional?
Incidence of “Very” and “Somewhat Important” Rating
Has clear understanding of research and metric terminology
87%
Understands how analytics is part of integrated communications efforts across channels
87%
Recognizes how to measure traditional and social media consistently
87%
Understands how to write goals and objectives
87%
Selects appropriate metrics for outputs, outcomes and business results
85%
Knowledge of quantitative and qualitative research techniques
85%
Emphasizes transparency in all facets of research
81%
Create effective research proposal requests, vendor and tool selection, develop research budgets
70%
0%
25%
50%
75%
100%
Base: Public Relations Professionals
n=47
14
Effect of Economy on Communications Budget
During the past three years, what impact, if any, has the economy had on your organization’s
overall communications budget?
Very nega ve effect
0.25
Somewhat nega ve effect
0.35
Very posi ve effect
0.02
Somewhat posi ve effect
0.05
Net:
Positive effect
7%
Negative effect 60%
Base: All respondents
n=57
Neither nega ve nor posi ve effect
0.33
15
Key Findings
• Practitioners see many challenges to measuring the value of PR and
implementing measurement into standard practice. Limitations on staff, time,
budgets and a general lack of understanding of measurement principles
contribute to this challenge.
• In the past three years, the economy has had a negative effect on
communications and measurement budgets, limiting resources while
simultaneously increasing the importance to show the value
in our work for clients.
• Within five years, over 90% of surveyed public relations professionals
indicated a need for the development of communication measurement
standards and practices.
• A desire for more effective measurement creates a need for measuring ROI
and capturing true business impact.
16
Solution
As an industry, we need to:
•Change the way we communicate and present our value
•Incorporate measurement in our missions and objectives
•Use research-driven data to demonstrate the value that
strategic communications delivers in a way that can be
universally understood and embraced by budgets across the
globe
17
Call to Action
As an industry, we have a shared responsibility to:
•Reframe how we talk about measurement and communications and
the value of communications and PR
•Seize opportunities to gain more understanding in measurement
and identify how to implement into practice
•Utilize communication as a tool in problem solving
•Meet the demand for data-driven campaigns
18