A presentation for AMEC
Katie Delahaye Paine
CEO kdpaine@kdpaine.com
www.measuresofsuccess.com
http:/kdpaine.blogs.com
Member, IPR Measurement Commission www.instituteforpr.org

A measurement timeline
MSM
Eyeball counting
Online
HITS
Social
Media
Engagement

The changing “Holy Grail” of measurement

The immutable laws of 21 st Century PR Measurement
3.
4.
1.
2.
5.
6.
7.
There is no market for your message
All the benchmarks have changed
Size doesn’t matter so stop screaming, start listening
It’s not how many eyeballs, it’s the right eyeballs
HITS = How Idiots Track Success
ROI doesn’t mean what you think it does because you can’t divide by zero
You become what you measure, so match the measurement tool to your objective

Signs that it’s the end of the world as we know it
8. BestBuy measures 85% lower turnover as a result of its Blue Shirt community
7. State Farm measures it’s internal blog by the improvement in morale
6. ASPCA can track on-line donations and increased membership back to its social media efforts.
5. On Twitter, a start up company got 100 great marketing ideas for free, women raised over $6000 in a day and a wooden toy maker in NH got a nationwide contract
4. $0-budget YouTube videos about Barack Obama were seen by 120 times the audience of Hilary Clinton’s “largest town hall meeting in US history” that cost millions
3. IBM receives more leads, sales and exposure from a $500 podcast than it does from an ad
2. Advertisers are starting to admit that all their measures are flawed
1. Measurement is easy

What do you need to measure?
Outputs?
Did you get the coverage you wanted?
Did you produce the promised materials on time and on budget?
Outtakes?
Did your target audience see the messages?
Did they believe the messages?
Outcomes?
Did audience behavior change?
Did the right people show up?
Did your relationship change?
Did sales increase?

Goals, Actions and Metrics
Goal Action Output Metric Outtake Metric Outcome Metric
Increased on-line Revamp website reservations
Amount of content on web site
% perceiving state as a destination
% increase in web traffic and reservations
#1site for visitors to NH
Increase staffing and resources for communications
Increased exposure of
“visit NH” message
Increased perception of
NH as an extreme destination
% increase in agreement with the statement
Website is preferred site for information
Add content, features to web site, keep up to date
% increase in traffic % agreeing with the statement
# 1 rankings, and time spent on site

The 7 steps to Social Media ROI
3.
4.
1.
2.
5.
6.
7.
Define the “R” – Define the expected results?
Define the “I” -- What’s the investment?
Understand your audiences and what motivates them
Define the metrics (what you want to become)
Determine what you are benchmarking against
Pick a tool and undertake research
Analyze results and glean insight, take action, measure again

Step 1: Define the “R” what return do you expect?
If you are celebrating complete 100% success a year from now, what is different about the organization?
If you eliminated your department or failed utterly, what would be different?
Typical Outcomes:
Increased Donations
Reductions in unnecessary pet euthanasia
Improved competitive positioning
Increase loyalty
Lower recruitment costs
Cost Savings
Increased customer satisfaction
Reduce complaints
Help recruit volunteers
Reduce opposition?
Increase community support
Lower costs of communications

Step 2: Define your investment
You can’t divide by $0
People time
Opportunity Cost
Executive time/goodwill

Step 3: Understand what motivates your stakeholders to engage

Step 4: Define your KPIs
Share of thought leadership over time
Cost per message communicated efficiency/efficiency of different channels
Increase in employee engagement in/credibility of communications
Improvement in relationship /reputation scores with customers and communities
Improvement in Optimal Content Score (OCS) over time

Why an Optimal Content Score?
You decide what’s important:
Benchmark against peers and/or competitors
Track activities against OCS over time

Optimal content score for social media
Positive:
Mentions of the brand
Key messages
Positioning
Visibility
Negative
Omitted
Negative tone
No key message

Standard classifications of discussion
• Acknowledging receipt of information
• Advertising something
• Answering a question
• Asking a question
• Augmenting a previous post
• Calling for action
• Disclosing personal information
• Distributing media
• Expressing agreement
• Expressing criticism
• Expressing support
• Expressing surprise
• Giving a heads-up
• Responding to criticism
• Giving a shout-out
• Making a joke
• Making a suggestion
• Making an observation
• Offering a greeting
• Offering an opinion
• Putting out a wanted ad
• Rallying support
• Recruiting people
• Showing dismay
• Soliciting comments
• Soliciting help
• Starting a poll

Standard classifications of videos
Advertisement
Animation
Demonstration
Event/Performance
Fiction
Film
Home Video
Instructional Video
Interview
Lecture
Montage
Music Video
News Broadcast
Promotional Video
Sightseeing/Tour
Slideshow
Speech
Television Show
Video Log

Most frequently used metrics
For External blogs and other Consumer Generated Media (CGM)
Share of positioning
Share of preferred types of conversations
Share of desirable vs. undesirable
Share of visibility
Share of quotes
Optimal content score
For your controlled social media program
Traffic
Repeat visits
Engagement

Step 5: Define your benchmarks
Past Performance
Peer companies
Whatever keeps the C-suite up at night
Think 3-5:
A stretch goal
The underdog who’s nipping at your heels
Peer organizations
Anyone that you compete with for share of mind or share of wallet

Possible your benchmarks
Emerging benchmarks
Engaged = 13 comments per post
Hyper-engaged = 35 comments per post
After 3 days most comments are done, 14 days max
Social Bookmarking momentum = 1 submitted item every other day
Message should be communicated in 2 out of 5 blogs
Past Performance
Think 3
Peer
Underdog nipping at your heels
Stretch goal
Whatever keeps the C-suite up at night

Past performance: tonality of blog content
Tonality of Coverage Over Time
Tonality of all blog postings
150
Total, 10%
125 Total, 19%
100
75
50
25
0
14
Total, 71%
17
27
4
Oct
9
37
9
Nov
2006
43
5
Dec
12
91
9
Jan
2007
Positive
Neutral
Negative

The competitive landscape
Technorati mentions with high authority
US Cellular
1%
Cingular
7%
Sprint
7%
Verizon
10%
Company "sucks" mentions in Technorati with high authority
US Cellular
2% Cingular
16%
T-Mobile
75% T-Mobile
51%
Sprint
12%
Verizon
19%

Step 6: Conduct research (if necessary)
First: find out what already exists
Web traffic
Customer Satisfaction data
Customer Loyalty data
Second: Decide what research is needed to give you the information you need.

Selecting a measurement tool
Objective
Increase inquiries, web traffic, recruitment
KPI
% increase in traffic
#s of clickthrus or downloads
Increase awareness/preference % of audience preferring your brand to the competition
Engage marketplace
Tool
Clicktracks, Web trends
SurveyMonkey, Zoomerang
Conversation index greater than .8
Rankings
TypePad, Technorati
Communicate messages % of articles containing key messages
Total opportunities to see key messages
Cost per opportunity to see key messages
% aware of or believing in key message
Media content analysis –
Dashboards
Survey

The tools you need
Google News/Google blogs
Technorati, Ice Rocket
Cyberalert, CustomScoop, e-Watch
BuzzLogic , Radian 6
RSS feeds
Tweetscan , Twemes
SiteVolume

Your tool box also needs to include:

Your tool box also needs to include:

A Proposed Engagement Index
Time on site
Repeat visits
Forwards/links
/comments +
Relationships
Tone/content of conversation
Membership +
Clickthru
Donations/orders
Signups

11 numbers your web analytics guru should give you every month
% increase or decrease in unique visits
Change in page rank - i.e. a list of the top ten most popular areas and how it has changed in the last week
How many sessions on our blog or web site represent more than 5 page views
In the past month, what % of all sessions represent more than 5 page views
% of sessions that are greater than 5 minutes in duration
% of visitors that come back for more than 5 sessions
% of sessions that arrive at your site from a Google search, or a direct link from your web site or other site that is related to your brand
% of visitors that become a subscriber
% of visitors that download something from the site
% of visitors that provide an email address
Courtesy of Eric Peterson

Components of a Relationship Index
Control mutuality
In dealing with people like me, this organization has a tendency to throw its weight around. (Reversed)
This organization really listens to what people like me have to say.
Trust
This organization can be relied on to keep its promises.
This organization has the ability to accomplish what it says it will do.
Satisfaction
Generally speaking, I am pleased with the relationship this organization has established with people like me.
Most people enjoy dealing with this organization.
Commitment
There is a long-lasting bond between this organization and people like me.
Compared to other organizations, I value my relationship with this organization more
Exchange relationship
Even though people like me have had a relationship with this organization for a long time; it still expects something in return whenever it offers us a favor.
This organization will compromise with people like me when it knows that it will gain something.
This organization takes care of people who are likely to reward the organization.
Communal relationship
This organization is very concerned about the welfare of people like me.
I think that this organization succeeds by stepping on other people. (Reversed)

How to implement relationship metrics
Step 1: Conduct a benchmark relationship study
Step 2: Implement PR program
Step 3: Conduct a follow up relationship study
Step 4: Look at what’s changed

Step 7: Analysis

Data mining the numbers you have
Look for failures first
Check to see what the competition is doing
Then look for exceptional success
Compare to last month, last quarter, last year
Figure out what worked and what didn’t work

Case Study: ASPCA

ASPCA Toolbar
ASPCA Community (blog, MySpace)
Pet Food Recall Resource Center
Updates on home page
To members/ constituents via weekly
News Alert
Courtesy: ASPCA

Used Both “old” and “new” Media
Micro-site featuring a “pledge to fight animal cruelty”
Dedicated URL: www.fightcruelty.org
Postings on ASPCA’s online community, MySpace and
Facebook pages
A radio media tour featuring Ed Sayres on December 12
Pre- and post-surveys to gauge post-launch and campaign awareness

Crunching the Numbers
Expense budget: $0
CPI (cost per impression): $0.01
Every new user is worth $3.50
each to the organization after 12 months; $6 each after 24 months
Overall Web traffic jumped, on an annual average increase, from 25 percent to 52 percent
For every of 100,000 visitors in Web traffic:
Secure 7,000 new registered users
Ultimately worth about $175,000 total net to the organization over their lifetime
Courtesy: ASPCA

Results?
Secured earned media coverage in all top 50 DMAs (goal):
Print: 2.3 million+ impressions
Radio: 7.5 million listeners
TV: 16,000,000+ viewers
Raised Web traffic by >22% over November (more than double goal of
10%)
Secured 4,700 new registered users via signing of Anti-Cruelty pledge:
56% > goal (3,000)
Pre- to post-survey awareness 28%
Expense budget: $130,000
CPI: $.10
Courtesy: ASPCA

Share of ASPCA Subject Exposure Over Time
800
700
600
500
400
300
200
100
-
39
Qtr1
Adoption / NYC Services
Animal Welfare Tips / Seasonal Tips
ASPCA Poison Control Center / Toxicology
Events
Courtesy: KD Paine & Partners
Animal Behavior
Anti-Cruelty
Corporate
47
56
117
100
59
Qtr2
2007
38
62
255
59
Qtr3
135
75
320
79
Qtr4
Animal Sheltering
ASPCA Mission: Orange/No-Kill Communities
Humane Law Enforcement

Share of ASPCA Favorable positioning Exposure Over Time
400
350
300
250
200
150
100
50
-
32
22
41
40
15
13
Qtr1
43
109
38
34
100
16
54
221
45
12
Qtr2
2007
28
27
Qtr3
30
165
25
129
Qtr4
Expert on pet care
Expert on animal cruelty issues
Expert in veterinary care
Expert in disaster relief/preparedness
Expert in animal welfare issues
Expert in animal toxicology
Expert in animal sheltering issues
Expert in animal behavior
National leader in creating a country of humane communities / no-kill
Courtesy: KD Paine & Partners

Social media growth
Courtesy: ASPCA
Blog and ASPCA Online Community Growth
Feb-08 Mar-08 Apr-08
Online Community: Members
4,500
5,900
6,995
Online Community: Total Visitors
Online Community: Unique Visitors
Blog: Visitors
22,148
32,576
37,107
24,145
27,497
33,436
41,846
55,326
55,097

Are they engaged?
Courtesy: ASPCA
Social Media Engagement
Apr-08 Mar-08 Feb-08
Flickr Photo views *
YouTube Channel
Views *
Bebo Friends
Facebook Cause
Members
Facebook Fans
20,000
21,930
39,918
101
140
168
9,000
10,000
13,600
3,000
4,000
4,785
1,000
2,900
6,000
MySpace Profile
Views*
MySpace Friends
30,000
33,900
36,700
166,000
178,079
197,800

KPIs Business Objectives
350,000,000
300,000,000
250,000,000
200,000,000
150,000,000
100,000,000
50,000,000
0
Ja nu ar y
Fe br ua ry
M ar ch
A pr il
M ay
Ju ne
Ju ly
A ug us t
S ep te m be r
O ct ob er r
N ov be em
D ec em be r
$1,800,000
$1,600,000
$1,400,000
$1,200,000
$1,000,000
$800,000
$600,000
$400,000
$200,000
$0
Overall exposure
Online donations

Georgia Tech Engagement Study
Content from Sep. 1 – Nov. 30, 2007
Metrics recorded after 7–30 days, depending on channel †
Athletic content only included on Facebook
Course blogs not included
Publicly available content only
On Facebook, no profiles or names were recorded

Types and Quantity of Media
Bookmarking Sites (341 items) del.icio.us, Digg, Fark, Newsvine, Reddit, Slashdot
External Blogs (332 posts)
50 blogs in 7 categories
Facebook (811+ discussion items)
Networks and Freshman Groups
Institution Blogs (1,901 posts, 3,911 comments)
114 blogs
YouTube (1,668 videos)
User Generated Media 44

Leadership in UGM channels not predicted by traditional media
Share of Desirable Discussion on Uncontrolled Channels
Traditional Media
(N=2,748)
25% 11% 5% 26% 33%
External Blogs
(N=328)
26% 11% 8% 31% 24%
Social Bookmarking
(N=249)
10% 12% 55% 22%
YouTube (N=1,705) 30% 19% 11% 12% 29%
Berkeley Carnegie Mellon Georgia Tech MIT Stanford
Georgia Tech was consistently ranked fifth.
Dynamic, channel-specific environment. Most channels varied significantly.
External blogs were the only outlets to mirror traditional media in discussion of orgs.
Social bookmarking favored tech institutes; significantly more likely to mention MIT than any other channel.
45

During a crisis, UGM channels more likely to be negative
Percent of Content Considered Negative Per Channel
10.6%
14.5%
13.2%
1.9% 1.9%
3.8%
0.0%
Traditional
Media
(2,802 | 154)
External Blogs
(367 | 26)
Facebook
Popular
Topics
(76 | 21)
Georgia Tech Only
1.3%
0.0%
All Institutions
0.0%
0.8% 0.5%
Institution
Blogs
(317 | 12)
Social
Bookmarking
(310 | 5)
YouTube
(1,718 | 194)
UGM amplified negative traditional media coverage.
Unusual negative stories, like MIT’s fake bomb scare, became popular on social bookmarking sites.
Negative news linked to politics was a mainstay on external blogs.
Facebook profiles amplified each of these effects, and also included critical pieces from campus newspapers.
User Generated Media 46

UGM channels offered equal opportunity for message communication
Percent of Georgia Tech Content That Communicated
One or More Strategic Messages
Traditional Media
(N=154)
External Blogs
(N=26)†
Facebook Popular
Topics (n=22)†
Institution Blogs
(N=12)†
Social Bookmarking
(N=5)†
18%
24%
40%
42%
42%
YouTube (N=194) 18%
No UGM channel differed significantly from traditional media in message communication rates.
Among UGM channels, only the difference between external blogs and YouTube can be considered significantly different.
Due to small base sizes, these findings should be considered directional only.
47

Users were engaged with Georgia Tech content when they found it
Georgia Tech’s most engaging items included a blog post about its blowable user interface, its Grand Text
Auto blog, and a video of a band member “crunking.”
Georgia Tech
Peer 1
Peer 2
Peer 3
Peer 4
OVERALL
External
Blogs
Facebook
Discussion
Institution Blogs Social Bookmarks YouTube
Avg.
Comments
18
11
6
13
15
13
Thread
Length
2
5
3
3
2
3
Avg.
Comments
Avg.
InLinks
Avg.
Bookmarks
Avg.
Comments
Avg.
Comments
1
5
2
0
1
2
8,246
590
762
1,405
661
1,687
659
140
305
206
169
228
105
22
40
70
36
59
8
4
2
7
4
5
48

Thank You!