COM215
Taejin Jung, Ph.D.
Week 12: Evaluation
The Purpose of Evaluation

The widespread adoption of the MBO (Management-by-Objectives)
system by clients and employers of PR specialists

The measurement of results against established objectives set
during the planning process

“The systematic assessment of a program and its results. It is a
means for practitioners to offer accountability to clients-and to
themselves”

During the campaign, constant research must be used to determine
the strategies and tactics are working

Shared Values Campaign (http://www.prwatch.org/node/6465)
Objectives: A Prerequisite for Evaluation

It is important to have a clearly established set of “measurable”
objectives
- PR personnel and management (clients) should agree on the criteria that
will be used to evaluate success in attaining objectives
- Pre-determined: Don’t wait until the end of the public relations program to
determine how it will be evaluated

Checklist for the basic evaluation questions
- Was the activity or program adequately planned?
- Were all primary and secondary audiences reached?
- Did the recipients of the message understand it?
- Was the desired organizational objective achieved?
- What unforeseen circumstances affected the success of the program or
activity?
- Did the program or activity fall within the budget set for it?
- What steps can be taken to improve the success of similar future
activities?
- How could the program strategy have been more effective?
Current Status of Measurement and
Evaluation goals for PR
Evaluation
programs

PR personnel use a mix of
evaluation techniques, many
borrowed from advertising and
marketing, to provide more
complete evaluation


Sophisticated techniques
- Computerized news clip
analysis
- Survey (quasi-experimental
designs)
PR budget devoted to
measurement and evaluation
is now more than 5 percent
Advanced (Motivation)
Behavioral Change
Attitude Change
Opinion Change
Intermediate (Awareness)
Retention
Awareness/Comprehension
Basic (Exposure)
Message distributions
Impressions
Media Placements
Exposure:
Measurement of Production

Count how many news releases, feature stories, photos,
letters, and the likes are produced in a given period of time
- Not meaningful because it emphasizes quantity instead of quality

Specify what the PR person should accomplish in obtaining
media coverage
- Unrealistic and impossible to guarantee

Quantify the distribution
- Ex) A total of 756 news releases were sent to 819 daily
newspapers, 250 weeklies…
Exposure: Measurement of
Message Exposure (placement)

The compilation of press clippings and
radio-television mentions
- Most widely practiced form of evaluating PR programs
- The publicity effort accomplish the first stage of the adoption
process by making people expose to it.
- Ex) Burrelle’s monitors nearly 400 local TV stations in 150
cities/ Web tracking systems and firms
- Count “placement” and calculate “impressions”
Measurement of Message Exposure

Media impressions
(← media placement)
- How many people may have
been exposed to the message
- The potential audience
reached by a periodical or a
broadcast program (circulation
x placements)
- They don’t disclose how
many people actually read or
heard the stories and how
many absorbed or acted on the
information

Hits (visits) on the
Internet
- Cyberspace version of media
impressions
- The number of people
reached via an organization’s
WWW or home page
Measurement of Message Exposure

Systematic Tracking
- New advances in computer
software make it possible to track
media placements
- Computer database analyze the
content of media placement
a. market penetration
b. type of publication
c. tone of coverage
d. sources quoted
- Continuing, regular feedback
during a campaign in terms of
placements and mention of key
messages
- To compare its media coverage
with the competitors
- ROI (Return on Investment):
Comparing the number of news
release sent with the number
actually published and in what
kinds of periodicals
<Example of Systematic Tracking>
Total Coverage
Total
Total impressions
% of positive impressions
% of negative impressions
89,641,378
26.98%
19.85%
Total articles
% of positive articles
% of negative articles
1,049
35.65%
16.02%
Measurement of Message
Exposure

Cost per Person
- To determine the cost of
reaching each member of the
audience
- Cost-effectiveness (CPM: Costper-Thousand)
The cost of publicity / the total
media impressions
- A 30-second commercial during
the 2004 Super Bowl cost $2.4
million ($2.4 million/145 million =
less than a half cent)

Audience Attendance
- Counting attendance at event
is a relatively simple way of
evaluating the effectiveness of
pre-event publicity
- Better objective evaluation is
needed
Measurement of Audience
Awareness

To determine whether the audience actually became
aware of the message and understood it
- The second level of PR evaluations
- Whether they paid attention (understood, have
retained) the messages in any shape or form

“Survey” is needed to answer such questions
- Target audience be asked about the message and what
they remember
- Public awareness of sponsoring organization
- Day-after recall
Measurement of Audience
Attitudes

Quasi-experimental study
- A measurement of audience attitudes and opinions
before, during, and after a public relations campaign
- Show the percentage difference in attitudes and
opinions as a result of publicity
- Considering intervening variables (e.g., coefficient and
R2)
“The only way to determine if communications are
making an impact is by pre- and posttest research”
Measurement of Audience
Action

The objective of “Greenpeace” is not to get editorials
written in favor of whales, but to motivate the
public…
a. to write elected officials
b. to send donations for its preservation efforts
c. to get protective legislation passed

Public relations efforts ultimately are evaluated on
how they help an organization achieve its objectives
Example: The Truth Campaign

Florida "truth" Campaign