Chapter 12
Measuring the effectiveness of
the IMC program
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-1
Learning objectives
1. To understand the importance of, and reasons
for and against, measuring the effectiveness
of marketing communication.
2. To establish some guidelines used in
measuring IMC effectiveness.
3. To evaluate alternative methods for measuring
IMC effectiveness against industry-developed
criteria for successful evaluation.
4. To explore the unique challenges of IMC
effectiveness and current approaches to
evaluation including process measures,
synergy measures and ROI.
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-2
Arguments for and against
Process
measures
What to
test?
Importance of
measurement
When to
test?
Synergy
Evaluating
IMC
Evaluation
Measurement
process
Where to
test?
ROI
New
measures
Concept
testing
How to
test?
Testing for
campaign
development
Rough art &
copy
Pre-testing
finished ads
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
Marketplace
testing
12-3
Singapore’s Ministry of
Community Development,
Youth and Sports (MCYS)
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-4
Singapore’s Ministry of Community
Development, Youth and Sports
(MCYS) (cont.)
Campaign objectives
Reach, engagement and impact objectives
 Make 500,000 people aware of the message
 Generate $1 million publicity
 Achieve 20,000 social media interactions
 Create 2000–3000 fans
Attitudinal and behavioural objectives
 To encourage people to re-think their relationships
and start dating.
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-5
Selected campaign results
Awareness
■ Watched by more than 3.2 million people on social media in less than 25
weeks
■ Generated more than S$2.3 million of print, radio and television PR
Likeability
■ 19th most favourite YouTube video in the world
■ 88th most watched YouTube video in the world
■ More than 120 000 unique visitors on Facebook
Engagement
■ More than 17 million impressions on Facebook;
■ More than 17 900 fans on the Facebook page
■ More than 100 000 interactions on Facebook page
Behaviour change
■ 83% felt that ‘the ad made me think more about the meaning of marriage’
■ 64% felt that ‘after seeing the ad, I view my partner differently’
■ 33% felt that ‘after seeing the ad, I feel like meeting more people and
start [sic] dating’
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-6
Arguments for and against
Process
measures
What to
test?
Importance of
measurement
When to
test?
Synergy
Evaluating
IMC
Evaluation
Measurement
process
Where to
test?
ROI
New
measures
Concept
testing
How to
test?
Testing for
campaign
development
Rough art &
copy
Pre-testing
finished ads
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
Market place
testing
12-7
Three phases of evaluation
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-8
Importance of measurement
The single biggest problem facing
marketers since the invention of
advertising is how to measure
return on investment.
Harold Mitchell, Mitchells Media, 2009
Half the money I spend on
advertising is wasted; the trouble is,
I don’t know which half.
John Wanamaker, US retailer, c.1910
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-9
Reasons for and against
measuring effectiveness
For
Against
Avoid costly mistakes
Cost of measurement
Evaluate alternative
strategies
Research problems
Increase efficiency in
general
Determine if objectives are
achieved
Disagreement on what to
test
Objections from creative
personnel
Time
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-10
Chiat/Day’s views on recall tests
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-11
Arguments for and against
Process
measures
What to
test?
Importance of
measurement
When to
test?
Synergy
Evaluating
IMC
Evaluation
Measurement
process
Where to
test?
ROI
New
measures
Concept
testing
How to
test?
Testing for
campaign
development
Rough art &
copy
Pre-testing
finished ads
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
Market place
testing
12-12
Measuring advertising
effectiveness
What to test
• Source factors
• Message variables
• Media strategies
• Budget decisions
How to test
• Testing guidelines
• Appropriate tests
Where to test
• Laboratory tests
• Field tests
When to test
• Pretesting
• Posttesting
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-13
What to test?
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-14
When to test?
Pretests
Measures taken before the
campaign is implemented
Posttests
Measures taken after the ad or
commercial has gone to air
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-15
When and where to test?
Pretesting methods
Laboratory
Field
Consumer juries
Dummy ad vehicles
Portfolio tests
On-air tests
Physiological measures
Theatre tests
Rough tests
Concept tests
Readability tests
Comprehension and reaction tests
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-16
When and where to test?
Field posttesting methods
Recall tests
Association
measures
Tracking studies
Posttesting
methods
Recognition
rests
Single-source
systems
Inquiry tests
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-17
How to test? Positioning
advertising copy testing (PACT)
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-18
Arguments for and against
Process
measures
What to
test?
Importance of
measurement
When to
test?
Synergy
Evaluating
IMC
Evaluation
Measurement
process
Where to
test?
ROI
New
measures
Concept
testing
How to
test?
Testing for
campaign
development
Rough art &
copy
Pre-testing
finished ads
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
Market place
testing
12-19
Testing for campaign development
1. Concept testing
Occurs at
various stages
of campaign
development
2. Rough testing
3. Finished art or
commercial testing
4. Market testing
(posttesting)
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-20
Concept testing
Objective
Explores consumers’ responses to ad concepts expressed
in words, pictures or symbols.
Alternatives are exposed to consumers who match the
target audience.
Method
Reactions and evaluations are sought through focus
groups, direct questioning, surveys, etc.
Sample sizes depend on the number of concepts and the
consensus of responses.
Output
Qualitative and/or quantitative data are used to evaluate
and compare alternative concepts.
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-21
Rough art, copy and
commercial testing
Comprehension and reaction tests
Consumer juries
Advantages
Disadvantages
Control
Consumer may become a selfappointed expert
Cost effectiveness
Number of ads that can be
evaluated is limited
Endorsements by
independent third parties
A halo effect is possible
Achievement of credibility
Preference for ad types
may overshadow objectivity
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-22
Rough testing terms
Animatic rough
Terms
Photomatic rough
Live-action rough
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-23
Online methods
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-24
Pretesting finished print ads
A laboratory method
Portfolio
tests
Includes test and control ads
Portfolio test have problems
Readability
tests
Based on syllables per 100 words
Dummy
advertising
vehicles
Distributed to random sample of homes
Other factors also considered
Product interest may still bias results
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-25
Dummy advertising vehicles
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-26
Pretesting finished digital ads
Theatre tests
On-air tests
 Measures changes in product
preferences
 Insertion in TV programs in
specific markets
 May also measure:
 Interest in and reaction to the
commercial
 Reaction from an adjective
checklist
 Recall of various aspects
included
 Interest in the brand presented
 Continuous (frame-by-frame)
reactions
 Limitations are imposed by
‘day-after recall’
 Physiological measures
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-27
Physiological measures
Galvanic skin
response
Pupil dilation
Testing
Brain waves
Eye tracking
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-28
Physiological measures: eye
movement research
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-29
Market testing print ads
Inquiry tests
Recognition tests
Market testing:
print ads
Tracking studies
Recall tests
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-30
Starch-scored advertisements
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-31
Market testing broadcast
commercials
Day-after recall
tests
Tracking studies
Single-source
tracking
Persuasive
measures
Market
testing:
broadcast
Diagnostics
Comprehensive
measures
Test marketing
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-32
World’s most popular posttest
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-33
Tracking studies
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-34
Tracking studies (cont.)
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-35
Arguments for and against
Process
measures
What to
test?
Importance of
measurement
When to
test?
Synergy
Evaluating
IMC
Evaluation
Measurement
process
Where to
test?
ROI
New
measures
Concept
testing
How to
test?
Testing for
campaign
development
Rough art &
copy
Pre-testing
finished ads
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
Market place
testing
12-36
Measuring the effectiveness of
the IMC program
Alternative IMC measures
Process
measures
Synergy
The audit
• Media interactions
• Interactions between
communications mix elements
ROI
• Return on brand Investment
• Return on touchpoint investment
New
measures
• Need for common media
currency; metrics for social media
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-37
New measures
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-38
Digital touch points
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-39
Social media
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-40
Summary and conclusions
 Research to measure the effectiveness is
important to the IMC program.
 Not enough companies systematically test
their programs.
 Problems exist with current research
methodologies.
 A variety of research methods—including
pretesting and posttesting—were outlined.
 It is important that effectiveness measures
are linked to campaign objectives.
Copyright  2012 McGraw-Hill Australia Pty Ltd
PowerPoint Slides t/a Advertising and Promotion 2e by Belch, Belch, Kerr & Powell
12-41