19
Measuring the
Effectiveness of the
Promotional Program
McGraw-Hill/Irwin
Copyright © 2009 by The McGraw-Hill Companies, Inc. All rights reserved.
Pros and Cons of Measuring Effectiveness
Advantages
Disadvantages
Avoid costly mistakes
Cost of measurement
Evaluate alternative
strategies
Research problems
Increase efficiency in
general
Disagreement on
what to test
Determine if objectives are
achieved
Objections of creatives
Time
Evaluating Alternative Strategies
Measuring Effectiveness: Marketers’ Views
Issue
Agree
MPM is a key priority for today’s
technology companies.
90%
I am dissatisfied with our ability to
demonstrate marketing programs’
business impact and value.
80%
It is important to define, measure, and
take concrete steps in the area of
advertising accountability.
61.5%
I can forecast the impact on sales of a
10% cut in marketing spending.
37%
Measuring Advertising Effectiveness
What to test
•
•
•
•
Source factors
Message variables
Media strategies
Budget decisions
How to test
• Testing guidelines
• Appropriate tests
Where to test
• Laboratory tests
• Field tests
When to test
• Pretesting
• Posttesting
Pretesting Methods
Laboratory
Field
Consumer Juries
Dummy Ad Vehicles
Portfolio Tests
On-air Tests
Physiological Measures
Theater Tests
Rough Tests
Concept Tests
Readability Tests
Comprehension and
Reaction Tests
Posttesting Methods
Recall Tests
Association
Measures
Tracking
Studies
Methods
SingleSource
Systems
Recognition
Tests
Inquiry Tests
Posttesting the Value of a Sponsorship
Test Your Knowledge
Laboratory methods of testing for advertisement
effectiveness generally offer high control but low
_____.
A)
realism
B)
external validity
C)
costs
D)
generalization
E)
value
Where to Test
In the Field
In the Lab
Positioning Advertising Copy Test (PACT)
1. Provide measurements relevant to objectives of advertising
2. Require agreement on how results will be used
3. Provide multiple measures
4. Be based on a model of human response to communications
5. Consider multiple versus single exposure to the stimulus
6. Require alternative executions to have same degree of finish
7. Provide controls to avoid biasing effects of exposure context
8. Take into account basic considerations of sample definition
9. Demonstrate reliability and validity
Test Points
1.Concept Testing
2.Rough Testing
Occurs at
Various Stages
3.Finished art or
commercial pretesting
4.Market testing
(posttesting)
Concept Testing
Objective
Explores consumers’ responses to ad
concepts expressed in words, pictures, or
symbols
Alternatives are exposed to consumers who
match the target audience
Method
Reactions & evaluations sought through focus
groups, direct questioning, surveys, etc.
Sample sizes depend on the number of
concepts and the consensus of responses
Output
Qualitative and/or quantitative data
evaluating and comparing alternative
concepts
Rough Art, Copy, and Commercial Testing
Comprehension and Reaction Tests
Consumer Juries
Advantages
Disadvantages
Control
Consumer may become a
self-appointed expert
Cost effectiveness
Number of ads that can be
evaluated is limited
Endorsements by
independent third parties
A halo effect is possible
Achievement of credibility
Preference for ad types
may overshadow objectivity
Rough Testing Terms
Animatic Rough
Terms
Photomatic Rough
Live-action Rough
An Animatic Example from Airwalk
*Click outside of the video screen to advance to the next slide
Spot Resulting from Animatic Rough
*Click outside of the video screen to advance to the next slide
Pretesting Finished Print Ads
A laboratory method
Portfolio
Tests
Includes test and control ads
Portfolio test have problems
Readability
Tests
Dummy
Advertising
Vehicles
Based on syllables per 100 words
Other factors also considered
Distributed to random sample homes
Product interest may still bias results
Test Your Knowledge
_____ is a method of testing ads by placing them
randomly in certain copies of regularly distributed
magazines.
A)
Vehicle source testing
B)
Burke's reflection test
C)
A Flesch test
D)
Dummy advertising vehicle testing
E)
A contextual test
Pretesting Finished Broadcast Ads
Theater Tests
On-Air Tests
• Measures changes in
product preferences
• Insertion in TV programs
in specific markets
• May also measure . . .
• Interest in and reaction
to the commercial
• Reaction from an
adjective checklist
• Recall of various
aspects included
• Interest in the brand
presented
• Continuous reactions
• Limitations are imposed
by “day-after recall”
• Physiological Measures
Physiological Measures
Galvanic skin
response
Pupil dilation
Testing
Brain waves
Eye tracking
Market Testing Print Ads
Inquiry Tests
Recognition Tests
Testing
Tracking Studies
Recall Tests
Starch-Scored Sports Illustrated Ad
Posttests of Broadcast Commercials
Day after recall
tests
Tracking
studies
Persuasive
measures
Diagnostics
Testing
Comprehensive
measures
Single-source
tracking
Test marketing
Comprehensive Testing by Ipsos-ASI
Problems With Current Research Methods
Essentials of Effective Testing
Use a consumer
response model
Establish
communications
objectives
Testing
Understand and
implement
proper research
Use pretests and
posttests
Use multiple
measures
Test Your Knowledge
Good tests of advertising effectiveness must address
the nine principles established by PACT. One of the
easiest ways to do this is to follow a decision sequence
model. The first step in the model is to:
A) Understand the appropriate research
B) Create a model that uses multiple
measures
C) Establish communication objectives
D) Decide whether to use posttests or
pretests
E)
Develop a consumer response model
Measuring Effectiveness of Other Programs
Sales
promotions
Shopping cart signage
Nontraditional
media
Ski resort-based media
In-store radio and television
Other media
Sponsorships
Exposure methods
Tracking measures
Measuring Effectiveness + Efficiency
