8
Measuring the Effectiveness of the
Promotional Program
Chapter Objectives
• To understand reasons for measuring
promotional program effectiveness.
• To know the various measures used in
assessing promotional program effectiveness.
• To evaluate alternative methods for
measuring promotional program
effectiveness.
• To understand the requirements of proper
effectiveness research.
© 2005 McGraw-Hill Ryerson Limited
Measuring Advertising Effects
SALES?
Product Quality
Promotion
Competition
SALES
The Economy
© 2005 McGraw-Hill Ryerson Limited
Price Policy
Distribution
For and Against Measuring Effectiveness
Reasons to Measure
Reasons Not to Measure
Avoid Costly Mistakes
Cost of Measurement
Evaluate Strategies
Problems With Research
Increase Efficiency of
Advertising in General
Disagreement About What
to Test
Determine If Objectives
Are Achieved
Creative Objections
© 2005 McGraw-Hill Ryerson Limited
Measuring Advertising Effects
• PRETESTING
– Concept tests
– Rough testing
• COMMERCIAL TESTING
– Consumer juries
– Physiological measures
– Theatre tests
• MARKET TESTING
– on-air (split run cable) tests
– Recall and Recognition tests (day after)
© 2005 McGraw-Hill Ryerson Limited
The Testing Process
1.Concept Testing
2.Rough Testing
Occurs at
Various Stages
3.Finished Art or
Commercial Testing
4.Market Testing (Posttesting)
© 2005 McGraw-Hill Ryerson Limited
8
Concept Testing
Concept Generation and Testing
Objective
Explores Consumers’ Responses to Ad Concepts
Expressed in Words, Pictures, or Symbols
Alternatives Are Exposed to Consumers Who
Match the the Target Audience
Method
Reactions and Evaluations Are Sought Through
Focus Groups, Direct Questioning, Surveys, Etc.
Sample Sizes Depend on the Number of Concepts
and the Consensus of Responses
Output
Qualitative And/or Quantitative Data Evaluating
and Comparing Alternative Concepts
© 2005 McGraw-Hill Ryerson Limited
Research House (Canada)
© 2005 McGraw-Hill Ryerson Limited
Concept Testing
• Limitations:
– Sample sizes are small (not quantifiable)
– Consumers become instant “experts”
– Members may not represent the target
market
– Results may seem definitive than they are
• Company example
© 2005 McGraw-Hill Ryerson Limited
8
Commercial Testing
Rough Art, Copy, and Commercial Testing
Comprehension and Reaction Tests
Consumer Juries
Advantages
Disadvantages
Control
Consumer May Become a
Self-appointed Expert
Cost Effectiveness
Number of Ads That Can Be
Evaluated Is Limited
Endorsements by
Independent Third Parties
A Halo Effect Is Possible
Achievement of Credibility
Preference for Ads Types
May Overshadow Objectivity
© 2005 McGraw-Hill Ryerson Limited
Consumer Juries
Potential viewers evaluate ads
•
50 to 100 participants
•
Overall reaction to each ad
•
And rank ordering of ads
•
Examples of questions:
1. Which ads would you most likely read?
2. Which headlines interests you the most?
3. Which ad convinces you of the product’s quality?
4. Which ad did you like best?
5. Which ad did you find most interesting?
•
Company example
© 2005 McGraw-Hill Ryerson Limited
© 2005 McGraw-Hill Ryerson Limited
Starch Test
Used for print Ads
• Asses what people saw
• Measures what they
remember
• Used to compare ads
© 2005 McGraw-Hill Ryerson Limited
© 2005 McGraw-Hill Ryerson Limited
Physiological Tests of Finished Ads
1. Pupil dilation
2. Galvanic skin response
3. Eye tracking
4. Brain waves
© 2005 McGraw-Hill Ryerson Limited
Pupil Dilation
Size of Subjects pupils are
measured
– Action  Dilation
– Disinterest  Constriction
Male
Female
Nude
Female
Nude
Male
© 2005 McGraw-Hill Ryerson Limited
Nude
Infant
Pupil Dilation - Camera
© 2005 McGraw-Hill Ryerson Limited
Galvanic Skin Response
Measures sweating
• GSR responds to:
• feelings
• attention
• may indicate:
• long-term recall
• effectiveness
© 2005 McGraw-Hill Ryerson Limited
GSR Devices
© 2005 McGraw-Hill Ryerson Limited
Brain wave activity (EEG)
Measures brain wave activity
1. Alpha activity is bad
• Associated with
Inactivity, resting & sleeping
• So: avoid ads with high alpha
2. Hemispheric lateralization
• Distinguishes between activity in the
different hemispheres of the brain
Tells you what people are attending to
© 2005 McGraw-Hill Ryerson Limited
Cap
© 2005 McGraw-Hill Ryerson Limited
© 2005 McGraw-Hill Ryerson Limited
© 2005 McGraw-Hill Ryerson Limited
© 2005 McGraw-Hill Ryerson Limited
© 2005 McGraw-Hill Ryerson Limited
Eye Movements
Measures what is looked at
• Method
•Eye camera and advanced
electronics
• Output:
– Eye movements tell what is seen
–And what will be…
•comprehended
•recalled
© 2005 McGraw-Hill Ryerson Limited
Device
© 2005 McGraw-Hill Ryerson Limited
© 2005 McGraw-Hill Ryerson Limited
© 2005 McGraw-Hill Ryerson Limited
Theater tests
– Product preferences
– Interest in the commercial
– Reactions (an adjective checklist)
– Recall of particular aspects
– Interest in the brand
– Continuous (frame-by-frame) reactions
© 2005 McGraw-Hill Ryerson Limited
Theater Test
© 2005 McGraw-Hill Ryerson Limited
Theater Test – Continuous Reaction Measures
1000
Composite 48
900
676
8
800
700
600
500
400
300
200
100
Scores by Total Sample
0
0
2
4
6
8
10
12
14
16
© 2005 McGraw-Hill Ryerson Limited
18
20
22
Theater Test – Recall Measures
© 2005 McGraw-Hill Ryerson Limited
8
Market (Post) Tests
Broadcast Tests
• On-air tests
– TV programs in specific markets
– Day-after recall
• Split Cable studies
– Ad run in one area, but not another
– One ad in one area, different ad in the
other
– Day after recall
– SALES
© 2005 McGraw-Hill Ryerson Limited
1. Please look over these pictures and words from a TV commercial.
(Announcer) Remember Jared from Name? Turns out he has inspired a lot of people. (Singer) …You’re
still looking good. He’ll show you the way. His name is Jared, and he’ll lead you to Name.
2. Do you remember seeing that commercial on TV?
Yes No Not sure
3. How interested are you in what that TV commercial is trying to tell you or show you about what's
being advertised?
Very interested Somewhat interested Not interested
4. How does it make you feel about what's being advertised?
Good OK Bad Not sure
5. Please check any of the following if you feel they describe that commercial.
Amusing
Appealing Familiar Fast moving
Pointless Seen a lot
© 2005 McGraw-Hill Ryerson Limited
Summary
• 4 Basic Types of Ad Research:
– Concept tests
– Pilot tests
– Post tests
• Each has it’s own use
– Use the one appropriate for
what you want to know
© 2005 McGraw-Hill Ryerson Limited
Quote of the day
Advertising people who ignore
research are as dangerous as
generals who ignore decodes of
enemy signals.
• David Ogilvy
(Ogilvy & Mather)
© 2005 McGraw-Hill Ryerson Limited