Pros and Cons of Measuring
Advertising Effectiveness
Reasons to Measure
Ad Effectiveness
Reasons Not to Measure
Effectiveness
 Avoid costly mistakes
 Evaluate alternative
strategies
 Increase efficiency of
advertising in general
 Determining if objectives
are achieved
 Cost of measurement
 Problems with research
 Disagreement about
what to test
 Objections of creative
personnel
 Lack of time
Evaluating Creatives*
 Concept Testing: Assessment of potential
creatives
 Communication Research: A Look at the
Advertising Creative
 Copy Testing: Assessment of creative
achievement
*
The word creative when used as a noun refers to an
actual advertisement
Concept Testing
 A concept is similar to the actual ad, but
more succinct without a lot of selling and
posturing
 The concept states in clear terms the
selling premise, unique position or benefit
of the advertised product or service
 We test the concepts in order to determine
which of a set of concepts would be the
most useful in accomplishing our objective
 See handouts to see the difference between
a concept and the corresponding creative
Types of Concept Tests
 New Product Concept Tests
 Generate concepts for multiple new product
possibilities and test the concepts to assess which
new product has the most potential for success
 Benefit Positioning Concept Tests
 Generate multiple concepts for one product to test
which concept has the most potential for generating
desired outcome, generally purchase intent
 Each concept will focus on a different benefit
(position)
 Example: concept test for a new PDA
 concept 1 focuses on ease of use
 concept 2 focuses on technical capability
General Outline for Performing
Concept Testing




Generate Concepts
Generate Questionnaire
Administer the Test
Organize, analyze, interpret data
Concept Generation
 Concepts are typically presented on
8.5”X11” cardstock or board, and are
sometimes called “white card” tests
 Card contains:
 Headline
 Body copy
 Illustrations
White Card Headline
 This is NOT the same as an advertising headline
 A succinct summary of the main selling proposition or
product benefit
 Leads into the body copy
 Gives consumer a clear understanding of the ideas
that are to be presented in the body copy
 Use simple, single-minded declarative statements
 “The new GeoNET PDA integrates many wireless
technologies so you can do more when you travel.”
Concept Body Copy
 Be direct and clear, using short paragraphs
and active, declarative statements
 Do not use:
 Advertisingese; do not embellish
 Industry jargon that is unfamiliar to the
audience
 Overwritten or long concepts
 If new product concept, you should also
 Identify product category membership
 List all primary benefits
 Identify differentiating attributes
Concept Body Copy
The new GeoNET PDA integrates a wireless phone, GPS
receiver, and Blackberry wireless technology with
512MB of internal memory and coast to coast Web
access.
With all of these features in one convenient device,
working while traveling is easier than carrying
multiple devices to accomplish the same tasks.
The GeoNET PDA is available with either Palm OS or
Windows Mobile for even more flexibility
Suggested retail prices: $299 with black and white
display; $449 with 16-bit color display
Concept Generation Considerations
 Using Brand names is generally accepted as
concept perceptions are often impacted
greatly by brand name presence
 Pricing information should be present
 If not in the concept itself, integrate it into the
questionnaire
 Price does not need to be explicitly stated, but if
it is, it should accurately reflect the actual
expected price
 Illustrations and format between tested
concepts should be as identical as possible
Generate Questionnaire
 Demographic and brand usage questions
 Diagnostic questions pertaining to specific
characteristics of the concept
 Core questions







Communication of the main idea
Believability of the main idea
Uniqueness of the main idea
Personal relevance of the main idea
Purchase Intent
Purchase frequency
Reasons for purchase intent and purchase
frequency
Concept Test Administration





Target definition
Research methodology
Questionnaire construction
Concept exposure
Data analysis and interpretation
Data Organization, Analysis and
Interpretation
 Split sample by respondent characteristics
(quota groups) and assess differences
between concepts
 Split sample by concepts and assess
differences between quota groups
 Crosstabs on core questions by purchase
intent, brand usage, and communications
 Correlation matrix between believability,
uniqueness, and relevance
Group Practice Time
 Concept Testing Handout
 Read through the description of the
study and purpose of the research
 The actual concept and a variety of
results tables are provided for you
 Answer the 4 questions at the end of
the handout using the results
provided
Communication Research
 After concept testing, we have identified
the concept, that is, the main idea that is to
be communicated in the advertisements
 Now we must develop the actual
communications, the ads, and determine if
we are communicating our ideas effectively
 Basic goal is to determine how consumers
react to the ad and why they are reacting
that way
General Outline for Communication
Research





Create test ads
Specify Respondent Characteristics
Select Methodology
Generate Questionnaire
Analyze and Report findings
Creating the Test Ads
 Print ads should be created to the semi-comp stage,
meaning it is full size, with color and art in place and
scaled to size
 May not be exactly how ad will look in print, but
should be close
 Radio ads can be scratch tracks, meaning that all
aural elements are there, but recording may not be
professional quality yet
 TV ads may be storyboarded as long as the ad comes
across to the audience in the correct way
 May use animatics, photoboards, or photomatics if
needed to get across important visual cues
 In all cases, copy should be precise
Generate the Questionnaire
 Brand usage, demography, etc.
 Core questions to assess:
 Whether the respondent accurately processed
the information presented
 Respondent attitudes toward the
communication, i.e. the message was well
received
 Should probe cognitive and emotional
responses
 Respondent attitudes toward the ad itself
 Questions specific to the particular creative
Analysis and Interpretation of
Communication Research Data
 Typically standard tests (t-tests, chisquare tests) are all that are needed
to describe the results
 Relationships between variables can
be interesting, but is more difficult
and requires some creativity on the
part of the researcher since no
hypotheses are developed for testing
Copy Testing
 Communication research is performed during creative
development and is diagnostic in nature
 Copy testing takes place after creative developments
using actual ads, and is evaluative in nature
 Independent marketing research companies specialize
in copy testing, and have established a set of norms
upon which go/stay decisions are made
 The majority of copy testing is done on television
advertising, although options exist for print, radio,
and outdoor advertising as well
 Television advertising is far more expensive to create
and run than other media, hence the abundance of tv
copy testing
Four key decisions to make
 Place: in home viewing or out of home
 Naturalness: forced versus unforced
exposure
 Number of exposures before data collection
 Key measures:






Recall
Message communication
Attitude change
Purchase intention
Ad attitude
Et cetera
Typical structure of Television
Copy Test



Show a sample of 200 respondents a 30 minute program with test
ad imbedded
Program is broadcast on an unused cable channel so respondents
may view the program in their homes
The day after viewing, respondents are contacted and asked to
name the commercials they remember seeing during the program



If unable to name the commercials, they are prompted with correct and
incorrect answers
This tests both aided and unaided recall
Purchase intent can be ascertained with a pre-post exposure
measurement



Prior to watching, respondents are asked what brand they are likely to
purchase next time
After exposure, respondents are asked what brand they would prefer to
win
Asking the same preference question in a different context masks the
purpose of the question to ensure accurate responses
Other Key Measures




Intrusiveness: measure of the ads ability to break through
the clutter and earn a place in the consumer’s mind



Unaided recall – correct brand
Proved unaided recall
Proved brand aided recall

The Lead Idea is the idea that is most often cited by
respondents
Communication: identifies the ideas that consumers got out
of the ad
Persuasion: ad’s ability to generate favorable brand
attitudes


Purchase intent
Standard brand attitude measure

Standard ad attitude measures
Commercial Reaction
Typical Structure of a Magazine
Copy Test
 125 respondents are recruited through mall intercepts
 They are asked questions about media and product
usage
 They are provided a fake magazine with test ad
imbedded and asked to quickly flip through and give
initial reactions
 Ask respondents what ads they remember seeing
 Have them look through magazine again, for a longer
time this time
 Now ask recall, persuasion and communication
questions
 Direct respondents to the test ad and ask them to
read it
 Ask customized communication and diagnostic
questions
Typical Measures in Print Copy
Testing






Recall/Recognition
Idea Communication
Brand Attitude
Ad Attitude
Purchase Intent
Stopping Power
 Noted Readers: only remember seeing it
 Associated Readers: read part of it
 Read Most Readers: read more than half of it
Copy Testing Notes
 Methodology should be as close to
“real world” as possible
 Using multiple exposures and
collecting different data after each
exposure allows identification of long
term effects
 Responses are more valid when
exposure occurs without respondent
knowledge of participation
When Using Outside Firms
Research Services
 Some measures between companies may
have similar names, but measure
completely differently. Understand the
method completely so you can truly
ascertain the meaning of the data
 Know what the “norms” are, and be sure
that they are current and appropriate to the
product
 Ensure that measures used by research
firm are reliable and valid; if firm does not
share, or claims to not know, reliability and
validity diagnostics, be wary of results