Advertising Measurement
A Framework for Evaluating
Impact & Results
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
1
Agenda
Context
Considerations For Establishing Objectives & Goals
Capturing Information Systematically
Communicating Insights & Learning
A Frame Of Reference
Appendix
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
Executive Summary
Our measurement approach must readily enable business decisions.
 Comparability is essential for evaluating alternatives
 Throughout campaign
 Across vehicles
 By segment
 By product
 We will need to establish & capture common “Advergraphics” in order to evaluate
relative business results
 Apples to apples comparisons
 Most of the key questions posed by Marketing Leadership require rigorous attribution
capabilities
 While some metrics will be common and/or we will have the ability to
aggregate/disaggregate, others will be unique to a specific vehicle. Some of the
metrics will also roll-up and down between advertising and the “Engagement Cycle”
dashboard.
 Advertising is relatively continuous & so too should be the metrics
 On-going sampling or tracking versus static periodic reads
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
Marketing Dashboards
Best practices of successful marketing dashboards:
 Tie measures to strategy – aggregated & disaggregated.
 Finance
 Customer relationships
 Brand health
 Internal efficiency
 Focus on small set of metrics
 Organizational focus
 Ensure stakeholder buy-in
 People who’s compensation and performance evaluation will be impacted
 Metric owners
 Finance
 Tie to decision making process
 Design dashboard to feed the decision-making process
 Frame #’s within a qualitative context
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
Marketing Measurement Today
In general, companies should spend between 5-10% of their total
marketing budget on measurement.1
Measurement Relative Sophistication
Today’s Capabilities
80%
Insight
Gap
% of US
Companies2
Lo
15%
5%
Activity
Planned & Systemic
Activity
Characteristics
ROI
Questions
Activity & Analyses
• Labor intensive
• Single question or lens
focused
• Labor intensive, low volume
• Advanced analytic tools and
techniques are leveraged
• Premium associated
with data gathering
• Measures are often
defined “on the fly”
• Not limited by need to information
gathering to be systematic
• Addresses questions or analyses
that do not need frequent reexamination
• Campaign-centric data
accumulation and test
design
• Designed to produce primary
measurement and customer
insight
High
Planned & Systemic
• Process intensive
• Well-established success metrics
• Data provides analysis dexterity (ability to answer
multiple questions or be viewed through multiple
lenses)
• Customer-centric and relationship perspective
• Supports single, multiple and ongoing contact streams
• Comprehensive view of a complex business
• Supports strategic decision-making systemically
• Integrated with strategic planning process
• Returns financial results
1Source:
2Source:
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
Boston Consulting Group & Millward Brown
Marketing Leadership Council
Advertising Response Curves By Product
Financial Services has one of the most challenging consumer response curves1
 The complexity of the decision-making process, financial magnitude & the extraordinary influence of
external factors are certainly some of the many contributors to this.
Impulse Products
Trial
Rate
Personal Care
Trial
Rate
% Consumer Awareness
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
Financial Services
Trial
Rate
% Consumer Awareness
% Consumer Awareness
Source: Millward Brown
Response Curves By Vehicle
The same weight produces different patterns in awareness build and decay
% Awareness
Magazine
TV
TV GRPs
Magazine
GRPs
Time
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
Source: Millward Brown
Advertising Effectiveness Framework
Clients are spending a significant amount of money on advertising and needs
measurement insights in order to optimize the impact of these expenditures.
 The Marketing Leadership Team has a need and desire to begin evaluating advertising
effectiveness.
Key Question:
How Might We Think About Evaluating Advertising?
1. Establishing Clear
Objectives &
Goals
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
2. Capturing
Information
Systematically
3. Communicating
Insights &
Learning
Advertising Objectives & Goals
There are many considerations which need to be taken into account when establishing
objectives & goals for an individual advertisement and they may vary based upon the strategic
intent of each campaign.
 Specify which part of the customer engagement cycle the advertisement is striving to influence
 Establish brand objectives (if any) as well as setting specific product objectives
 Create/reinforce or change
 Perception or behavior
 Select a specific target audience
 Prospect or customer
 Emerging, Engaged, Established, High Net Worth, Active Trader
 Understand the context of the campaign as well as the marketplace
 Integrated
stand alone
 Competitive activity
 Leading economic indicators & stock market performance
 Create the learning plan
 Rigorous test design
 Data requirements
 Analytic framework
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
Vehicle Impact Varies Across Engagement Cycle
As the strategy shifts from expansion towards acquisition, the marketing
measurement and infrastructure must evolve.
Customer
Engagement
Cycle
Objectives
Vehicle
TV
Awareness
Consideration
• Breakthrough
4 • Effective Reach
Inquiry
• Relevance
3 • Delivers rich imagery
• Differentiate
• Inform
2 • Product
& personality
Radio
Print
4 • Augments Reach
3 • Reinforces key
& Frequency
messages
2 • More Targeted
3 • Readily enables
reach
Internet
1 • Targeted but ext.
limited reach
Direct Mail
1 • Targeted Reach
1 • Product features/
• benefit
Sales Force
Event/
Sponsorship
1 • Reinforces brand
• Portal proliferation
2 • High level
• Engage
0 • No significant role
image
1 • Reinforces brand
0 • No significant role
image
4 • Detailed explanation
1 • Offer inducement
0 • No significant role
4 • Extremely detailed
4
3 • Ongoing source of
comparisons
2 • Rich info/exp
Expansion
• Activate
features/benefit
• Self directed
• Personalized
4 • Highly targeted
comparisons
Call Center
Investment
message
• Reinforces brand
image
• Enabling tool
• Immediacy
• Personalized
4 • Offer inducement
1
• Call to action
value
• Loyalty mktg vehicle
• Penetrate enhanced
needs set
• Penetrate enhanced
needs set
0 • No significant role
1 • Prospecting
2 • Detailed experience
3 • Personalized offer
4 • Customer experience
1 • Prospecting
2 • Enhanced credibility
4
3 • Share of wallet
3 • Rich customer exp.
4 • Effective reach
1 • Can deliver some
1 • Deliver/sample
1 • Reinforces brand
0 • No significant role
imagery & key
message
experience
• Positive Association
(Halo)
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
• Product
features/benefit
• Can overcome
specific barriers
• Real-time dialing
• Real time dialing
• Depth of knowledge
image
• Relationship may drive
loyalty
End-to-End Measurement
Advertising metrics will need to span the “Engagement Cycle” and serve as inputs to modeling.
TIME
Advertising
Cycle
Metrics
Exposure
Invest
Use/Interaction
Perception
Message
Immediate Relevance
Credibility
Experience
Interest
Status
Salience
Memory
Enhancement
Brand Interaction
• An additional challenge is accounting for the “cumulative” effect of advertising as well as “latency” – Ad
impressions accumulate to form brand equity.
• As relevance increases so does the time horizon for which the advertising has an impact.
• Enhancement is the interaction between experience & expectations where associations are elevated
beyond purely functional attributes to beliefs
bonding.
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
Capturing Information
A “Cradle To Grave” approach is required and the measurement metrics and methodology need
to be consistent throughout
Advertisement Development
Execution
• Ad strategy & theme
– “What”
• Early in creative process
• Qualitative & Quantitative
• May be iterative; pre & post production
• Standardized questions – “How”
– Main message
– Persuasiveness
– Likes/dislikes
• Clear action standards developed in
advance
• Precisely specified target market
• Evaluating copy & vehicles (integrated
campaign)
Evaluation
• Intercept interviews
– Branch
– Call center
– Web Site
• Standardized questions
– Communication clarity
– Relevance
– Appropriateness
– Likelihood of action
• Tracking
– Enjoyment
• Business results
• Return on marketing investment* (ROMI)
– CPI
– CPA
– Etc.
• Brand imagery & equity
• Comparison to objectives
• Media execution
TIGHT LINKAGE
•
•
Awareness Index
Persuasion
•
•
Imagery
Media Plan
•
•
Recall
Message Retention
*Caution: Using ROI as a key performance metric for marketing is dangerous because:
1) It gives excessive importance to the “I”
2) Is easier to achieve by reducing costs than increasing revenue
3) Leads to lower marketing spending and sub-optimal, relative to goals,
performance
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
Source: Marketing Science Institute, “The Long & Short View of Marketing Metrics”, Tim Ambler
Integrated Advertising Measurement
Holistic advertising measurement is an on-going cycle of testing & learning which
requires considerable cross-team collaboration.
• Brand Equity
• Response
Modeling
Brand
Positioning
• Performance
Measurement
• Qualitative &
Quantitative Research
• Customer
Segmentation
• Concept
Testing
In-Market
Monitoring
Measurement
Objectives
Communicatio
n Strategy
• Execution
Insights
• Tracking
Customer
Data Mart
Project
Point of Entry
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
Communicatio
n Assessment
• Pre-Test
Copy/Executio
n
MRD
Advergraphics
To enable robust analytics, companies must capture common “AdverGraphics” across all of its advertising – A
process challenge
 “AdverGraphics” -- Demographic type characteristics which describe in detail the nature of the ad itself AND all
individual placements
Advertisement


•
•


?

?
•
?
•
•
•


?

Size – inches & % of page – Print
Color or B&W – Print
Lenth of ad -- Broadcast
Proportion brand: product
How many products
Which products
Message category
Proof points
Call to action: y/n, desired response
channel(s) & #s/URLs
Pre-test scores
Target audience; prop/cust.,
EM/EN/ES/AT/HNW; SD/V/D Planning doc
Target engagement cycle stage
Behavioral objective
Strategic intent
Creative name, Oasis #
Development cost
Incentive or promotion: y/n & specific
offer
Integrated campaign: y/n & other
vehicles
Broadcast Placement
Print Placement
? Show name/program
? Media type: network, spot, cable, etc.
? Position within commercial break: first,
last, other
? Daypart
? Audience: audited impressions &
demographics
? Audited GRP’s
? Run date(s)
? Cost of airing
 Network TV; Ad position & Audited
Audience
MEETING WITH ARNOLD TO REVIEW
THEIR DONOVAN SYSTEM - Much of
his type of data captured there
Network TV has much more robust data
capture than other broadcast vehicles
 Name
 Publication type
 Newspaper: financial,
national, regional
 Magazine; business,
lifestyle, etc.
 Frequency; daily, weekly,
monthly, quarterly
 Publications Format; size,
color/B&W
? Audited circulation
? Audited GRP’s
? Publication demographics
 Location within publication (timing
• Need to evaluate the option of
incorporating syndicated data such as
Starch/Samms, MRI/Monroe Mendleson,
Neilsen, Arbitron & potentially others
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
varies)
 Drop date
 Cost of insertion
 Currently available
 Planned or readily incorporated
? Likely, pending detailed MRD questionnaire review
Advergraphics
Internet AdverGraphics will vary somewhat based upon which Internet vehicles are used
 In addition to the standard advertisement AdverGraphics outlined on the previous page, for Internet
advertising we will also want to capture the degree of motion and the level of graphic delivery (flash
& HTML)
Paid Search Engine Inclusion
•
•
•
•
•
•
Order in which Co is served up
Page location where it is placed
Ad or link
Size
Raw # of times served up
Proportion of click throughs
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
Email
•
•
•
•
Subject line
Number of recipients
Response rate
HTML, Plain Text or both
Banner
•
•
•
•
•
Size
Flash or other motion; y/n
Triggers for serving
Number of banner ads served
Proportion of click throughs
Attribution
Attribution will manifest itself differently across the continuum of measurement options, however planned &
systemic is what will be required to answer the insightful questions being asked by Marketing Leadership
Activity
Measurement
Characteristics
Advantages
Disadvantages
Activity & Analyses
Measurement
Planned & Systemic
Measurement
• Scale the Call Center pilot
• Institute source coding on all
marcom
• Institute source coding on all
marcom
• Establish business rules for
attribution across touchpoints
• Can answer some of the
“What” happened
• Develop hypotheses about
“Why”
• Ease of implementation
• Short lead time to
implementation
• Can answer all of “What” &
some of “Why”
• Precise understanding of
market drivers
• A frame of reference for
interpreting relative
performance
• Can answer all of the IMM
questions; “What, Why & How”
• Can generalize &
institutionalize learnings
• Able to quantify interactions
between vehicles
• Can’t generalize learnings
• Based entirely on consumer
recall which has been
empirically proven as
unreliable
• Assumes same response
profile across channels
• Can’t generalize learning
• Ignores the cumulative effect
of advertising & attributes
impact entirely to the most
recent marketing event
• Limited understanding of
interaction between marketing
vehicles
• Remains dependent upon
rigorous test design
• Must capture detailed
“AdverGraphics”
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
Insight By Vehicle
A Combination of Media Generally Has A Greater Impact Than Any Individual Media Type.
TV
Radio
• Creative needs to be brand linked
• Awareness responds to GRPs immediately
• Impact per GRP tends to be lower than TV
• Best if enjoyable and well branded
• Position w/in commercial break
• Flighting best when dispersed across
network/cable/spot & daypart
• Benefits greatly from synergies w/TV and
can extend TV campaign
• High level of targeting
• Need 100 GRPs/wk to breakthrough
• Synergistic w/other media particularly when
there’s a schedule overlap
Internet
• Active goal-seeking medium can help ads be
very effective
• Recall on par w/TV1
• Can be effective even without click-through
• ‘Rich Media’ #2 x’s more effective than
banners
Public Relations
• Independent source creates greater
credibility
• Works very effectively as a multiplier for
other messages
Print
• Print w/TV more likely to result in active
consideration than TV alone
• Active consumption; fast wear out of
individual execution
• Works best where reader is interested in
category Otherwise need strong creative
hook to overcome filters
• Messages perceived as more credible & less
pushy
• Interest & prestige can be heightened by
context
Event/Sponsorship
• Tight linkage between event message and
brand
• Can be effectively leveraged when part of
brand multimedia campaign
• TV sponsorship can be as powerful as TV
advertising
• High level of targeting & direct response
• Possible irritation
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
Source: Millward Brown, Real-Time Marketplace Feedback
Differences Between Media
TV
Magazines
Passive
Active/selective
Product Category Effect
Little
Large
Attention-Getting Wearout
Little
Rapid
ST as Ad
LT as Fact
Need For Creative “Hook”
Less
High
Recall of Visual Elements
Well remembered
Less well remembered
Little
High degree
Advertising Exposure
How Stored In Memory
Mental Processing
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
Source: Millward Brown
Brand Health Metrics
Consumer
Engagement
Cycle
Awareness
 Top-of-mind
awareness
• Presence
 Aided awareness
 Familiarity
• Competitive
comparison
 Share of voice
• Effective share of
voice
Consideration

•


Hedonic evaluation
Understand positioning
Recall key copy points
Perceived rational
attributes & benefits
 Functional
 Process
 Consideration
? Relevance
Inquiry
 Cognitive responses &
judgments
• Factual recall
 Perceived emotional
attributes & benefits
 Purchase intent
• Brand Portfolio Impact
– Family
– Category
 Currently available
 Planned or readily incorporated
? Likely, pending detailed MRD questionnaire review
Measurement Considerations:
Objective
Investment




Current usage
Usage trend
Cross-utilization
Frequency of consumer
interaction
 Brand strength
 Product performance
 Attributes
 Advantages
 Deficits
 Customer service
satisfaction
Expansion
•

?
?

Customer loyalty
 Overall
– Key proposition
elements
Likelihood of switching
 Next month
 With incentive
Affinity
Relationship
? Bonded
Brand Advocacy
Brand Adv
Product Adv
Favorable Attitudes
Information & Action
Cognitive Responses &
Consumer Judgments
“Brand Image & Equity”*
Factual Data & Recall
“Brand
Salience/Relevance”
*Note: Brand equity has a strong linear relationship for the next 2 years
with Market Share (R2=5) & EBITDA (R2=.7)
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
Source: Customer Ergonomics Analyses & MSI; “The Long & Short View of Marketing Metrics”, Tim Ambler
Individual Advertisement Metrics
Consumer
Engagement
Cycle
Awareness
•
•
•
•
•
•
•
•
•
•
•
Recall key elements
Content playback
Brand linkage
Communication clarity
Ease of understanding
Awareness index
– Cut Through
– Branded
memorability
– Enjoyment
– Involvement
Effective GRPs/TRPs
Effective “Share of
Voice”
Relative perception of
the amt. of advertising
Interest
Total Communication
Consideration

?
•
•



•
•

Hedonic
Relevance
New news
Difference/
uniqueness
Believability
Trust/ confidence
Reasonable
fees/value
Corp. values &
ethics
Persuasion
Unaided & aided
consideration
Inquiry
• Perceived
Advantage
 Satisfaction
w/customer service
• Likelihood to take
action; call, click,
or visit
 Intention to invest
• Previous
investment
experience
• Knowledgeable
Sales Reps
Investment
• Perceived
advantage
• “Relative” wealth
Expansion
• Brand affinity
• Number of vehicles &
channels
 Frequency of
interaction
• Perception of
relationship
• Confidence & Security
− External Factors
− Personal
− Fidelity
 Currently available
 Planned or readily incorporated
? Likely, pending detailed MRD questionnaire review
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
Unique Internet & Site Visit Metrics
Consumer
Engagement
Cycle
Awareness
 Page & banner
impressions
 Unique visitors –
Cookies 1 machine,
logins for multiple
pc’s
 Total visitors
 Pgs viewed/visit
• Effective GRP’s
Consideration
Inquiry
 Advertising &
promotion click
through
 Downloaded
pages
 Tool usage
 Visitor origin
- Search engine
- Paid Inclusion
- Referral site
• F.S. sites visited
 Time spent/ page
- calculated
 Site registration –
? Visitors by ISP
? Browser used - @
Investment
 Trades
 Value of trade
 Opt-ins
bare minimum
 Repeat visitors
 Topics/categ
visited/session or
/visitor
 Page exited from
 Proportion
inquiries via Web
IP address level
Expansion
 Site registration – In
depth & complete
 Visit frequency
 Personalized Home
Page
 Alerts set up
 Elapsed time
between visits
 Time per visit
 Pages viewed per
visit
 Multi-channel
customers
• Many of the MRD metrics exclude Internet customers. This research will need to be supplemented or revised to reflect these
customers too.
• Need to further explore syndicated metrics as Financial Services frame of reference; ComScore, Nielsen Net Ratings, Jupiter
Media Matrix
 Currently available
 Planned or readily incorporated
? Likely, pending detailed MRD questionnaire review
All metrics #/%/$
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics
Communicating Insights
Communicating Insights Drives Organization Learning
 Sequencing analytics
 Quick read vs. depth
 Standardized
 Deep dive
 Establishing a frame of reference for objective interpretation
 Normative database
 Outside your industry
 Creating institutional knowledge
 Best practices
©2008-9 Customer Ergonomics. Confidential property of Customer Ergonomics