Concept Testing
Teaching materials to accompany:
Product Design and Development
Chapter 9
Karl T. Ulrich and Steven D. Eppinger
5th Edition, Irwin McGraw-Hill, 2012.
Product Design and Development
Karl T. Ulrich and Steven D. Eppinger
5th edition, Irwin McGraw-Hill, 2012.
Chapter Table of Contents:
1.
Introduction
2.
Development Processes and Organizations
3.
Opportunity Identification
4.
Product Planning
5.
Identifying Customer Needs
6.
Product Specifications
7.
Concept Generation
8.
Concept Selection
9.
Concept Testing
10. Product Architecture
11. Industrial Design
12. Design for Environment
13. Design for Manufacturing
14. Prototyping
15. Robust Design
16. Patents and Intellectual Property
17. Product Development Economics
18. Managing Projects
Product Development Process
Planning
Concept
Development
Qualitative
Concept
Testing
System-Level
Design
Detail
Design
Testing and
Refinement
Production
Ramp-Up
Quantitative
Concept
Testing
Concept Development Process
Mission
Statement
Identify
Customer
Needs
Establish
Target
Specifications
Generate
Product
Concepts
Select
Product
Concept(s)
Test
Product
Concept(s)
Perform Economic Analysis
Benchmark Competitive Products
Build and Test Models and Prototypes
Set
Final
Specifications
Plan
Downstream
Development
Development
Plan
Outline
• Essence of concept testing
• Process for product concepts testing
3/23/2016
5
Nature of concept testing
• Further narrow the set of concepts under
consideration,
– based data gathered from potential customers in
the target markets, rather than the judgments
made by the development team
• Specific Objectives
– Select one from multiple concepts,
– Gather information on how to improve a concept,
and
– Estimate the sales potential of the product
3/23/2016
6
Input and output
• Input to the potential customer
– Prototype
• Output from the potential customer
– Likelihood for the potential customer to buy
the product
– Estimate of how many units of the product
the company is likely to sell
3/23/2016
7
Purposes for Concept Testing
•
•
•
•
•
•
•
•
Go/no-go decisions
What market to be in?
Selecting among alternative concepts
Confirming concept selection decision
Benchmarking
Soliciting improvement ideas
Forecasting demand
Ready to launch?
3/23/2016
8
Concept testing process
1. Define the purpose of the concept testing
2. Choose a survey population and sample
size
3. Choose a survey format
4. Communicate the concept
5. Measure customer response
6. Interpret the results
7. Reflect on the results and the process
3/23/2016
9
Define the purpose (step 1)
• Which of the alternative concepts
should be pursued
• How can the concept be improved to
better meet customer needs
• Approximately how much units are likely
to be sold
• Should the development be continued
3/23/2016
10
Choose a survey population and
sample size (step 2)
1. Sample size varies from a few to thousands
2. Factors affecting the sample size
1.
2.
3.
4.
5.
The stage of product development
Cost to conduct survey
Nature and intent of the survey
Budget (amount) of the development project
How possible to collect the intended information.
3. Possible to structure multiple surveys with
different objectives at different stages.
3/23/2016
11
Choose a survey format (step 3)
• Formats
– Face to face interaction
– Telephone
– Postal mail
– Electronic mail
– Internet (a test site on the internet)
• Each has its pros and cons
• Each has its bias.
3/23/2016
12
Communicate the concept (step 4)
• Communication means
– Verbal description
– Sketch
– Photos and renderings
– Storyboard (a series of images shown a temporal sequence
of actions involving the products)
– Video (allowing more dynamic than the story board)
– Simulation
– Interactive multimedia (video and simulation)
– Physical appearance model (looks-like)
– Working prototypes (works-like)
• Survey formats vs. means, page 154
3/23/2016
13
Measure customer response (step 5)
• Measurement
–
–
–
–
–
–
–
–
–
Mere their preferences among alternative concepts
Understand why and how they respond to the product concepts
Attempt to measure purchase intent (the likelihood of buying)
But avoid aggressively promoting the product concepts
The solution space?
Alternative function diagrams?
Alternative ways to decompose the problem?
Additional external resources?
All ideas generated and integrated?
• Survey form, page 156.
3/23/2016
14
Interpret the results (Step 6)
•
•
Q=NxAxP
– Where P = Cd x Fd + Cp x Fp
• Q = the quantity of the expected sales
• N = the number of potential customers expected to buy
• A = the fraction of these potential customers aware of the product and
the product is available
• P = the the probability that the product is purchased if the customer is
aware of it and it is available.
• Fd = the the fraction of survey respondents indicating that they would
definitely purchase
• Cd = the percentage that those in Fd will actually buy (.1-.5)
• Cp = the percentage that those in Fp will actually buy (0-.25)
Be aware that sales also depends on
– Words of month
– Fidelity of the concept description
– Pricing
– Level of promotion
3/23/2016
15
Market sizes
• Population and demographic data
• Sales volume of various products
–
–
–
–
–
–
–
3/23/2016
Airplanes
Machine tools
Cars
Hand tools
Printers
Ball pens
Razor blades
16
Concept Testing Example:
emPower Electric Scooter
Scooter Example
• Purpose of concept test:
– What market to be in?
• Sample population:
– College students who live 1-3 miles from
campus
– Factory transportation
• Survey format:
– Face-to-face interviews
Communicating the Concept
•
•
•
•
•
•
•
•
•
Verbal description
Sketch
Photograph or rendering
Storyboard
Video
Simulation
Interactive multimedia
Physical appearance model
Working prototype
Verbal Description
• The product is a lightweight electric scooter that can
be easily folded and taken with you inside a building
or on public transportation.
• The scooter weighs about 25 pounds. It travels at
speeds of up to 15 miles per hour and can go about
12 miles on a single charge.
• The scooter can be recharged in about two hours
from a standard electric outlet.
• The scooter is easy to ride and has simple controls
— just an accelerator button and a brake.
Sketch
Rendering
Storyboard
3D Solid CAD Model
Appearance Model
Working Prototype
Beta Prototype
Video
Animation
Interactive Multimedia
Live Demonstration
Survey Format
• PART 1, Qualification
– How far do you live from campus?
• <If not 1-3 miles, thank the customer and end interview.>
– How do you currently get to campus from home?
– How do you currently get around campus?
• PART 2, Product Description
– <Present the concept description.>
Survey Format
• PART 3, Purchase Intent
– If the product were priced according to your
expectations, how likely would you be to purchase
the scooter within the next year?
I would
definitely not
purchase
the scooter.
I would
probably not
purchase
the scooter.
I might
or might not
purchase
the scooter.
I would
probably
purchase
the scooter.
I would
definitely
purchase
the scooter.
“second box”
“top box”
Survey Format
• PART 4, Comments
– What would you expect the price of the scooter to
be?
– What concerns do you have about the product
concept?
– Can you make any suggestions for improving the
product concept?
• Thank you.
Interpreting the Results:
Forecasting Sales
Q=NxAxP
•
•
•
•
Q
N
A
P
= sales (annual)
= number of (annual) purchases
= awareness x availability (fractions)
= probability of purchase (surveyed)
= Cdef x Fdef + Cprob x Fprob
“top box”
“second box”
Forecasting Example:
College Student Market
•
•
•
•
•
N = off-campus grad students (200,000)
A = 0.2 (realistic) to 0.8 (every bike shop)
P = 0.4 x top-box + 0.2 x second-box
Q=
Price point $795
Forecasting Example:
Factory Transport Market
• N = current bicycle and scooter sales to
factories (150,000)
• A = 0.25 (single distributor’s share)
• P = 0.4 x top-box + 0.2 x second-box
• Q = 150,000 x 0.25 x [0.4 x 0.3 + 0.2 x 0.2]
= 6000 units/yr
• Price point $1500
emPower’s Market Decision: Factory Transportation
Production Product
Sources of Forecast Error
•
•
•
•
•
Word-of-Mouth Effects
Quality of Concept Description
Pricing
Level of Promotion
Competition
Discussion
• Why do respondents typically overestimate
purchase intent?
– Might they ever underestimate intent?
• How to use price in surveys?
• How much does the way the concept is
communicated matter?
– When shouldn’t a prototype model be shown?
• How do you increase sales, Q?
• How does early (qualitative) concept testing
differ from later (quantitative) testing?
Other Images
Issues to discuss
• Why do respondents typically overestimate purchase
intent?
– Might they ever underestimate intent?
• How to use price in surveys?
– Or how much would the customer be willing to pay?
• How much does the way (the concept is communicated)
matter?
– When shouldn’t a prototype model be shown?
• How do you increase sales?
• How does early (qualitative) concept testing differ from
later (quantitative) testing?
3/23/2016
42
Sources of Forecast Error
•
•
•
•
•
Word-of-Mouth Effects
Quality of Concept Description
Pricing
Level of Promotion
Competition
3/23/2016
43