Research Process (with a
focus on Problem
Formulation)
Jeremy Kees, Ph.D.
Review from last week…
•
Marketing research is the marketer’s link
to understanding the consumer and the
external environment
The main purpose of marketing research
is to inform decisions
•
–
•
Is a source of a competitive advantage for
many successful firms
Every research project is different
–
It often takes many research projects over
many years to really understand a
phenomenon
Ongoing “PharmFirm” Case Study
• Will be used as an example for many of
the concepts we discuss
• Large-scale, multi-study research project
for a major pharma company in the
Philadelphia area
• Key research questions focused on “fair
balance” of a online medical resource
targeted at physicians
– Proactive study in anticipation of pushback
from FDA
Stages
in the
Research
Process
(Researchers
Perspective)
Formulate Problem
Determine Research Design
Design Data Collection
Method and Forms
Design Sample and Collect Data
Analyze and Interpret the Data
Prepare the Research Report
Problem Formulation
• Differentiate between a decision
problem and a research problem
• Try to get beyond an “info request”
Decision Problems
Research Problems
Develop package for a new
product
Evaluate alternative package
designs
Increase store traffic
Measure current image of the store
Increase market penetration
through the opening of new stores
Evaluate prospective locations
Decide which merchandise will be
available for purchase over the
Internet
Determine consumers’ confidence
in purchasing different categories
of products unseen
Research Process: Problem
Formulation
• Perhaps the most important step in the research
process analysis
• A well-defined study begins with a clearly
defined objective
“The formulation of a problem is often more essential
than its solution” - Albert Einstein
• The problem is rarely clear-cut
• Slight variations in research questions can lead to
substantial changes in the research process (so be
careful)
– Drives decisions related to research design,
measurement, sampling, etc.
Problem Formulation
• Differentiate between a decision problem
and a research problem
• Try to get beyond an “info request”
Decision Problems
Research Problems
Develop package for a new
product
Evaluate alternative package
designs
Increase store traffic
Measure current image of the store
Increase market penetration
through the opening of new stores
Evaluate prospective locations
Decide which merchandise will be
available for purchase over the
Internet
Determine consumers’ confidence
in purchasing different categories
of products unseen
Research Process: Problem
Formulation
Stage in the
Process
Typical Questions
Formulate problem
• What is the purpose of the study (i.e.,
to solve a problem? Identify an
opportunity?)
• Is additional background information
necessary?
• What are specific research questions
and what information is needed to
make the decision?
• How will the information be utilized?
• Has a decision already been made?
• Should research be conducted?
Research Process: Problem
Formulation
PharmFirm
• Longest stage in the process
– ~4 months
– Talked with LOTS of people
• Over-arching problem
– Need evidence that a medical information
website is ‘fairly balanced’
• Unclear what fair balance means in an computer
mediated environment
• Anticipated issues from FDA
Research Process: Problem
Formulation
PharmFirm
• Overarching Research Objective
– provide an empirical test of perceived benefit
and risk information (i.e., fair balance)
presented on the medical information website
Research Process: Problem
Formulation
PharmFirm
• More specific study objectives
– Phase I: Develop a metric for the construct of
fair balance
– Phase II: Test the perceived fair balance of the
website
• Different versions
• Versus print marketing materials
– Phase III: Test the perceived fair balance of
the website versus other websites
Research Process: Problem
Formulation
PharmFirm
• 4 separate studies, each with its own
very specific objectives
• Expected Decisions
– Positive results = move forward
confidently
– Negative or mixed results = adapt
website
Research Process: Determine
Research Design
• Dictated by the problem or research
question
• Exploratory Research
• Descriptive Research
• Causal Research
Research Process: Determine
Research Design
Stage in the
Process
Typical Questions
Determine research
design
• How much is already known?
• Can a hypothesis be formulated?
• What types of questions need to be
answered?
• What type of study will best address
the research questions?
Research Process: Determine
Research Design
• PharmFirm research questions call for….
1. Some pretesting to develop and refine the
metric
2. Experiments
1.
2.
3.
To test versions of the website with different
presentations of risk information
To test the website versus print marketing
materials (a context where fair balance is welldefined)
To test the website versus other existing online
resources that are deemed “fairly balanced”
Research Process: Design
Data Collection & Forms
• Secondary research – not necessary
• Survey Research
– Lots of ways to collect data
• Measurement
– Constructs / Variables
Research Process: Design
Data Collection & Forms
Stage in the
Process
Typical Questions
Determine data
collection method and
forms
Can existing data be used to advantage?
What is to be measured? How?
What is the source of the data?
Can objective answers be obtained by asking
people?
How should people be questioned?
Should the questionnaires be administered in
person, over the phone, or through the mail?
Should electronic or mechanical means be used to
make the observations?
Should structure or unstructured items be used to
collect the data?
Should the purpose of the study be made known to
the respondents?
Should rating scales be used in the questionnaire?
Research Process: Design
Data Collection & Forms
• PharmFirm data collection requires….
1. Scale development and testing
1.
2.
Define construct
Develop valid and reliable measures
2. Stimuli development
1.
Website mock-ups
3. Measurement instrument (i.e., survey)
1.
2.
3.
4.
Manipulation checks
Dependent variables
Individual difference variables
Demographic/categorical information
Research Process: Design
Sample and Collect Data
• Determine your “target market”
• Determine necessary sample size
– Dictated by study design
•
•
•
•
Sampling Methodology
Logistics
Cost
Ethics
Research Process: Design
Sample and Collect Data
Stage in the
Process
Typical Questions
Design sample and
collect the data
• Who is the target population?
• Is a list of population elements available?
• Is a sample necessary?
• Is a probability sample desirable?
• How large should the sample be?
• How should the sample be selected?
• Who will gather the data?
• How much supervision is needed?
• What operational procedures will be
followed?
• What methods will be used to ensure the
quality of the data collected?
Research Process: Design
Sample and Collect Data
• PharmFirm sample call for….
1. ~1000 physicians from three specialty
groups
•
Outsource!!
2. Data collected online
•
•
Sample is too geographically dispersed and time
sensitive to collect data in person
Qualtrics research software
Research Process: Analyze
and Interpret the Data
• Edit / clean / code the data
• Analyze
• Interpret
Research Process: Analyze
and Interpret the Data
Stage in the
Process
Typical Questions
Analyze and interpret
the data
• Who will handle the editing of the
data?
• How will the data be coded?
• Who will supervise the coding and
keypunching?
• What tabulations / statistical tests /
analysis techniques will be used?
Research Process: Analyze
and Interpret the Data
• PharmFirm analysis calls for….
1. Factor analysis to develop metric
•
Dictated by study design
2. Multivariate Analysis of Covariance
(MANCOVA) to analyze experimental data
•
Dictated by study design
Research Process: Prepare
Research Report
• Know your audience
• Sell your findings
Research Process: Prepare
Research Report
Stage in the
Process
Typical Questions
Prepare the research
report
• Who will read the report?
• What is their technical level of
sophistication?
• Are managerial recommendations
called for?
• What will be the format of the
written report?
• Is an oral report necessary?
• How should the oral report be
structured?
Research Process: Prepare
Research Report
• PharmFirm report out calls for….
1. Four separate presentations to the client
2. Final comprehensive report outlining
purpose, research questions, design, sample,
results, etc.
3. Compilation of data in easy-to-use format
•
•
Measurement definitions
Coding schemes
Form Groups and…
Contrast the process we have just
talked about with that of
Andreasen
Problem Formulation (as
discussed in Andreasen)
• “Actionable marketing research”
– Identify the intended action FIRST;
avoid “these are the things that would
be nice to know”
• Exceptions: basic, exploratory research
that lays the groundwork for future
investigation and methodological research
– Close and consistent collaboration
between researcher and client (decision
maker)
Problem Formulation (as
discussed in Andreasen)
“Backward Approach”
1.
2.
3.
4.
5.
6.
7.
8.
How will the research results be implemented? (i.e.,
what action will be taken)
Determine what information will make up the final
report
Specify analyses that will need to be done to “fill in
the blanks” in the report
Determine what kind (format) of data is needed to run
the particular analyses (simple as possible)
Does this data already exist?
If primary data is needed, design sampling strategy
and instruments
Collect data
Fill in the blanks from above
Problem Formulation (as
discussed in Andreasen)
• Try to avoid “interesting” or “nice to know”
questions at all costs
• Lay out the action alternatives on the front end
• What key information in the final report will be
used to make decisions one way or another?
• Present the client with hypothetical scenarios to
ensure that the information is actionable
– This often leads to more questions that the client hadn’t
thought of previously
– It also helps the client:
•
•
•
•
commit to the research
deepen their understanding of the nature of the research
minimize surprises
recognize limitations
Andreasan (1985) Take-Aways
• This “backward” process helps us
avoid research that does not lead to
action (or directly benefit decision
making)
• The emphasis on close collaboration
with the client helps to ensure
maximum usefulness and minimum
surprises
TEAM ASSIGNMENT #1
•
Discuss some issues that you deal with at
your workplace that can be answered
with primary research
1. Discuss the issue broadly (i.e., provide some
context)
2. Develop several clear, concise research
questions
3. Develop hypotheses (and null hypotheses)
4. Discuss the decisions that would be made
depending on the different outcomes of the
research
Hints….
• Try to acutely identify the problem at hand (and
avoid “nice to know” questions)
– What information is necessary in order to make a
decision?
• Determine what course(s) of action will result
from your findings
– If we find A, what will you do?
– What about if we find B?
– Make all alternative courses of action explicit