Week 12:
Quantitative
research designs
SOWK 621: DeCarlo
Contents
• Video mini-lectures
• Survey design
• Experimental design
• Design & methods
• Virtual office hours
• Questions
This Photo by Unknown author is licensed under CC BY-NC-ND.
Survey design
Video mini-lecture
Design: What your study will do
• You’ve read your literature (lit review)
• You’ve created your question (research question)
• You’ve got your people (sampling)
• You’ve figured how to measure your variables (measures)
• Now you need to figure out what you’ll do with your participants when they come into your
study
• This is called design
• Quantitative: experiment, survey, secondary data analysis, chart reviews, etc.
• Qualitative: interview, focus group, secondary data analysis, etc.
Strengths
Strengths and
limitations of
surveys
Limitations
Cost-effectiveness
Inflexibility
Generalizability
Lack of depth
Reliability
For cross-sectional
designs, difficulty
with time order
Versatility
Longitudinal designs
are great
Phone, mail,
internet, and inperson surveys all
have issues
Time
Time &
administration:
Types of
surveys
• Longitudinal: measure people over
time
• Trend, panel, cohort, retrospective
• Cross-section: measure only one time
Administration
• Phone
• In-person
• Electronic
• Mail
• BRUSO: “brief,” “relevant,” “unambiguous,” “specific,”
and “objective.”
• Questions are based on operational definitions
• But also include other variables and characteristics
Questionnaires:
A tool for
surveys
• Concise, easy to understand
• Address the “most knowledgeable people” about a
topic
• Think back to sampling
• Clear wording
• Double negatives, double-barreled questions and
answers, jargon, slang
• Neutral wording
• Leading language, social desirability
• Pretesting is key
Questionnaires:
A tool for
surveys
• Closed-ended questions
• Mutually exclusive and exhaustive
response options
• Fence-sitters and floaters
• Filter questions, Matrix questions
• Group your questions by theme
• Ordering is important, though tricky
• Think about the time needed to complete
the questionnaire
• Look professional
Experimental
design
Video mini-lecture
Experimental design: Probably not your design
• It is my design, though
• I have an intervention to test
• Program evaluation is similar to experimental design
Experimental design: Step by step
Logic of experimental design
• Maximize internal validity
• Changes in DV are the result of IV (intervention)
• Control for confounding/extraneous variables
• Laboratory settings, treatment fidelity
• Consistent treatment
• Control vs. experimental group
• Comparable groups
• Random assignment
• Avoid selection bias
• Blinded
• Avoid placebo effect, resentful demoralization, compensation
• Pretest/posttest
• Establish baseline, time order
Experimental designs vary in rigor
• Classic experimental designs (very rigorous)
• Random assignment, control & experimental group, pretest & posttest
• Many variables (dissembling, alternate treatment, etc.)
• Quasi-experimental designs (somewhat rigorous)
• No random assignment, may use comparison group or match cases
• Time-series design
• Non-experimental designs (not that rigorous)
• No random assignment, may not have comparison group, or pretests
• Still valuable
Design &
methods (Quant)
Video mini-lecture
Let's look at
the Design &
Methods
(proposal)
• I filled it out. And I'll walk you through it.
Virtual office
hours
Design & methods
Agenda
• Calendar
• Questions
• Design & methods
Calendar
• Questions?
Design & Methods
• What have I seen so far?