Quantitative Research
Designs
1
Questions for Thought
What is the difference between experimental,
quasi-experimental, and nonexperimental?
2
Research Design
• Research Design
– It is the researcher’s overall plan for
• Answering the research question
• Testing the research hypotheses
3
Research Design
• Research Design
– Will involve decisions regarding
• Will there be an intervention
• What types of comparison will be made
• What procedures will be used to control extraneous variables
– Extraneous are variables that may affect the independent
or dependent variables, and need to be controlled
• When and how many times will data be collected from study
participants
• In what setting will the study occur
4
Types of Quantitative Research
Designs
1. Experimental
2. Quasi-experimental
3. Non-experimental
5
Experimental Research Designs
• Experimental Designs
6
Experimental Research Designs
• Greatest amount of control over the independent
variable
• Researcher is an active agent rather than a
passive observer
7
Experimental Research Designs
• Characteristics of
Experiments
– Manipulation
– Control
– Randomization
• Need to have these
three factors to be
considered an
experimental design
8
Characteristics of Experiments
– Manipulation
• The researcher does something to study participants
• Independent variable is manipulated by administering an
experimental treatment or intervention to some participants
and withholding it from others or administering another
treatment (control group)
• The researcher controls and varies the independent variable
and observes its effect on dependent variable
9
Characteristics of Experiments
– Control
• The researcher introduces controls over the experimental
situation through use of control groups
• Control group is the participants in an experiment that do not
receive the experimental treatment and whose performance
provides a baseline against which the effects of the treatment
can be measured
• The researcher compares the control group’s performance
on a dependent variable to the experimental group
10
Characteristics of Experiments
– Randomization (random assignment)
• The researcher assigns study participants to control or
experimental groups randomly
• Each participant has an equal chance of being included in
any group
• Each group is considered to be comparable therefore any
changes could be attributed to the independent variable
(treatment)
11
Experimental Designs
• Types:
–
–
–
–
–
1. After-only (post-test-only) design
2. Before-after (pretest-posttest) design
3. Factorial Design
4. Repeated-Measures Design
5. Clinical Trials
12
Experimental Designs
1. After-only (post-test-only) design
– Two groups
– Collection of data after intervention
13
Experimental Designs
2. Before-after (pretest-posttest) design
– Collects baseline (pretest data) data before
intervention
– Then collects data after the intervention (post-test
data)
14
Experimental Designs
3. Factorial Design
– Manipulation of two or more variables simultaneously
– Allows evaluation of main effects (effects resulting from
the intervention) and interaction effects (effects
resulting from combining the treatment methods)
– Participants are assigned at random to a combination
of treatments
15
Experimental Designs
4. Repeated Measures Design (crossover design)
– Within-subjects designs
• A research design in which a single group of subjects is
compared under different conditions or at different points in
time (i.e. before and after surgery)
– The same study participants are used to evaluate more than one
treatment/intervention
– Participants are randomly assigned
– Participants serve as their own control group
16
Experimental Designs
5. Clinical Trials
– Involves the testing of a clinical treatment
– Random assignment of participants to experimental or control
groups
– Large sample, can be across the world to increase
generalizability
– Usually use a before-after or after-only design
17
Advantages/Disadvantages of
Experimental Design
• Advantages
– Most powerful for testing
cause and effect
hypotheses
• Disadvantages
– Some variables can not be
manipulated
– Not feasible
– Not ethical
– Potential of Hawthorne
effect
• Being in a study causes
people to change their
behaviour
18
Advantages/Disadvantages of
Experiments
• To reduce the potential for the Hawthorne effect
– Researchers use double-blind experiments
• Neither researcher nor participant know who is getting which
treatment
19
Quasi-Experimental Research
Designs
• Quasi-Experimental Designs
20
Quasi-experimental Research
Designs
• Involves manipulation of an independent variable
• Lacks either randomization or control-group or both ***
• Weaker than experimental designs
• Uses the term comparison group instead of the term
control group
• Also known as pre-experimental design
21
Quasi-experimental Research
Designs
• Types:
– 1. Nonequivalent Control-Group Design
– 2. Time-Series Design
22
Quasi-experimental Research
Designs
1. Nonequivalent Control-Group Before-After
Design
– Most frequently used quasi-experimental design
– Involves a treatment and two or more groups of participants
– Collects data before and after intervention
– No randomization
• Therefore the groups can not be assumed to be equivalent
23
Quasi-experimental Research
Designs
2. Time-Series Design
– Has neither a control group nor randomization
– Involves the collection of data over an extended period of time
and the introduction of the intervention during that period
– Therefore data is collected before the intervention (multiple
collection points) and again after implementation (multiple
collection points)
24
Advantages/Disadvantages of
Quasi-Experiments
• Advantages
– Practical
• Disadvantages
– Cause and effect can not
be determined as easily as
experimental designs
– Could be other reasons for
the change in dependent
variable (rival hypotheses)
25
Nonexperimental Research Designs
• Nonexperimental Designs
26
Nonexperimental Research Designs
• Used where the independent variable can not be
manipulated
• Used when it is unethical to manipulate the
independent variable
• Useful in descriptive studies
27
Nonexperimental Research Designs
• Types:
– 1. Ex post facto (correlational)
– 2. Descriptive
28
Nonexperimental Research Designs
1. Ex post facto (correlational)
– Research is conducted after the independent variable
has been manipulated or intervention applied
– Studys relationships among variables
– No control of independent variable
– Can be retrospective or prospective studies
29
Nonexperimental Research Designs
– Retrospective
• Looks at dependent variable in the present and attempts to
link this effect to cause in the past
• Looks at present outcomes and tries to determine what
factors caused it
– Lung cancer currently, linked to smoking in the past
– Prospective
• Looks at the presumed cause and then goes forward in time
to observe presumed effects
• Considered stronger than retrospective studies
– Smoking currently may cause lung cancer in the future
30
Nonexperimental Research Designs
2. Descriptive Design
– Purpose is to observe, describe and document aspects
of a situation
31
Advantages/Disadvantages of
Nonexperimental Research Designs
• Advantages
– Good for problems not able
to be studied through
experimentation
• Disadvantages
– Can’t determine causal
relationships conclusively
32
Research Design and Time
• When to use multiple points of data collection
– Time-related processes
• Phenomena evolves over time
• i.e. healing, growth
– Time-sequenced phenomena
• Sequencing of phenomena
– Comparative purposes
• Compare phenomena over time
– Enhancement of research control
33
Research Design and Time
– Cross-sectional Studies
• Collection of data at one point in time
• Difficult to infer changes and trends over time
– Longitudinal Studies
• Collect data over an extended period of time
• Can show changes over time
34
Research Design and Time
– Longitudinal Studies
• Trend studies
– Different samples from a population are studied over time, always
from same population
• Panel studies
– Same participants supply the data at two or more points in time
• Follow up studies
– Determine the outcomes of people with a specific condition or who
have received a specific treatment
35
Techniques of Research Control
• A major purpose of research design is to
maximize the researcher’s control
over the research situation
36
Techniques of Research Control
• Research Control
– Concerned with eliminating possible extraneous
influences on the dependent variable
• A variable that confounds (confuses) the relationship between
the independent and dependent variables
– Concerned with understanding the true relationship
between independent and dependent variables
– Research control attempts to exclude contaminating
factors, to control the extraneous influences
37
Techniques of Research Control
• Controlling External Factors
– Want to achieve constancy of conditions
• Researcher is confident that situational contaminants or conditions are not
affecting the data
– Want to control the environment
• As environment influences people’s emotions and behaviour
– Want to control the time
• Maintain constancy of time, control time of day, time of year
– Research protocols
• Specific procedures are outlined to maintain consistency
38
Techniques of Research Control
– Controlling Intrinsic Factors
• Intrinsic factors are participant's characteristics
– Age, gender, habits
39
Techniques of Research Control
– Methods of Controlling Intrinsic Factors
• Randomization
– Equalized groups in relation to extraneous variables
– The best, most effective method
• Homogeneity
– Only participants who are the same with respect to
extraneous variables are included in the study
• Matching
– Using information about participants’ characterisitcs to
form comparison groups
• Statistical Control
– Statistical analysis
40
Internal and External Validity
• Internal Validity
– The extent to which it is possible to make an inference that
the independent variable is truly influencing the dependent
variable
– Quasi-experimental, preexperimental and correlational studies
are especially susceptible to threats to internal validity
– Experimental designs usually reduce threats to internal validity
41
Internal and External Validity
• Threats to Internal Validity
–
–
–
–
History threat
Selection threat
Maturation threat
Mortality threat
42
Internal and External Validity
• Threats to Internal Validity
– History threat
• Refers to events external to the intervention but occur at the
same time which can affect the dependent variable
• Threat in quasi-experimental and non-experimental designs
• Not usually threat in experimental studies as all groups will
be affected by history
43
Internal and External Validity
• Threats to Internal Validity
– Selection threat (self-selection)
• Results from pre-existing differences between groups which
may affect the dependent variable
• Groups may not be equivalent
• Most problematic threat to studies not using experimental
design
44
Internal and External Validity
• Threats to Internal Validity
– Maturation threat
• Occurs when changes to the dependent variable (outcome)
results from the passage of time
45
Internal and External Validity
• Threats to Internal Validity
– Mortality threat
• Refers to the loss of participants (attrition) from the different
groups in the study
46
Internal and External Validity
• External Validity
– Refers to the generalizability of the research
findings to other settings or samples
– Does intervention work in another setting or with
different people
– A good sampling design increases chance of
generalizability
47
Reference
Loiselle, C. G., Profetto-McGrath, J., Polit, D. F., &
Beck, C. T. (2011). Canadian essentials of nursing
research (3rd ed.). Philadelphia, PA: Lippincott,
Williams & Wilkins.
48