Questions
• I have had some professors who have a
preference on APA style, is the library website a
good source for APA format? Do you have a
particular preference?
• What are the consequences if a researcher does
not behave ethically towards individuals?
• How do you determine the sample size in a
given research study?
• How do you know whether or not the chosen
sample is representative of the target
population?
More Questions
• How do you know if you are inflicting any
psychological harm on subjects in animal
research? (such as in a controlled lab
experiment and not just in observation).
• In the abstract of a research report, do you need
to separate the sections into background,
methods, results, conclusions or can you make it
one paragraph?
• If a research has to ethically release all
information to a participant before an
experiment, how can you ethically run a double
blind study where the participant has no idea
what is being done.
More Questions
• How can the way in which results are
reported become an ‘ethical issue’ of the
research process?
• To do a research, you have to take the
Ethics code test. Do you have to retake
the test for every research
study/experiment that you do or do you
only have to take it once?
Research strategies and
validity
Chapter 6
Dusana Rybarova
Psyc 290B
May 22 2006
Outline
1. Introduction to research strategies
2. Validity and its threats
- internal validity
- external validity
3. Research strategies, research designs
and research procedures
1. Introduction to research
strategies
• research strategy
– reflects the general approach and goals of a
research study
• types of research strategies
– descriptive strategy
– Nonexperimental strategy
– correlational strategy
– experimental strategy
– quasi-experimental strategy
1. Introduction to research
strategies
• Descriptive strategy
– the goal is to describe the state of affairs at the time of the study
– measures variables as they exist naturally
– e.g. 19% of eligible voters participated in the election
• Correlational strategy
– measures two variables, usually as they exist naturally
– the goal of this strategy is to describe a relationship between the
two variables without attempting to explain the cause of the
relationship
– e.g. Are students GPA’s related to their parent’s income?
• Nonexperimental strategy
– Answers questions about the relationship between two variables
by demonstrating a difference between two groups or two
threatment conditions
– E.g. verbal scores of 6-years old boy and 6-years old girls
1. Introduction to research
strategies
• Experimental strategy
– the researcher manipulates one variable (called
independent variable) while observing or measuring a
second variable (dependent variable)
– this is the ‘true’ experiment because independent
variable is manipulated by the researcher (e.g. room
temperature)
– the goal of experimental strategy is to determine
whether a causal relationship exists between two
variables
1. Introduction to research
strategies
• Quasi-experimental strategy
– uses a nonmanipulated variable to define
groups or conditions (e.g. time or age) or pre
and post threatment
– controls other variables as much as possible
– the goal is to obtain evidence in support of a
cause-and-effect relationship
– however, a quasi-experimental strategy can
not unambiguously establish a causal
relationship
2. Validity and its threats
• validity is the standard criterion by which
researchers judge the quality of research
• in this case the concept of validity applies
to an entire research study
• any component of a research study that
introduces questions or raises doubts
about the quality of the research process
or the accuracy of the research results is a
threat to validity
2. Validity and its threats
• Internal validity
– is concerned with factors within the research
study that raise doubts about the results or
the interpretation
– any factor within the study that allows an
alternative explanation for the results is a
threat to internal validity
– e.g. example with room temperature and
performance
2. Validity and its threats
• threats to internal validity
– Extraneous variables
• any variable in a research study other than the two
variables being studied (both systematic and
unsystematic)
– unsystematically changing variables are usually not a
problem
• confounding variable is an extraneous variable that
is allowed to change systematically along with the
two variables being studied (e.g. time of the day in
the temperature-performance study)
2. Validity and its threats
• threats to internal validity
– sources of extraneous/confounding variables
• participant variables
– assignment bias – when the process used to assign different
participants to different threatments produces groups of
individuals with noticeably different characteristics (e.g. one
group is smarter, more motivated)
• environmental variables
– size of room, time of day, or gender of the experimenter
• measurement variables
– practice effects – prior exposure to a measurement procedure
provides participants with additional skills that produce improved
scores (e.g. the same exam)
– fatigue – prior participation tires the participants so that their
scores on subsequent measurements are lower
2. Validity and its threats
• External validity
– concerns the extent to which the results
obtained in a research study hold true outside
the constraints of the study
– Can the results be generalized to other
populations, other settings, other
measurements?
– e.g. can we generalize results from a wellcontrolled laboratory situation to the
uncontrolled chaos of the real world?
2. Validity and its threats
• threats to external validity
– Participants
• characteristics unique to a specific group of participants in a
study may limit ability to generalize the results to individuals
with different characteristics
– e.g. college students, volunteer bias, cross-species
generalizations
– Features of the study
• characteristics unique to the specific procedures used in a
study may limit ability to generalize the results to situations
where other procedures are used
– e.g. novelty effect, reactivity, specifics of the study (masking
experiments, lexical decisions)
2. Validity and its threats
• threats to external validity
– experimenters
• characteristics unique to the specific experimenter
conducting the study may limit ability to generalize the results
to situations with a different experimenter
– e.g. experimenter bias, experimenter characteristics
– measurements
• characteristics unique to the specific measurement
procedure may limit ability to generalize the results to
situations where a different measurement procedure is used
– e.g. sensitization (the impact of being assessed), generality
across different measures (heart beat vs. questionnaire), time
of measurement
2. Validity and its threats
• Balancing internal and external validity
– attempts to increase internal validity can reduce
external validity (laboratory experiments)
– research that attempts to gain a high level of external
validity will often create a research environment that
closely resembles the outside world
– there tends to be a tradeoff between internal and
external validity (if you increase internal validity,
external validity decreases a vice versa)
2. Validity and its threats
• Validity of individual research strategies
– descriptive strategy
• high external validity
• low internal validity
– Nonexperimental strategy
• High external validity
• Low internal validity
– correlational strategy
• high external validity
• low internal validity
2. Validity and its threats
• Validity of individual research strategies
– experimental strategy
• high internal validity
• low external validity
– quasi-experimental strategy
• higher internal validity than descriptive and
correlational studies
• lower internal validity than true experiments
• relatively high external validity
3. Research strategies, research
designs, and research procedures
• Research strategy
– refers to the general approach and goals of the study
• Research design
– general plan for implementing a research strategy
(e.g. group versus individual, same individuals vs.
different individuals, number of variables included)
• Research procedure
– an exact, step-by-step description of a specific
research study (exact involvement of individuals,
measurement of variables etc.)