EDF 802
Dr. Jeffrey Oescher
Principles of Research Design
Revised 30 January 2014
I.
II.
The purposes of research design
A.
Provide credible answers to research questions
1.
Different research questions are answered by different research designs
2.
Descriptive questions like ascertaining the organizational health of a group of
schools or determining the factors related to teachers leaving the profession are
examples of questions that would use survey or causal comparative designs
3.
Questions about relationships such as the organizational health of high and low
performing schools or school climate and the leadership style of principals are
best answered using correlational designs (e.g., comparative, simple
correlational, multiple regression)
4.
Questions about the causal effects of specific programs or instructional
approaches can only be effectively answered using experimental group
comparison designs
B.
Control variance
1.
Controlling variance is discussed in terms of influencing the variation in the
scores for the dependent variable (DV)
2.
Three fundamental principles
a)
Maximize experimental variation
b)
Minimize error variance
c)
Control extraneous variation
d)
MAXMINCON
The three principles of research design
A.
MAXimize experimental variation
1.
Variation in the DV influenced by the independent variable (IV)
2.
One way to maximize experimental variance - design experimental conditions as
different as possible so any differences in the dependent variable can manifest
themselves
a)
Choose holistic and phonetic approaches to teaching reading
b)
Choose transformational and transactional leadership styles
B.
MINimize error variance
1.
Variation in the DV due to random fluctuations related to individual differences,
measurement error, the reliability of instruments, guessing, inattention, fatigue,
etc.
2.
Two ways to minimize error variance
a)
Reduction of errors through controlled conditions associated with the
study
b)
Increase the reliability of the instruments
3.
Reducing error variance results in increasing the probability of seeing systematic
variance (i.e., that variance attributed to the IV)
C.
CONtrol extraneous variance
1.
Variation in the DV due to the influence of IVs extraneous to the purpose of the
study
a)
The effectiveness of an instructional strategy could depend on the
academic level of the students being exposed to the program
b)
The effectiveness of a leadership strategy could depend on the gender of
those being exposed to it
2.
Five ways to control extraneous variance
a)
Eliminate all but one level of the extraneous variable from the study
(i)
Study only males or females, but not both
(ii)
Always results in a loss of generalizability
1
b)
c)
d)
e)
Randomly assign subjects to groups and groups to treatments
Build the extraneous variable into the design as an attribute variable in a
factorial design
Match subjects
(i)
The matching variable must be related to the DV
(ii)
Split the extraneous variable into two or more levels and
randomly select subjects from within each level
(iii)
Once subjects are matched randomly assign matched subjects
to treatments
(iv)
Matching often is very difficult to accomplish, especially when
matching on more than one variable
Statistically control the influence of an extraneous variable
(i)
Research design and statistical control are intimately related
(e.g., factorial designs, repeated measures, ANCOVA,
regression)
2