Educational Research:
Data analysis and interpretation – 2
Inferential statistics
EDU 8603
Educational Research
Richard M. Jacobs, OSA, Ph.D.
Statistics...

A set of mathematical procedures for
describing, synthesizing, analyzing,
and interpreting quantitative data
…the selection of an appropriate
statistical technique is determined
by the research design, hypothesis,
and the data collected

inferential statistics...
…mathematical tools that permit the
researcher to generalize to a
population of individuals based
upon information obtained from a
limited number of research
participants

sampling error...
…the differences in samples due to
random fluctuations within the
population
…sampling errors vary in size
…but are normally distributed around
the population mean (M)
…and take the shape of a bell curve

standard error...
…the standard deviation of the sample
means (SEx)
…tells the researcher by how much
the researcher would expect the
sample means to differ if the
researcher used other samples
from the same population

but...
…the researcher does not have to
select a large number of samples
from a population to estimate the
standard error

a mathematical formula can be used to
estimate the standard error...
SD
.
SEx = √ N - 1
…a smaller standard error indicates
less sampling error
…the major factor affecting the size
of the standard error of the mean is
sample size
…but, the size of the population
standard deviation also affects the
standard error of the mean
The null hypothesis (H0)...

the statement that the difference
between two sample means is due to
random, chance, sampling error
…indicates that there is no true
difference or relationship between
parameters in the populations

the null hypothesis differs in most
instances from the research
hypothesis (H1)
…which states that one method is
expected to be more effective than
another

rejecting the null hypothesis
provides evidence (but not proof)
that the treatment had an effect
…in other words, that the difference
between dependent variables is
due to something other than
random, chance, sampling error

The research question, then, is:
…whether to accept the null
hypothesis or to reject it

There are four possibilities:
1. The null hypothesis is true and the
researcher concludes that it is true
A = B…a correct decision
2. The null hypothesis is false and the
researcher concludes that it is false
A ≠ B…a correct decision
3. The null hypothesis is true but the
researcher concludes that it is false
A = B…an incorrect decision
4. The null hypothesis is false but the
researcher concludes that it is true
A ≠ B…an incorrect decision
The researcher’s decision
about the null hypothesis…
Decisions concerning rejecting the
null hypothesis…
The true status of the null hypothesis…
True
False
True
Correct
Incorrect
False
Incorrect
Correct
The researcher’s decision
about the null hypothesis…
Decisions concerning rejecting the
null hypothesis…
The true status of the null hypothesis…
True
False
True
Correct
Type II
Error
False
Type I
Error
Correct

researchers use a test of significance
to determine whether to reject or fail
to reject the null hypothesis
…involves pre-selecting a level of
probability, “α” (e.g., α = .05) that
serves as the criterion to determine
whether to reject or fail to reject the
null hypothesis
Steps in using inferential statistics…
1. select the test of significance
2. determine whether significance test
will be two-tailed or one tailed
3. select α (alpha), the probability level
4. compute the test of significance
5. consult table to determine the
significance of the results
Tests of significance...

statistical formulas that enable the
researcher to determine if there was
a real difference between the sample
means
…different tests of significance account
for different factors including: the scale
of measurement represented by the
data; method of participant selection,
number of groups being compared, and,
the number of independent variables
…the researcher must first decide
whether a parametric or
nonparametric test must be selected

parametric test...
…assumes that the variable measured
is normally distributed in the
population
…the data must represent an interval
or ratio scale of measurement
…the selection of participants is
independent
…the variances of the population
comparison groups are equal
…a “more powerful” test in that it is
more likely to reject a null hypothesis
that is false, that is, the researcher is
less likely to commit a Type II error
…used when the data represent a
interval or ratio scale

nonparametric test...
…makes no assumption about the
distribution of the variable in the
population, that is, the shape of the
distribution
…used when the data represent a
nominal or ordinal scale, when a
parametric assumption has been
greatly violated, or when the nature
of the distribution is not known
…a “less powerful” test in that it is less
likely to reject a null hypothesis at a
given level of significance
…usually requires a larger sample size
to reach the same level of
significance as a parametric test

The most common tests of
significance…
t-test
ANOVA
Chi Square

t-test...
…used to determine whether two
means are significantly different at
a selected probability level
…adjusts for the fact that the distribution
of scores for small samples becomes
increasingly different from the normal
distribution as sample sizes become
increasingly smaller
…the strategy of the t-test is to
compare the actual mean
difference observed to the
difference expected by chance
…forms a ratio where the numerator is
the difference between the sample
means and the denominator is the
chance difference that would be
expected if the null hypothesis were
true
…after the numerator is divided by the
denominator, the resulting t value is
compared to the appropriate t table
value, depending on the probability
level and the degrees of freedom
…if the t value is equal to or greater
than the table value, then the null
hypothesis is rejected because the
difference is greater than would be
expected due to chance
…there are two types of t-tests: the
t-test for independent samples
(randomly formed) and the t-test for
nonindependent samples
(nonrandomly formed, e.g.,
matching, performance on a
pre-/post- test, different treatments)

ANOVA...
…used to determine whether two or
more means are significantly
different at a selected probability
level
…avoids the need to compute duplicate
t-tests to compare groups
…the strategy of ANOVA is that total
variation, or variance, can be
divided into two sources: a)
treatment variance (“between
groups,” variance caused by the
treatment groups) and error
variance (“within groups”
variance)
…forms a ratio, the F ratio, with the
treatment variance as the
numerator (between group
variance) and error variance as the
denominator (within group
variance)
…the assumption is that randomly
formed groups of participants are
chosen and are essentially the
same at the beginning of a study
on a measure of the dependent
variable
…at the study’s end, the question is
whether the variance between the
groups differs from the error
variance by more than what would
be expected by chance
…if the treatment variance is
sufficiently larger than the error
variance, a significant F ratio
results, that is, the null hypothesis
is rejected and it is concluded that
the treatment had a significant
effect on the dependent variable
…if the treatment variance is not
sufficiently larger than the error
variance, an insignificant F ratio
results, that is, the null hypothesis
is accepted and it is concluded
that the treatment had no
significant effect on the dependent
variable
…when the F ratio is significant and
more than two means are involved,
researchers use multiple
comparison procedures (e.g.,
Scheffé test, Tukey’s HSD test,
Duncan’s multiple range test)

FANOVA...
…used when a research study uses
a factorial design to investigate
two or more independent variables
and the interactions between them
…provides a separate F ratio for each
independent variable and each
interaction

Multiple Regression...
…a prediction equation that includes
more than one predictor
…predictors are variables known to
individually predict (correlate with) the
criterion to make a more accurate
prediction

Chi Square (Χ2)...
…a nonparametric test of significance
appropriate for nominal or ordinal
data that can be converted to
frequencies
…compares the proportions actually
observed (O) to the proportions
expected (E) to see if they are
significantly different
…the chi square value increases as
the difference between observed
and expected frequencies
increases
…ANCOVA can also be used to
increase the power of a statistical
test by reducing within-group
(error) variance, that is, to make a
correct decision to reject the null
hypothesis
One- and two- tailed tests of
significance...

tests of significance that indicate the
direction in which a difference may
occur
…the word “tail” indicates the area of
rejection beneath the normal curve

A = B…
…no difference between means; the
direction can be positive or negative
…direction can be in either tail of the
normal curve
…called a “two-tailed” test
…divides the α level between the two
tails of the normal curve

A > B or A < B…
…there is a difference between means;
the direction is either positive or
negative
…called a “one-tailed” test
…the α level is found in one tail of the
normal curve
Degrees of freedom (df)...

a statistical concept indicating that one
degree of freedom is lost each time a
population parameter is estimated on
the basis of a sample of data from the
population
…indicates that there is no true
difference or relationship between
parameters in the populations

the ability for the sample means to
vary which is dependent upon the
number of participants and the
number of groups

for example: as the number of
participants increases (df) the value
needed to reject the null hypothesis
becomes smaller
Mini-Quiz…

True and false…
…inferential statistics are concerned
with determining whether results
obtained from a sample(s) are
equivalent to those in the entire
population
True

True and false…
…inferential statistics are used to
make inferences about parameters,
based on the statistics from a
sample
True

True and false…
…inferential statistical analyses
prove the results are either true or
false
False

True and false…
…the word error in the term
“standard error of the mean”
indicates that the various sample
means making up the distribution
contain some error in their
estimate of the population mean
True

True and false…
…purely by chance a researcher
once in a while will select a
sample that is quite different
from the population
True

True and false…
…to find the mean of the sample
means, the researcher adds up
all of the sample means and
divides by the number of means,
as long as the size of each
sample is the same
True

True and false…
…the size of the sample and the
standard error of the mean
negatively correlate
True

True and false…
…the difference between two
sample means being a true or
real difference means that the
difference was caused by the
dependent variable and not by
chance
False

True and false…
…the null hypothesis states that
any difference or relationship
found for the samples is the
result of sampling bias
False

True and false…
…the null hypothesis is the
research hypothesis
False

True and false…
…tests of significance deal with
probability not certainty
True

True and false…
…tests of significance enable the
researcher to know for sure
that the researcher’s analysis
correct
False

True and false…
…the researcher makes the decision
to reject or not reject the null
hypothesis with a given
probability of being correct
True

True and false…
…rejecting the null hypothesis
represents the researcher’s
conclusion that the means are
significantly different
True

True and false…
…a significant difference between
means indicates that they are too
different to be the result of random,
chance, sampling error
True

True and false…
…accepting the null hypothesis
indicates that the means are
determined not to be significantly
different, that is, the difference is
due to sampling error
True

True and false…
…researchers must always set the
probability level, α, prior to testing
for significance
False

True and false…
…testing for significance is actually a
matter of comparing the
consequences of making two
possible incorrect decisions
True

True and false…
…with α = .05, the researcher
believes the null hypothesis will be
true 95% of the time
False

True and false…
…as a researcher decreases the
chances of committing a Type I
error, the researcher increases the
probability of committing a Type II
error
True

True and false…
…rejecting a null hypothesis at α = .001
proves the research hypothesis, that
is, the independent variable causes
the dependent variable
False

True and false…
…a “more powerful” statistical test of
significance means that the
researcher is less likely to commit a
Type II error
True

True and false…
…a parametric test of significance
should be used when the data
represent an ordinal or nominal scale
False

True and false…
…generally speaking, a parametric test
of significance should be used when
the data represent interval or ratio
scale
True

True and false…
…a significant F ratio indicates that
there is at least one significant
difference somewhere among the
means but not which one it is
True

True and false…
…when many tests of statistical
significance are performed, the
probability level, α, tends to decrease
because performing a large number
of tests makes it more likely to obtain
significant differences
False

True and false…
…when the chance of finding a
significant difference between means
is increased, so is the chance of
committing a Type I error
True

Fill in the blank…
…an inferential statistic that tells the
researcher how much the researcher
would expect the sample means to
differ if the researcher used other
samples from the same population
standard error of the mean

Fill in the blank…
…a means by which researchers
determine whether there is a
significant of real difference between
the sample means, one due not to
random sampling error
tests of significance

Fill in the blank…
…the statement explaining that the
difference between two sample
means is the result of chance,
random sampling error
null hypothesis

Fill in the blank…
…the type of error when the null
hypothesis is true but the researcher
concludes that it is false
Type I error

Fill in the blank…
…the type of error when the null
hypothesis is false but the
researcher concludes that it is true
Type II error

Fill in the blank…
…the term indicating the probability
that the researcher is correct
level of significance
probability level

Fill in the blank…
…when α = .05, the probability that a
difference is significant will be
accurate within ___ standard
deviations of the sample means
(SEX)
+/- two SEX

Fill in the blank…
…when α = .01, the probability that a
difference is significant will be
accurate within ___ standard
deviations of the sample means
(SEX)
+/- three SEX

Fill in the blank…
…a null hypothesis which states that
one difference can only occur in one
direction requires a ____ test of
significance
one-tailed

Fill in the blank…
…the type of error committed when a
researcher does not reject a null
hypothesis that should be rejected
Type II error

Fill in the blank…
…a statistical test of significance
which determines whether the
observed difference is sufficiently
larger than a difference that would
be expected solely by chance
t-test

Fill in the blank…
…multiple comparisons of the means
that is decided upon before not after
the study is conducted and is based
upon research hypothesis
a priori comparisons
planned comparisons

Fill in the blank…
…the situation where multiple
comparisons of the means cannot be
decided upon before the study is
conducted and is based upon
research hypothesis
a posteriori comparison
post hoc comparison

Fill in the blank…
…the ability of a test of significance to
reject a false null hypothesis, that is,
to make a correct decision to reject
the null hypothesis
power
This module has focused on...
inferential statistics
...the statistical procedures for
describing, synthesizing, analyzing,
and interpreting quantitative data
The next module will focus on...
post-analysis considerations
and research reports
...the procedures for checking and
storing all data in an organized
manner and general guidelines for
reporting findings