About Validity and
Reliability:
Examples
TABLE OF CONTENTS
Types of Validity ................................................................................................................ 1
Face Validity................................................................................................................... 1
Content Validity.............................................................................................................. 1
Criterion Validity ............................................................................................................ 1
Concurrent Validity ........................................................................................................ 1
Predictive Validity .......................................................................................................... 1
Construct Validity........................................................................................................... 2
Types of Reliability............................................................................................................. 2
Test-Retest Reliability .................................................................................................... 2
Spearman-Brown and Cronbach’s Alpha ........................................................................... 2
Internal and External Validity............................................................................................. 3
History............................................................................................................................. 3
Maturation....................................................................................................................... 3
Testing............................................................................................................................. 3
Instrumentation ............................................................................................................... 3
Selection.......................................................................................................................... 3
Experimental Attrition .................................................................................................... 3
What is External Validity?.............................................................................................. 4
Reactive Effects of Testing......................................................................................... 4
Unrepresentative samples ........................................................................................... 4
Reactive Settings......................................................................................................... 4
Multiple-Treatment Interference................................................................................. 4
About Validity and Reliability: Examples
Types of Validity
Face Validity
►
You create a test to measure whether people with the name Brandon
are generally more intelligent than people with the name Dakota. If
everyone shown the proposed test agrees that it seems valid, the face
validity of the test has been established.
Content Validity
►
You wish to measure an individual's sleepiness. Noting that
sleepiness is affected by a number of external and internal influences
(e.g. amount of sleep the previous day, amount of activity during the
day, etc.), a valid measure of sleepiness would account for these
various factors.
Criterion Validity
►
You wish to measure risk of failure amongst first-year chemistry
students, with a 25-item questionnaire. Criterion validity is established
if your proposed measure correlates with a previously validated
measure on risk of failure amongst first-year chemistry students.
Concurrent Validity
►
Your test which measures the average number of bananas consumed
by an infant monkey correlates strongly with a validated test
measuring the same information, when applied to a sample in the San
Francisco Zoo.
Predictive Validity
►
The predictive validity of the MCAT for students applying to medical
school is measured through the correlation of student MCAT scores to
their undergraduate marks. If high-scorers on the MCAT perform
better in their undergraduate courses than low-scorers, then the
MCAT is presumed as a predictively valid measure of academic
ability.
1
Construct Validity
►
You wish to develop a new measure to assess euphoria. Construct
validity is established by demonstrating that the new measure
accurately predicts the myriad hypotheses that can be derived from a
theory of euphoria.
Types of Reliability
Test-Retest Reliability
►
The chart given below represents the data associated with two
applications of the same test.
r = 0.98
The r, Pearson's correlation coefficient, indicates a strong positive
correlation.
Spearman-Brown and Cronbach’s Alpha
►
A researcher has developed a new 50-item scale to measure an
individual's attentiveness. In evaluating the reliability of the measure,
she decides to use the split-half reliability approach. After she collects
data from an administration of the measure, she divides the scale
equally, treating each half as its own scale in the application of
correlational analysis. It is found that r = 0.73, which is too low to
show adequate reliability. What should she do?
She can apply the Spearman-Brown formula!
2
►
When the researcher plugs her values into the formula:
r = 2 (0.73) = 1.46 = 0.84
1 + 0.73 1.73
Therefore, we see that after correcting with the Spearman-Brown
formula, the researcher can now consider her measure of attentiveness
to be reliable.
Internal and External Validity
History
►
A 20-year study of Internet crime may be affected by a change in
related laws, during the study period.
Maturation
►
A child may have a longer stride due to physical growth, as opposed
to continual correction from an adult (independent variable).
Testing
►
A good example of this is multiple testing effects.
Instrumentation
►
One might measure a subject's propensity for laughter at the start of a
study on happiness, while measuring a number of physical
characteristics observed at the end of the same study.
Selection
►
In a study on innate athletic ability, two groups are selected – one
composed entirely of NBA starters, the other made up of couch
potatoes.
Experimental Attrition
►
In a test comprising two groups of 100 subjects each, the first group
has 11 drop-outs, while the other has 50 drop-outs.
3
What is External Validity?
Reactive Effects of Testing
►
Measurement of a paper and pencil test will likely yield different
results for those who have written them previously, as opposed to
those who have not.
Unrepresentative samples
►
A group of German midgets is selected to measure average height
tendencies in Europe.
Reactive Settings
►
A subject may feel depressed based on the peeling wallpaper and
black ceiling of the room, while the experimenter may provide clues
to their expectations of how the experiment will go based on their
demeanour.
Multiple-Treatment Interference
►
Cue cards are presented to subjects in a particular order in a loud
room, while these same cue cards are presented in the same order to
another group of subjects in a quiet room. Unnoticed by the
experimenter was that the first room had three mirrors on the walls
within the subjects’ immediate view and peripheral vision, in addition
to a few stains on the carpet, while the second room was clean, with
one mirror out of view of the subjects.
4