Basic Logic of Experimentation
The design of an Internally valid experimental procedure requires us to:
Form Equivalent Groups
Treat Groups Identically in All Respects Except for the IV
Assess the Significance of Differences in DV Scores Between Conditions
A conclusion is based on the logic that
(1) if the groups did not pre-experimentally differ with respect to the DV
(2) then the groups were treated identically in all ways except for one (the IV)
(3) and now the DV scores for the groups do significantly differ
There is only one way to account for the differences in the DV scores, they must be
due to the influence of the IV
Matching
Matching involves initially premeasuring subjects on some attribute (believed to be
strongly correlated with the DV) and assigning subjects to conditions based on these
scores.
How to match:
1) Collect data on the matching variable (MV)
2) Rank order subjects according to MV scores
3) Segregate subjects into matched pairs (or matched trios, or matched quartettes, etc.)
4) Randomly assign one pair member to a condition of the experiment
5) The assignment of the second member of the pair is dictated
Matching
Matching involves initially premeasuring subjects on some attribute (believed to be
strongly correlated with the DV) and assigning subjects to conditions based on these
scores.
Matching always produces groups which are more equivalent than random assignment.
But how much more? That depends on the degree of correlation between MV and
DV.
If the MV is not correlated with the DV, matching will produce no effect on equivalence.
How to choose a matching variable:
(1) Consult earlier published studies – there may be reported observed correlations
between your DV and possible MV’s
(2) Conduct pilot research to determine which possible MV correlate best with your DV
(3) Choose a MV on logical grounds. (Example: if your DV involves complex problem
solving, its logical to assume that performance is correlated with intelligence).
Matching
Since matching always produces groups which are more equivalent than random
assignment, why not always use matching?
Only Disadvantage of Matching – Extra work and time of conducting the matching
procedure prior to commencement of actual experimentation (plus the increased
possibility of mortality with the greater demands being placed on the subject).
Ultimately, the decision to use matching hinges on whether the advantages to be gained
justify the time and effort required to do the matching.
Examples:
A common practice in early psychological experimentation was to routinely match on the
basis of intelligence under the belief that intelligence is related to everything we do.
(Abandoned as a general practice).
Using rats in an instrumental task where body weight reduction is the definition of the
motivating factor – deprivation level. Matching on pre-deprivation body weight is a
simple easy practice which is therefore routinely done.
More Advanced Matching
Matching always produces groups which are more equivalent than random assignment.
(Match by correlated criterion design)
Even greater equivalence can be attained by matching on multiple variables
simultaneously. (Match by correlated criteria)
You can match on the basis of one criterion, two criteria, three criteria,… But would it
ever be possible to match on the basis of all criteria?
The ultimate match in correlated criteria would be the case of identical twins (the so
called “Co-Twin Design”)
Who would be the ultimate match for you? The Within-Subject design is in essence the
ultimate matched groups design. The same exact people are used in all conditions of
the experiment.
Not exactly – due to progressive error. If someone participates in two conditions they
have to participate in one of them first. This changes the person in some way so they
are not exactly the same person in the second condition that they were before the first
condition. And, of course, if they participate in a third condition they are changed even
more by their previous participation in the first two conditions.
Independent Conditions vs Dependent Conditions
Independent Conditions are called that because the group compositions are arrived at
independent of each other. Who is assigned to one condition is in no way determined by
the assignment of subjects to the other condition. There is no systematic relationship
between the subjects in one group vs the other. (E.G., Random Assignment)
Dependent Conditions are formed when there is some dependence or relationship
between those assigned to one condition vs another. There are three situations where
dependent conditions are formed:
Matched-groups designs
Matching done on variables
that are organismic
Matching done on
stimulus variables
Within-subject designs
Yoked-control designs
It is essential that the distinction between independent condition and dependent
conditions be recognized as it dictates the appropriate statistical test for the analysis of
differences between conditions.
Yoked-Control Group
Procedure in which the environmental
conditions affecting a control subject are
manipulated so that they exactly correspond
to those affecting an experimental subject
Designs which Create
“Independent Groups”
• Randomized Groups Designs
• Any Between-Subjects Design
Lacking a Logical Connection
Between Subjects in its
Conditions
Designs which Create
“Dependent Groups”
• Matched-Groups Designs
•
Co-Twin Investigations
• Within-Subjects Designs
• Yoked-Control Designs
Appropriate Analysis
(two-groups design)
T-test for Independent Means
T-test for Dependent Means