Two types of empirical questions
• Descriptive
– This kind of empirical question requires a researcher to
describe some aspect of behavior
– For example, a researcher might ask, What are people’s
attitudes toward the homeless?
• Causal
– This type of empirical question requires a researcher to
determine what causes something to happen
– For example, a researcher might ask, Does stress cause
people to have road rage?
Our studies
• We want to know whether or not verbal
estimate-based depth perception training
benefits subsequent performance on verbal
and active tasks?
• In other words, we want to know whether or
not such training causes better performance
later on
Question
• How do you determine that one thing
caused something else to happen?
• For example, how could we determine
that a new training simulation improved
performance?
A simple logical method
1.
2.
3.
4.
5.
Collect data about current behavior
Change the suspected cause
Do not change anything else
Collect data about subsequent behavior
Compare data collected before and
after the change was made
Example
Example
1.
2.
3.
4.
5.
Pre-test painting ability
Provide training via simulator
Do not change anything else
Post-test painting ability
Compare pre and post-test data
Complications
• The logical process outlined earlier is
intuitive and straightforward
• When studying behavior, however,
several issues could occur that would
complicate the interpretation of the data
Potential Complications 1 & 2
• Something other than the suspected
cause changes
– Something inside the participants changes
• This is known as a maturation problem
– Something outside the participants changes
• This is known as a history problem
Maturation
1. Pre-test painting ability
–
2.
3.
4.
5.
The participant warmed up during the pre-test
Provide training via simulator
Do not change anything else
Post-test painting ability
Compare pre and post-test data
–
Is the difference due to training or warm-up?
History
1. Pre-test painting ability
2. Provide training via simulator
–
The simulator technician provides some advice
3. Do not change anything else
4. Post-test painting ability
5. Compare pre and post-test data
–
Is the difference due to training or advice?
Training, Maturation or History?
Potential Complication 3
• The initial data collection may bias
participants
– This is known as a testing problem
Testing
1. Pre-test painting ability
–
Certain aspects of painting are assessed
2. Provide training via simulator
–
Participants work hard on aspects of painting that will
be assessed
3. Do not change anything else
4. Post-test painting ability
5. Compare pre and post-test data
–
Is the difference due to training or bias?
Training or Testing?
Potential Complication 4
• How one collects the Pre-Test data may
differ from how the Post-Test data are
collected
– This is known as a instrumentation problem
Instrumentation
1. Pre-test painting ability
–
Test involves a door panel
2. Provide training via simulator
3. Do not change anything else
4. Post-test painting ability
–
Test involves a trunk lid
5. Compare pre and post-test data
–
Is the difference due to training or tasks?
Training or Instrumentation?
Potential Complication 5
• Sometimes the Pre-Test scores are
extreme, so it is likely that Post-Test
scores will be different, no matter what
– This is known as a regression problem
Regression
1. Pre-test painting ability
–
A number of participants score abnormally low
2. Provide training via simulator
3. Do not change anything else
4. Post-test painting ability
–
Those low scoring participants score more average,
while others stay the same
5. Compare pre and post-test data
–
Is the difference due to training or abnormal scores?
Training or Regression?
Solution
• There is a simple way to capture these issues,
if they occur
– Include a control group
– If Pre and Post-Test scores differ for both the
experimental and control groups, then it is likely
that the study was affected by one of these
problems
• This is known as having a confound in a study
A more complex method
1. Collect data about current behavior
2a. Experimental group - Change the suspected cause
2b. Control group - Don’t change the suspected cause
3. Don’t change anything else
4. Collect data about subsequent behavior
5. Compare data collected before and after the change
was made
Example
1. Pre-test painting ability
2. Provide training
–
–
Via simulator (Experimental group)
Via standard method (Control group)
3. Don’t change anything else
4. Post-test painting ability
5. Compare pre and post-test data
No Confounds
Confounds
Confounds
Our studies
• Experimental group
– Pre-Test, Verbal Training w/ Feedback, Post-Test
• Control group
– Pre-Test, Verbal Training w/o Feedback, Post-Test
Potential Confounds
•
•
•
•
•
Maturation
History
Testing
Instrumentation
Regression