UNIT 1:
RESEARCH
METHODOLOGY
ILLUSIONS
• “We live in a world of illusion…” Steve Miller band
• In their book The Invisible Gorilla,Chabris & Simons details
some of these including the illusions of attention, memory,
confidence, knowledge, cause, and potential.
• Rate yourself: On a scale of 1-5 How attentive are you? How
good of a driver are you? How good are you at multi-tasking?
Forming a scientific attitude
• “To believe with certainty we must begin by doubting.”
Polish proverb
• Scientific attitude:
• Be skeptical but not cynical
• Be open but not gullible
• Be a critical thinker by developing the following skills:
Follow the steps to good scientific
methodology…
• Make observations that lead to the proposal of a problem
• Propose a researched based hypothesis
• Design and conduct an experiment
• Gather and analyze data
• Conclude and replicate, replicate, replicate!
CRITICAL THINKING SKILLS
As taken from Ch. 1 of “Challenging Your Preconceptions”
• Be flexible—tolerate ambiguity & uncertainty
• Identify bias—accept its existence, be on the lookout, try to
•
•
•
•
•
prevent it
Be skeptical…but not cynical.
Distinguish between FACT & OPINION
Don’t oversimplify
Use logical inference processes
Look at the evidence and then draw conclusions.
Forming a scientific attitude
• To put the scientific attitude to practice requires skepticism
and humility. At times we have to reject our own ideas.
• Example: The Mozart Effect: Research was published
several years ago that showed a relationship between
listening to Mozart and improved problem skills. This
became the phenomenon known as the Mozart Effect.
This was a founding principal behind “Baby Einstein”
products.
• http://en.wikipedia.org/wiki/Mozart_effect
• http://www.nytimes.com/2009/10/24/education/24baby.ht
ml?_r=0
TYPES OF STUDIES THAT DESCRIBE
• CASE STUDIES: study of an individual in great depth in
hopes to reveal universal truths. They suggestion
direction for further study, but atypical cases can be
misleading. Ex: early brain research
• NATURALISTIC OBSERVATIONS: watching and
recording behavior of organisms in their natural
environment; this only describes behavior, it does not
explain it.
• SURVEYS: Surveys look at many individual cases but
not in depth like a case study. Design of the tool is
important….
WHEN DEVELOPING SURVEYS…
• Wording of surveys: Slight word changes can have a
significant impact,
• “not allowing” vs “forbidding” or “censoring”
• “in favor of” vs “not opposed to”
As a result you want to be very cautious in your wording when
developing surveys.
Also be critical of survey results. Consider your thoughts in relation to
call in voting shows result such at American Idol or Dancing
W/Stars…
Random sampling important.
BIAS—Something to recognize and
avoid
Bias: a particular tendency or inclination,
especially one the prevents unprejudiced
consideration of a question;
In can present itself in the experimenter or
the research participant or both.
Experimenter bias: movie clip Ghostbusters
Can lead to self-fulfilling prophecy, false consensus effect,
overconfidence…
Random Sampling: a sample that fairly
represents the population because each
member had equal change of inclusion
 You cannot survey everyone.
 Select a random sample.
› Not just family, friends, people of same age, gender or ethnicity.
Otherwise you possibly end up with a false consensus effect.
 Random sampling is where every person has an equal
chance of being selected for participation.
 Basic statistical rule of thumb is 10% of your population
is an appropriate size for a sample.
DIFFERENT STUDY METHODS
 LONGITUDINAL STUDIES: Studies that occur over a
long period of time. Typical of developmental studies
and some drug studies. Many problems: time
consuming, expensive, subject drop out, etc.
 CROSS-SECTIONAL STUDIES: Alternative to the
longitudinal study where subjects from different age
groups are tested at the same time. Usually not as indepth (ex: National testing data)
Operational definitions
• A statement of procedures (operations) used to define research
variables. For example, intelligence may be operationally
defined in a project by the exact tool used to measure it.
• If you wanted to measure how “emotional” a population is, how
exactly would you measure “emotional?”
• When Levine & Norenzayan (1999) measured the “pace of life”
in 31 countries they defined “pace” by walking speed, speed at
which postal clerks completed a task, and accuracy of public
clocks.
Operational definitions
(additional examples)
• If you want to improve your grades by being more
“organized,” how would you measure your organization in
order to know if you are more or less?
• As you visit college campuses you may have a
“friendliness” scale you use to make judgments about that
campus. What is on your scale? How do you determine
if A&M is “friendlier” than “UT?”
• Who told you this class was “easy?” Did you ask them to
operationally define that…
STUDIES THAT SHOW RELATIONSHIPS
Experimentation is use to determine cause
& effect relationships.
• Make observations that lead to the proposal of a problem
• Propose a researched based hypothesis
• Design and conduct an experiment
• Gather and analyze data
• Conclude and replicate, replicate, replicate!
Experimentation is use to determine
cause & effect relationships.
• To conduct an experiment, you must vary the independent
variable (IV) and keep all other variables constant. An
independent variable is the variable in an experiment that is
being implemented and its effect is being measured.
• You must have a control group (does not received the IV)and
an experimental group (does receive the IV).
• You will measure the outcome which is the dependent
variable.
• Researcher should randomly assign participants to either the
control group or the experimental group.
• Experiment simulation
Conducting experiments
• With some experiments, particularly drug tests, it is ideal
to have a double-blind test.
• A double blind test is when neither the participant nor the
researcher knows whether someone is in the control
group of the experimental group.
• The experimental group gets the drug and the control
group receives a placebo, an inactive drug.
• The purpose of this is to reduce the self-fulfilling prophecy
on the part of both experimenter and participant. Often a
placebo effect is documented in studies where
participants simple believe the drug will have an effect
and therefore they report experiencing those effects.
CONFOUNDING VARIABLES
• Sometimes not all variables can be held constant, and
these out of control variables may impact your research.
• These are known as CONFOUNDING VARIABLES.
• Examples could include in our research with music/no
music, people’s height in relation to the table on the smart
board.
QUASI-EXPERIMENT
• This occurs when you cannot randomly assign your
participants to the experimental and the control groups.
• Sometimes random assignment isn’t possible; therefore,
one must recognize their experiment as “quasi.”
• In education, this is common when student may be
exposed to different teaching methods by class.
Ethics, Code of Conduct
• Psychologists strive to benefit those with whom they work
and take care to do no harm.
• We uphold professional standards of conduct…
• We seek to promote accuracy, honesty, and truthfulness
in science.
Ethics, Code of Conduct
• In situations in which deception may be ethically
justifiable, we strive to maximize benefits and minimize
harm.
• We exercise reasonable judgment to prevent unjust
practices.
• We respect the dignity and worth of all people, and the
rights of individuals to privacy, confidentiality, and selfdetermination.
Ethics, Code of Conduct
Two things that must be done:
1. Receive informed consent: You must
inform the participant of what they will be
experiencing, make them aware that they can
quit at any time, ensure confidentiality, and
obtain their approval to participate.
2. Debrief: Research design determines
what form this will take. It must be done if
deception was used to minimize harm or
distress.
Correlation
• Often when conducting research, we seek to find a
relationship among measured variables.
• How similar are personality test scores for identical twins?
Fraternal twins?
• Do SAT scores predict college GPA’s?
• Are more babies born on nights of a full moon?
• Is stress and disease related?
Correlation
 Most studies designed to answered the above questions
are seeking a relationship between the two variables.
 Often the results are re-constructed from “correlation” to
“causation.”
 YOU CANNOT ASSUME CAUSATION FROM A
CORRELATION!
 Just because a relationship has been found between
watching violence and being violent, one cannot
assume watching violence makes someone perform
violent acts.
Correlation
• Low self-esteem correlates, and therefore predicts,
•
•
•
•
depression.
Does that mean low self-esteem causes depression?
No, depression might cause low self-esteem.
OR some other related variable may be contributing to
both low self-esteem AND depression.
Though a causal relationship may exist, a correlation
doesn’t guarantee this.
Correlations
• Correlations can be positive:
• As study time increases, test scores increase
Correlations can be negative:
As time spent sleeping decreases, hallucinations
increase.
Correlations
Identify these as either positive or negative:
• The more young children watch tv, the less they read. (Kaiser,
2003)
• The more sexual content teens see on tv, the more likely they
are to have sex. (Collins, et. al., 2004)
• The longer children are breast-fed, the greater their later
academic achievement. (Horwood & Fergusson, 1998)
• The more often adolescents eat breakfast, the lower their body
mass (Timlin, et al, 2008)
Correlation
• Correlation coefficient: statistical measure of relationship
that reveals how closely 2 things vary together and thus
how well either can predict the other. Correlation
coefficients range from -1.0 to +1.0. The closer to 1, the
stronger the correlation.
• Illusory correlation: perceived correlation that does not
exist. Ex: Adoption followed by conception stories…
When premonitions are reinforced, we are more likely to
believe them.
Practice Questions 2
• The correlation between two observed variables is -0.84. From this, it can
be concluded that
A. As one variable increases, the other is likely to increase.
B. As one variable increases, the other is likely to decrease.
C. The two variables are unrelated.
D. One variable causes the other variable to occur.
E. One variable causes the other variable to not occur.
• A researcher was interested in determining whether her students’ test
performance could be predicted from their proximity to the front of the
classroom. So she matched her students’ scores on a math test with their
seating position. This study is an example of
A.
B.
C.
D.
E.
Experimentation
Correlational research
A survey
Naturalistic observation
A case study
Practice Questions 3
• In an experiment to determine the effects of exercise on motivation,
exercise is the
A. Control condition
B. Intervening variable
C. Independent variable
D. Dependent variable
E. Illusory correlation
• In drug studies, your participants are randomly assigned to one
of two groups, the control group and the experimental group.
The control group would experience
A.
B.
C.
D.
E.
receiving the real drug being tested.
receiving a placebo.
receiving no intervention.
receiving an active drug that is already approved for use.
receiving an alternative therapy that is not a drug.
Data Analysis & Statistics
• Measures of central tendency: mean, median, mode.
One number represents the whole set.
• Mean: average
• Median: exact middle number
• Mode: most frequently occurring number
Example:
67, 94, 72, 88, 88, 95, 100, 88, 72
Mean = 84.9, Median = 88, Mode = 88
Data analysis
• Central tendency numbers are effected by extreme scores
and lop-sided or skewed distributions; therefore,
measures of variation must also be considered: range
and standard deviation
• A large standard deviation means that scores are very spread out
from the mean.
• A small standard deviation means that scores are relatively close to
the mean.
Data analysis
• Positive skew is associated with a tail to the right.
• A negative skew is associated with a tail to the left.
Standard Deviation of normally distributed
scores
2% 13.5% 34% 34% 13.5% 2%
Data Analysis
• Descriptive statistics: describe an entire population, ex:
teacher’s gradebook
• Inferential statistics: data collected from sampling, and if a
representative sample is collected, data can infer
information about the entire population
• Be cautious of big, rounded, undocumented statistics,
often stats are misquoted b/c gen. population doesn’t
understand them!
Statistical significance
• Statistical significance: the difference we observe is probably
not due to chance variations between samples.
• In research, depending on a statistical test, one may claim their
findings are “statistically significant.”
• Be cautious of significance, it needs to be validated by
replication of studies. Also, statistical significance doesn’t mean
the findings are relevant or important it is just another statistic.
Statistical caution
• "The secret language of statistics, so appealing in a fact-
minded culture, is employed to sensationalize, inflate,
confuse, and oversimplify," warns Darrell Huff, author of
How to Lie with Statistics.
Practice Questions 4
• What is the median of the following scores: 10, 7, 5, 11, 8, 6, 9?
A. 6
B. 7
C. 8
D. 9
E. 10
• What statistic would best describe a set of scores that are very spread out
from the mean?
A.
B.
C.
D.
A large standard deviation
A small standard deviation
A large range
A small range
E. Any measure of central tendency
• When a difference between two groups is “statistically significant,” this
means
A.
B.
C.
D.
E.
The difference is statistically real but of little practical importance.
The difference is probably the result of sampling variation.
The difference is not likely due to chance.
The difference is of importance to the world of research.
The difference found is probably a fluke.
Practice Questions Answers
• Set 1: B, E
• Set 2: B, B
• Set 3: C, B
• Set 4: C, A, C
• Sample quiz answers: 1. d, 2. e, 3. a, 4. d, 5. c, 6. b, 7. a,
8. a, 9. b, 10. a, 11. c, 12. d, 13. b, 14. d, 15. d