Research Methods
PSYA4
The major features of science
• Empiricism
– Information gained through direct observation or
experiment.
• Objectivity
– Observations and experiments should be unaffected by bias
(such as researcher expectations).
• Replicability
– It is important that research can be repeated and similar
results obtained, this adds to the reliability of the study.
• Hypothesis testing
– This means using the scientific process to test a hypothesis
under controlled conditions.
• Theory construction
– Observations help to construct theories to understand the
phenomena around us.
The scientific process
Induction
Deduction
Observations
Observations
Testable hypothesis
Propose theory
Conduct a study to test
the hypothesis
Testable hypothesis
Draw conclusions
Propose theory
Conduct a study to test
the hypothesis
Draw conclusions
Theory construction
Induction
Deduction
• Involves reasoning from the
• Involves reasoning from the
particular to the general.
general to the particular.
• For example a scientist may observe
instances of a natural phenomenon • Starts with a theory and looking
and come up with a general law or
for instances that confirm this.
theory.
• Darwin’s theory of evolution is an
• Before the twentieth century, science
example of this. He formulated a
largely used the principles of
theory and set out to test its
induction- making discoveries about
the world through accurate
propositions by observing animals
observations, and formulating
in nature. He specifically sought to
theories based on the regularities
collect data to prove his theory.
observed.
• Newton’s Laws are an example of
this. He observed the behaviour of
physical objects and produced laws
that made sense of what he
observed.
Can psychology claim to be a
science?
• Some psychologists are subjective (this means using their
own opinion when conducting research). These methods aim
to be valid, but data collected using interviews,
questionnaires, content analysis or observations which are
then triangulated could be less objective than experiments.
• The scientific method is reductionist and deterministic.
This is because we are attempting to explain complex human
behaviour in simplistic terms.
• Psychological treatments of mental health problems have had
modest success; therefore the goals of science are not always
appropriate in psychology.
• Experimenter bias and demand characteristics make
studying human behaviour less objective and this
compromises validity.
On the down side ..
Do we really want to be a science?
• The scientific approach is reductionist, simplifying
complex phenomena and theories down to basics.
• Science is also determinist in its search for causal
relationships, i.e. if X determines Y.
• Science also takes the nomothetic approach looking to make generalisations about people and
find similarities.
• Some psychologists argue the idiographic
(individual) approach is more suitable when treating
patients. Currently psychology has only moderate
success when treating mental illness.
• Qualitative research is seen as less than scientific
but triangulation can make this method more
objective and valid.
Validating new knowledge using
peer review
• Peer review is the assessment of research by others
who are experts in the same field (peers). This is
usually done before research is published.
• This is an essential check to prevent incorrect or
faulty data from entering the public domain.
• It is also necessary where any application for funding
is concerned so it affects not just the researcher but
also the university department that employs them.
public bodies allocate funding to the most worthwhile
research.
• Every researcher should be prepared for their work to
be scrutinised. Peer review is a way of establishing
the validity of scientific research.
• There are online journals (e.g. ArXiv) where readers
rate the articles and can form the basis of peer review.
Advantages of peer review
• It ensures that any research conducted and published is
of high quality
• Checks the validity of the research
• Judges the credibility of the research and assessing
the quality and appropriateness of the design and
methodology.
• Peers assess how original the work is and whether it
refers to relevant research by other psychologists.
• Make a recommendation as to whether the research
paper should be published in its original form, rejected
or revised in some way.
• Helps to ensure that any research paper published in a
well-respected journal has integrity and can,
therefore, be taken seriously by fellow researchers and
by lay people.
Criticisms of peer review
• Unachievable ideal – it isn’t
always possible to find a suitable
expert to review the report.
• Anonymity – research is
conducted in an environment
surrounded by competition.
Relationships between experts
sometimes affect objectivity. Now
journals prefer open reviewing.
• Publication bias – peer review
tends to prefer positive results.
• Preserving the status quo –
peer review tends to prefer results
that support previous theory, rather
than going against it. Science does
not really like huge shifts in opinion
or theory.
Peer review is...
1. Slow
2. Expensive
3. Profligate of academic
time
4. Highly subjective
5. Prone to bias
6. Easily abused
7. Poor at detecting
defects
8. Useless at detecting
fraud
Practice Questions
1.
2.
3.
4.
5.
6.
7.
8.
What is science? (2)
Explain two of the key features of
science? (3+3)
Outline the three goals of
science? (3)
Outline the scientific process
(hypothetico-deductive method)
(4)
Explain the principle of
falsifiability (2)
Explain the 3 stages in the
development of a scientific
discipline/paradigm? (2+2+2)
List 2 criticisms of the scientific
approach (3+3)
Why may the scientific nature of
research be reduced because it is
being carried out on human
beings? (3+3+3)
9.
10.
11.
12.
13.
14.
What is meant by the term peer
review in validating new scientific
knowledge? (3)
Name the sections of the report to
be sent for peer review?(2)
Outline 3 problems with peer
review (2+2+2)
Choose two psychological
approaches and explain, with
examples, why they are scientific
(3+3)
Choose one psychological
approach and explain, with
examples, why it is not scientific
(3)
Give 2 points to support and two
to criticise Psychology as a
science (2+2+2+2)
Layout of a Psychological
Investigation
1. Abstract
–
A summary of the study, covering 2-5.
–
What the researchers intend to investigate, refer to previous
studies, and state the hypotheses.
2. Introduction/aim
3. Method
–
What the researchers did, in enough detail to be replicated, also
mentioned are the participants, the environment, the
procedure, the data collection methods and the instructions to
the participants.
4. Results
–
Statistical data and descriptive statistics.
–
An explanation of the results is given, with implications for
future research.
5. Discussion
6. References
–
Full details of the journals mentioned.
Designing Psychological
Investigations
Consent form
Standardised instructions
Standardised debrief
The form should contain some
of the following:
•The purpose of the study
•The length of time required of
the participants
•Right to withdraw
•Reassurance about protection
from harm e.g. the availability
of medical supervision
•The requirement to undertake
a series of psychological tests
•Reassurance about
confidentiality of the data
Instructions should be clear and
succinct.
They must explain the
procedures of this study
relevant to participants and
include a check of
understanding of instructions.
They should also use language
appropriate and be as
courteous as possible.
This is not a consent form so
explicit references to ethical
considerations are not
necessary
It is perfectly acceptable to
include comments such as 'you
are free to withdraw from the
study at any time.'
This is where all is explained to
the participant .
They are thanked at the end of
his/her contribution.
The aim of your study is
included and also a reference to
ethical guidelines.
Refer to participant’s results
being confidential.
Make sure that participants’
names are not showed/written
on the results sheets, raw data
or anywhere else in the report
(re: confidentiality).
Display the standardised
instructions to the participant(s)
again.
Ethical and methodological
issues need to be included for
participants to make an
informed decision
Reliability
• Inter-observer reliability
– When there are two or more observers you need to ensure there are
general agreements in the observations. 0.80 or more is good.
• Internal reliability
– The measure of something that is consistent within itself (e.g. the
questions on an IQ test measure IQ).
• External reliability
– The measure is consistent over several occasions (e.g. the same
individual gets the same IQ score).
Methods used to assess/ensuring reliability
• Inter-rater reliability
– Two or more interviewers/observers must get the same outcome on
80% or more of the behaviours.
• Split-half method
– Compare an individual’s performance on two halves of a test.
• Test-retest method
– A person repeats a test a month or so after doing the test the first
time.
Validity
• Internal validity
– Whether the researcher measured what they intended to
measure.
– It is affected by extraneous variables. Changes in the DV can
be due to EVs rather than the IV.
• External validity
– The extent the results of the study can be generalised to
others (also known as ecological validity).
Methods used to assess/ensuring validity
• Face validity
– Does the test look as though it measures what it intends to
measure.
• Concurrent validity
– Results from a new test can be compared to a previously
well-established test.
Sampling
Opportunity
Volunteer
Random
Stratified
Quota
Snowball
•Use the people available at the time. But can be biased.
•Selected by advertisements. Biased due to the high motivation.
•Participants are identified and then selected by a lottery
method or a random name generator. Normally unbiased.
•Sub-groups are identified (e.g. boys or girls), then a
predetermined number from each group is selected randomly in
proportion to the target population.
• Same as above but with opportunity sampling.
• Participants recommend friends to join the study. Prone to
bias because you only get access to a small proportion of the
population.
Ethical Issues with Human Ppts
• Same as AS (informed consent, right to withdraw, debrief,
deception, protection from harm etc)
• What you need to know now is the BPS code of conduct:
1. Respect: the researcher needs to maintain privacy,
confidentiality, and have informed consent. You would not
need informed consent in an observation where people
would normally expect to be observed. Deception is only
acceptable when revealing the reasons for the study affect
the integrity of the results.
2. Competence: psychologists should maintain high
standards.
3. Responsibility: psychologists have responsibilities to the
clients, the public, and the science of psychology, this
includes a debrief and protection from harm.
4. Integrity: psychologists should be honest and accurate,
including the reporting of findings. Any misconduct should be
reported to the BPS.
Ethical Issues with Non-Human
Animals
You can use animals
when you can’t use
humans (e.g.
Harlow’s monkeys)
Because animals are
fascinating
Why would you choose to study
using non-human animals?
Animals allow for
greater control and
more objectivity
There is enough of a
similarity between
humans and animals
to draw conclusions
from one to the
other
Ethical Issues with Non-Human
Animals
Sentient beings
Speciesism
Do animals feel pain?
There is evidence that
they do…but is this the
same as being conscious?
There is evidence that
primates have selfawareness
Can you justify using
animals in research?
Animal research is strictly controlled and
you need a license. Licenses are only
granted if:
1. Results are important enough.
2. Research cannot be done without
animals.
3. There is a minimum number of
animals used.
4. Discomfort is kept to a minimum
Singer (1990) said that
testing on different
animals is no different to
sexism or racism. However
Gray (1991) says that we
have a duty to humans.
Animal rights
Singer’s view is utalitarian
(whatever is best for the
greater good is
acceptable), so if an
animal reduces human
suffering it is justifiable.
Regan (1984) says animals
should never be used
under any circumstances.
Ethical Issues with Non-Human
Animals
• Russell and Birch (1959) proposed the 3Rs
–Reduction
–Replacement
–Refinement
• The House of Lords endorses this principle.
• But still in the UK we use animal research, as
the law states that drugs need to be tested
on at least two species of live mammal.
Graphical representations
Top Tips for graphs
Give the graph a title.
Label both of the axes.
Use a ruler.
Make the graph at least half a side in size.
Make sure you plan the graph before rushing in.
Think carefully about your choice of graph (don’t
be tricked like you were before ).
Key Terms for Statistical Analysis
• Psychologists look at data to see if the pattern of results could have occurred by chance. If
there results did not occur by chance then we say they are significant.
Probability
• You need to have a null hypothesis (H0) and an alternative hypothesis (H1). What we are
looking for is a significant (large) difference in results so that the differences seen in our
Significance
samples are different and not due to chance; we want to accept the alternative hypothesis.
Chance
Observed
value
Critical
value
• Normally psychologists set the probability level a p≤0.05 which means there is a 5%
possibility the results occur by chance in the sample, when there was no real difference in the
results in the general population.
• The rho or u value calculated is called the observed value.
• You need to look in a table of critical values to see if the results are significant. You need to
know the 1) degrees of freedom (df) – normally the number of ppts in a study (N); 2) one- or
two-tailed test; 3) significance level – normally p≤0.05
Type I and II errors
• Type I accepting the alternative
hypothesis when you should
have rejected it and accepted
the null hypothesis.
» Type I errors are common when the significance
level is too high (e.g. 10%)
• Type II accepting the null hypothesis
when you should have rejected
it and accepted the alternative
hypothesis.
» Type II errors are common if the significance level is
too low (e.g. 1%)
Choosing statistical tests
You use chi-square...
...if the data have been put into categories,
You use Spearman’s rho...
they are classified as nominal data. This is ...if a test of correlation is needed as the
the test of a difference or of an
hypothesis predicted a correlation. The
association. The results are independent data involved ratings made by participants
in each cell, and the expected frequencies
that are ordinal/interval data. Each
(i.e. number of entries) in each cell are
participant has two co-variables.
greater than 4.
You use Mann–Whitney...
You use Wilcoxon...
...if a test of difference is required because ...if a test of difference is required because
the hypothesis predicts there will be a
the hypothesis predicts there will be a
difference between the two groups. The difference between the two conditions.
design is independent groups, and the
The design is repeated measures as all
data were scores on a test (ordinal or
participants were tested twice. Interval or
interval data).
ordinal data has been collected.
R
• Some tests are significant if the observed value is greater than
the critical value, while some tests are the reverse.
• In the chi-squared and Spearman’s rho the obtained (observed)
value has to exceed the critical value for an effect to be
significant.
• In the Mann-Whitney and Wilcoxon tests the observed value
has to be below the critical value for a significant result.
• You will also find this under each table.
• But try to remember this:
– If there is an R then the observed value should be
gReateR than the critical value (e.g. Spearman’s and chisquare). If there is no R (e.g. Mann-Whitney and
Wilcoxon) then the observed value should be less than
the critical value.
Wilcoxon T
• H1 = Participants recall significantly
fewer emotionally threatening
words than neutral words (1-tailed)
• H0 = There is no significant
difference in the number of
emotionally threatening words and
neutral words recalled
• N=9
N = the number of participants
• T=7
• Critical value = 8
• p0.05 (significance level)
• Is it significant?
Chi Square
• H1= There is a significant
association between subject
studied and personality (2tailed)
• H0 = There is no significant
association between subject
studied and personality
• χ2=3.22
• df= 1 (degrees of freedom)
(Row – 1) x (columns – 1)
• Critical value = 3.84
• p0.05 (significance level)
• Is it significant?
Mann Whitney U
• H1= Children using the maths
scheme attain significantly higher
scores than children not using
the maths scheme (1-tailed)
• H0 = There is no significant
difference in scores between
children using the maths scheme
and children not using the maths
scheme
• N1 = 9 (number of people in the
smaller group)
• N2 = 10 (people in larger group)
• U=8
• Critical value = 24
• p0.05 (significance level)
• Is it significant?
Spearman’s Rho
• H1 = There is a significant positive
correlation between test scores
and the amount of time spent
studying for the test (1-tailed)
• H0 = There is no correlation
between test scores and the
amount of time spent studying for
the test
• N = 10 (number of pairs/ppts)
• r = 0.88
• Critical value = 0.564
• p0.05 (significance level)
• Is it significant?
Qualitative Data
•
•
•
•
•
Qualitative researchers dislike quantitative data
because the results are not applicable to
everyday life.
Qualitative methods more subjective as the
results aim to represent the real world.
The qualitative researcher collects subjective data
using broad questions that allow the respondent
to answer in their own words. They may also
observe the behaviour indirectly (through looking
at other’s notes) or directly.
Data sets are very large (but may be through
small samples).
Qualitative data should not be converted to
numbers.
EXAMPLE OF QUALITATIVE
DATA ANALYSIS – CONTENT
ANALYSIS
Step 1 – Coding
• This is the process of identifying categories, themes,
phrases, or keywords in the data set.
– If a psychologist was carrying out an observation he
may identify a number of categories and then allocate
an individual observer to analyse each one.
– If a psychologist had conducted an interview, the
researcher identifies a theme (e.g. being upset) and
then analyses the entire script to find more examples of
this theme.
• Coding is a thoughtful process and not at all superficial. The
categories or themes are decided upon in two ways:
– Top-down approach (thematic analysis)
– Bottom-up approach (grounded theory)
Step 2 – Analysing the data
• Top-down approach (thematic analysis)
– When you analyse the data you find themes and
concepts by using an existing theory/explanation.
– For example, the clinical characteristics of schizophrenia
may be used as categories when coding selfdescriptions of patients with schizophrenia.
• Bottom-up approach (grounded theory)
– When analysing the data the codes and categories
emerge from the data. So codes remain grounded in
the observations rather than being generated before the
study.
– You would use grounded theory in an area of
psychology which is new or when developing new
insights.
Step 3 – Summarising the Data
• This is a process where it makes it possible to turn
qualitative data into quantitative data.
• Behavioural categories that are identified are listed
and used later when summarising the data.
• For examples, when analysing data the psychologist
lists the themes or categories, or gives examples of
behaviour within each category using quotes from
participants or descriptions of typical behaviour in
that category. The psychologist then counts the
frequency of each occurrence, turning qualitative
into quantitative data. The psychologist can then
draw conclusions.