DESIGNING PSYCHOLOGICAL INVESTIGATIONS
To read up on designing psychological investigations, refer to pages 699–720 of Eysenck’s A2 Level
Psychology.
Ask yourself
 What is the difference between a quantitative and qualitative approach to
data collection?
 What is the difference between reliability and validity?
 Can you remember which studies raised ethical issues from your studies at
AS level?
What you need to know
THE
SELECTION
AND
APPLICATIO
N OF
APPROPRIA
TE
RESEARCH
METHODS







Labor
atory
experi
ments
Field
experi
ments
Quasiexperi
ments
Natur
al
experi
ments
Correl
ationa
l
analy
sis
Obser
vation
al
techni
ques
Self-
EXPERIMEN
TAL AND
NONEXPERIMEN
TAL
DESIGNS

Exper
iment
al
resea
rch
desig
ns:
indep
ende
nt
group
s
desig
n,
matc
hed
partic
ipant
s
desig
n,
repea
ted
meas
ures
THE
STRENGTH
S AND
WEAKNESS
ES OF
DIFFERENT
FORMS OF
SAMPLE
METHODS



Rand
om
samp
ling
Oppo
rtuni
ty
samp
ling
Volu
nteer
samp
ling
RELIABILI
TY:
ASSESSME
NT AND
IMPROVE
MENT


Inter
nal
and
exte
rnal
relia
bilit
y
Tech
niqu
es to
chec
k
relia
bilit
y
VALIDITY:
ASSESSMENT
AND
IMPROVEMENT


Internal/e
xperiment
al validity
Checking
validity
ETHICAL
CONSIDE
RATIONS
AND
RESOLUT
IONS IN
PSYCHOL
OGICAL
RESEARC
H


Th
e
BP
S
gui
del
ine
s
Eth
ical
iss
ues
an
d
the
ste
ps
tak
en
to
dea
l
wit
h
the
repor
t
techni
ques
Case
studie
s
desig
n
 Nonexper

iment
al
resea
rch
desig
ns:
obser
vatio
n,
interv
iews,
quest
ionna
ires,
pilot
studi
es
THE SELECTION AND APPLICATION OF APPROPRIATE RESEARCH METHODS
m
Research methods take either a quantitative or qualitative approach, which depends on whether the data
collected is numerical or non-numerical. Thus, quantitative = numbers and qualitative = words. To decide
which method is the most suitable, careful consideration needs to be given to the strengths and weaknesses
of the research methods.
Laboratory experiments
The laboratory experiment takes place in a controlled environment and enables the experimenter to test the
effect of the IV (independent variable) on the DV (dependent variable). To establish a difference and so
detect cause and effect relationships, the IV is systematically varied between two conditions.
Advantages
Weaknesses
Cause and effect can be established
Lack mundane realism
Objective and so less researcher biased
Reductionism
Field experiments
Field experiments take place in natural settings, e.g. the work environment. The experimenter has control of
the IV and so causal relationships can be established. Participants are often not aware they are being
researched.
Advantages
Weaknesses
Have mundane realism
Confounding variables
Cause and effect
Ethical issues
Quasi-experiments
Quasi-experiments take place when the experimenter cannot control the IV—it is said to be naturally
occurring. For example, experiments involving gender, age, class, or cultural differences would be classed
as quasi because the experimenter cannot manipulate these factors as the IV. However, the experimenter
does have control of the research setting.
Advantages
Weaknesses
Can research phenomena that could not Cause and effect not established
otherwise be investigated experimentally
Controlled conditions
Natural experiments
Reductionist
A natural experiment is kind of quasi-experiment but the researcher has no control over the IV or the
research setting.
Advantages
Weaknesses
Can research phenomena that could not Cause and effect not established
otherwise be investigated experimentally
Have mundane realism
Confounding variables
Correlational analysis
Correlational analysis is a technique that measures the strength of the relationship between two variables.
The paired scores of the two variables are analysed to establish the strength and direction of the association,
e.g. the relationship between stress and illness. This can be illustrated visually through scattergrams and
numerically through correlation coefficients. These range from +1 to 0 to -1, where the sign shows the
direction and the number shows the strength of the association.
Advantages
Weaknesses
Shows direction and strength of
relationships
Cause and effect not established
It can be used when an experiment
cannot
Other factors may be involved
Observational techniques
Naturalistic observation
Naturalistic observation involves examining behaviour in a natural setting with minimal intrusion from the
researcher because it aims to observe people’s natural behaviour. Participants might be aware they are
being observed (overt observation), or might not (covert observation).
Advantages
Weaknesses
Has mundane realism
Observer bias
Less biased by demand characteristics
Describe behaviour but do not explain it
Controlled observation
Controlled observation takes place when the researcher has control of the environment in which the
observation occurs.
Advantages
Weaknesses
Less biased by confounding variables
Artificial
Richer data
Researcher and participant effects, e.g.
Hawthorne effect
Self-report techniques
Interviews
As a form of self-report technique, interviews take many different forms: they usually take place face to
face and can yield rich, in-depth data.
Non-directive interviews
These are led by the participant (the interviewee), who is free to discuss whatever he or she chooses. The
interviewer guides the discussion by encouraging the interviewee to elaborate on responses. Such
interviews tend to be used in the treatment of mental disorders but have little relevance to psychological
research.
Informal interviews
The interviewer has a list of topics that are to be discussed. Informal interviews resemble non-directive
interviews in that the interviewee is allowed to discuss the topics in whatever way and order he or she
chooses, with the interviewer mainly encouraging more depth and detail.
Semi-structured or guided interviews
All interviewees are asked precisely the same questions in the same order. Thus, these interviews possess
more structure than informal interviews because the interviewer takes control of the issues to be discussed
and decides in what order they will be covered.
Clinical interviews
These are often used by clinical psychologists to assess patients with mental disorders. All interviewees are
asked the same questions but follow-up questions depend on the answers given. This gives the interviewer
flexibility to explore and follow-up on interesting answers.
Structured interviews
A standard set of questions is asked in exactly the same order and the interviewees have a restricted range
of answers to select from.
Advantages
Weaknesses
Rich detailed data
Interviewer bias
Flexible due to different formats
Participant effects
Questionnaires
Written questionnaires are a type of interview. They can be conducted face to face, via the telephone, or by
post. A questionnaire consists of a standard set of questions, which are either closed (fixed response, e.g.
rating scales) or open-ended (requiring detailed responses). Questionnaires are used to survey attitudes,
beliefs, and behaviour.
Advantages
Weaknesses
Flexible as can use open and closed
questions
Researcher bias, e.g. leading questions
Quick and economical
Participant effects
Case studies
A case study is the in-depth study of an individual or small group. Examples that you will have come
across during your studies include: case studies of abnormality (e.g. Little Albert and Anna O), case studies
of brain damage (e.g. HM), and case studies of privation (e.g. Genie).
Advantages
Weaknesses
In-depth data
Generalisability
Influences future theoretical
developments
Subjectivity
EXPERIMENTAL AND NON-EXPERIMENTAL DESIGNS
To account for the application of research methods you need to be able to explain how different methods
are implemented.
Experimental research designs
There are three experimental designs that aim to control participant variation (i.e. individual differences
between the participants) that could interfere with the effect of the IV (independent variable) on the DV
(dependent variable). The three designs share a common characteristic of experiments: there are two
conditions, the control and experimental, and the IV is varied across these. The three designs are discussed
below.
1. Independent groups design
The independent groups design involves two or more groups of different participants. Thus, there are
different participants in each of the conditions, and each participant experiences only one condition.
Advantages
Weaknesses
Avoids order effects
Participant variables, i.e. individual
differences because there are different
participants in each group
Random allocation is possible
Need a larger number of participants
2. Matched participants design
The matched participant design is as it sounds—the participants in each condition are matched on certain
relevant variables. The participants experience only one condition. Thus, there are two groups, which are
matched, and each participant experiences a different condition. This involves matching the participants on
a one-to-one basis, not simply matching the groups as a whole.
Advantages
Weaknesses
Minimises participant variables
Cannot eliminate participant variables
Order effects are avoided
Difficult to achieve a good match
3. Repeated measures design
In the repeated measures design the same participants experience both conditions; thus, there is one group
of participants who take part in both conditions.
Advantages
Weaknesses
Minimises participant variables
Order effects. These are the effects of
participating in two conditions, e.g.
practice, fatigue, boredom
Fewer participants are needed
Demand characteristics are easier to
guess
Deciding which design to use often comes down to choosing between independent
measures and repeated measures, because matched participants is often just too
difficult to carry out economically. To decide which design is best, a decision has to
be made based on the key weaknesses of the two designs in terms of which will least
affect validity, participant variables (independent measures) or order effects,
including demand characteristics (repeated measures).
Non-experimental research designs
Observation design
Several factors need to be taken into account when performing non-experimental research. First,
researchers need to decide whether to conceal themselves (covert) or not (overt), and this will depend on
what is being investigated. A second consideration is whether to conduct a participant or a non-participant
observation. Participant observation is when the researcher becomes a member of the group under
observation in order to observe more natural behaviour. Reliability is a key design consideration as
observers must be consistent in their judgements if the data are to be reliable and valid. Behavioural
categories of what is being observed must be constructed so that observations are reliable (consistent), i.e.
are two observers categorising the same behaviour in the same (i.e. reliable) way? Reliability can be
assessed by having two or more observers and comparing their observations as a measure of inter-observer
reliability (sometimes known as inter-rater or inter-judge reliability).
Interviews
The first consideration is the format of the interview, which can be structured, semi-structured, or
unstructured. This format determines how researcher led the interview is versus how participant led the
interview is. A key issue is the construction of good questions. This is complex because it is important that
the questions are clear and unambiguous—as if they communicate different meanings to different
participants then the answers will not be comparable. In order to avoid leading the participant, the questions
should also be free from bias and subjectivity.
Questionnaires
Questionnaire design involves deciding on the format of the questions.
Closed questions involve a fixed response, which the participant must choose from, for example, yes/no
answers or a Likert scale. This generates quantitative data, which are easier to score and analyse.
Open questions allow the participants to answer freely and so qualitative analysis is needed, which can be
more difficult and time consuming, but can also yield more meaningful data.
As with the interview, a key design issue is the construction of questions that are free from ambiguity and
bias. Another design issue is the construction of Likert scales for self-report ratings. A Likert scale consists
of a five-point scale to indicate level of agreement or non-agreement with whatever was communicated in
the question. The scoring of positive and negative statements need to be reversed in order for the scores on
the questionnaire to relate to each other.
Pilot studies
Experimental and non-experimental research designs are often preceded by a pilot study, which allows for
a trial run of the materials so that questions can be checked for clarity and ambiguity, and adjusted if there
are problems before the main study. A pilot study also enables the researcher to check the experimental
procedure for design errors and timings. This saves time and money, as if there are issues with materials or
procedures these can be amended before the main study.
THE STRENGTHS AND WEAKNESSES OF DIFFERENT FORMS OF SAMPLE METHODS
Research is conducted on people, and the group of people that the researcher is interested in is called the
target population. However, as it is not usually possible to use all of the people in the target population, a
sample must be selected. Those selected are called participants for research purposes. Thus, research is
conducted on a sample but the researcher hopes that the findings will be true (valid) for the target
population. For this to be the case, the sample must be representative of the target population. If the sample
is representative then the findings can be generalised back to the target population. If not, the findings lack
population validity. Therefore, the key issue is the generalisability of the sample; this is based on two key
factors:
 type of sampling
 size of the sample.
Random sampling
Random methods mean every participant has an equal chance of being selected. They include methods such
as selecting names out of a hat, or everybody in the population being assigned a number and a computer or
random number table being used to generate the numbers that are selected for the sample.
Advantages
Weaknesses
Less biased than samples in which the
Difficult to obtain a truly random sample
Researcher selects the participants
Expensive and time consuming
Opportunity sampling
Opportunity sampling involves selecting anybody who is available at the time of the study to take part. This
is a popular method and as much as 90% of the research discussed in psychology textbooks will have used
this method—participants are mainly undergraduates at American universities who were selected based on
their availability.
Advantages
Weaknesses
Practical and economical
Usually drawn from a restricted target
population
Researcher bias in who is selected as
available
Volunteer sampling
Participants volunteer to take part in a research study, for instance by replying to an advertisement.
Advantages
Weaknesses
Practical and economical
Volunteers are not representative of
general population
RELIABILITY: ASSESSMENT AND IMPROVEMENT
Reliability is based on consistency. Research that produces the same results every time it is carried out is
reliable.
Internal reliability = consistency within the method
Measuring instruments such as rulers are consistent within the method of measurement, as the difference
between 0cm and 5cm is the same as that between 5cm and 10cm. However, Likert rating scales lack such
consistency, as the difference between 1 and 2 on the scale might not be perceived by participants to be the
same as the difference between 4 and 5. Unreliable measures reduce internal validity.
External reliability = consistency between uses of the method
External reliability refers to the consistency of psychological tests over time, i.e. the tests must be
consistent between uses of the measure, which can be checked if the test is taken once and then again on a
later occasion.
Techniques to check internal reliability
Inter-rater reliability (or inter-judge reliability)
Inter-rater reliability is used to test the accuracy of the observations. If the same behaviour is rated the same
by two different observers then the observations are reliable as there is consistency within the observation.
Techniques to check external reliability
Test–retest reliability
Test–retest reliability involves testing once and then testing again at a later date, i.e. replicating the original
research. Meta-analyses draw on this when they compare the findings from different studies that have
tested the same hypothesis, e.g. Milgram’s study of obedience and its variations. Strong consistency
between the different findings (i.e. reliability) indicates validity.
VALIDITY: ASSESSMENT AND IMPROVEMENT
Campbell and Stanley (1966, see A2 Level Psychology page 712) distinguished between internal and
external validity.
Internal validity/experimental validity
Does the research measure what it set out to? Is the effect genuine? Is the independent variable (IV) really
responsible for the effect on the dependent variable (DV)? To be valid, the research must measure what it
claims in the hypothesis, i.e. it must be the IV that causes the effect on the DV. If this happens, the research
has truth because the effect is genuine and is caused by the IV rather than by a confounding variable.
Coolican (1994, see A2 Level Psychology page 712) identified threats to internal validity, i.e. other factors
that could have caused the effect on the DV, such as confounding variables, unreliable measures, a lack of
standardisation, a lack of randomisation, demand characteristics, and participant reactivity.
Good research design increases internal validity as accounting for the above will increase internal validity.
Checking internal validity
If internal validity is high then replication should be possible; if it is low then replication will be difficult.
Thus, validity and reliability are interlinked: if the research has truth (validity) it should be consistent
(reliable) and so replication is possible. Reliability is also an indicator of validity.
Concurrent validity is another means of testing the internal validity of a new test. The scores from the new
test of unknown validity are compared against those of a test in which validity has already been established.
If the scores are similar then the new test is probably valid, i.e. a true measure.
External validity
Coolican (1994, see A2 Level Psychology page 713) identified four main aspects to external validity:
1. Populations: findings have population validity if they generalise to other
populations.
2. Locations: findings have ecological validity if they generalise to other
settings.
3. Measures or constructs: findings have construct validity if the measures
generalise to other measures of the same variable, e.g. does a measure of
recall of word lists generalise to everyday memory?
4. Times: findings have temporal validity if they generalise to other time
periods, e.g. do findings from the past generalise to the current context or do
current findings generalise to the past or future? This is difficult to achieve
as, to some extent, all research is dependent on era and context.
Checking external validity
A meta-analysis involves the comparison of findings from many studies that have investigated the same
hypothesis. Findings that are consistent (reliable) across populations, locations, and periods in time indicate
validity. Thus, if a study has validity then it is likely to replicate, and reliability in the meta-analysis is used
as an indicator of validity. So it would seem that you rarely have one without the other, apart from
consistently wrong findings!
Predictive validity is another means of checking external validity. It involves using the data from a study to
predict behaviour at some point in the future. If the prediction is correct, then this suggests that the original
data did generalise to a future context and so has external validity.
ETHICAL CONSIDERATIONS AND RESOLUTIONS IN PSYCHOLOGICAL RESEARCH
Ethical issues arise when ethical guidelines are breached. The need for ethical controls led to the
establishment of ethical guidelines, i.e. rules that can be used to judge the acceptability of research. Most
countries now have a psychological organisation that has devised its own code of conduct, such as the BPS
(British Psychological Society) guidelines.
The British Psychological Society guidelines for research with human participants
1. Introduction
Ethical guidelines are necessary to ensure psychological research is
acceptable.
2. General
The participants’ viewpoint of research should be considered and so
members of the target population from which the sample will be drawn
should be asked about the acceptability of the research.
3. Consent
Participants’ consent should be informed i.e. they should have full knowledge
of the nature and purpose of the research. A briefing should fully inform
them about the study and advise them of their rights (withdrawal and
confidentiality). If the participant is a child (under 16 years) or impaired,
adult consent must be gained from the parent or from those in loco parentis.
If informed consent was not gained at the outset then the safeguards needed
for such a deception would be as detailed below in the deception guideline.
4. Deception
Deception of the participants should be avoided wherever possible.
Information should not be deliberately withheld and nor should the
participants be misled. Deception should only be used when alternative
procedures, which do not involve deception, have been fully considered and
rejected as unfeasible by independent advisors. Also, participants should be
fully informed at the earliest possible stage and should be consulted in
advance as to how deception would be received.
5. Debriefing
At the end of a study the researcher should provide detailed information
about the research and answer any questions the participants might have.
The researcher should also monitor the participants for unforeseen negative
effects and is responsible for providing active intervention if necessary.
6. Right to withdraw
Participants’ right to withdraw from the study must be clearly communicated
at the outset of the research. Also, participants have the right to withdraw
their consent retrospectively, in which case their data must be destroyed.
7. Confidentiality
In accordance with the Data Protection Act, information disclosed during the
research process is confidential and, if the research is published, the
anonymity of the participants should be protected. If either of these is likely
to be compromised, then the participants’ agreement must be sought in
advance.
8. Protection of participants from psychological harm
Participants should be protected from psychological harm, such as distress,
ridicule, or loss of self-esteem. The risk of harm during the research study
should be no greater than that experienced in everyday life. If there is the
potential for harm then independent approval must be sought, the
participants must be advised, and informed consent gained.
9. Observational research
Studies based on observation must respect the privacy and psychological
well-being of the individuals studied. Consent should be gained unless the
observation is in a public situation where one could expect to be observed by
strangers.
10. Giving advice
Research might reveal physical or psychological problems of which the
participant is unaware. It is the researcher’s responsibility to inform the
participant if it is felt that to not do so would endanger the participant’s
future well-being.
Ethical issues and the steps taken to deal with them
Deception: Ethical issues
Participants in Milgram’s research into obedience were deceived in several ways. Can you think of at least
three of these deceptions? Deception is an ethical issue because it is often considered necessary to avoid
demand characteristics, which would invalidate the findings, and so it might be used despite the potential
harm to the participants.
Resolution: Role-play
One way of avoiding the ethical problems associated with deception is the use of role-playing experiments.
This approach eliminates many of the ethical problems of deception studies, but there is a danger that the
behaviour displayed by role-playing participants is not the same as the behaviour would be if they had been
deceived, and so findings are not as valid. Consider Zimbardo’s (1973) Stanford prison experiment—do
you think the fact that this was a role play affected the validity of the findings?
Debrief
Debriefing is an important method for dealing with deception and other ethical issues because this is an
opportunity to tell participants the actual nature and purpose of the research and so helps resolve the
deception. The debrief should also include the right to withhold data if the participants are unhappy with
the deception.
Informed consent: Ethical issue
Studies that have involved deception lead to the related issue of informed consent. Participants might have
consented to the research but this is not informed consent if they have been deceived. Even in studies such
as Zimbardo’s, where participants were briefed in advance, it is difficult to be sure that the true nature of
the study was grasped, and therefore consent might not have been fully informed and participants might not
have consented had they known the true nature of the research.
Resolution: Seeking presumptive consent
Presumptive consent involves asking the opinion of members of the population from which the participants
in the research are to be drawn. Milgram did this before his experiments.
Prior general consent
Prior general consent (also known as partially informed consent) involves asking participants to take part in
research and revealing that the research involves some deception; thus participants have agreed in advance
that they consent to being deceived about the true nature of the research. Of course this is very vague—they
couldn’t really give informed consent because they lacked detailed information.
Right to withhold data and retrospective consent
Another means of offering informed consent is to do it afterwards. When the experiment is over, during
debriefing, participants should be offered the chance to withhold their data. However, this of course does
not undo any distress participants experienced during the study. Participants who exercise their right to
withhold data might have had experiences during the experiment that they would not have agreed to if they
had realised beforehand what was going to happen to them.
Protection of participants: Ethical issues
The key test of whether or not a participant has been harmed is to ask whether the risk of harm was greater
than in everyday life. Harm includes physical distress, for example, some of Milgram’s participants
experienced physical harm as three had seizures and many perspired and bit their lips. Protection of from
psychological harm covers psychological distress, for example, the studies by Milgram and Zimbardo
demonstrate a lack of protection because of the suffering of the participants: sweating, trembling, and
seizures in Milgram’s study, and crying, screaming, and depression in Zimbardo’s study. Participants may
also have suffered long-term harm if they experienced a loss of self-esteem as a result of the study.
Resolution: Confidentiality
Confidentiality and the right to privacy protect participants from psychological harm. Confidentiality
means that participants are protected in the write-up of the research as they will not be named and that their
data will only be known to the researcher. Confidentiality is reassuring, particularly when the data is of a
sensitive nature. Right to privacy is a concern when conducting observational research as the participant
should not be observed without informed consent unless it is a situation in which they could be observed by
strangers in everyday life.
Right to withdraw
Participants should be informed of their right to withdraw at the outset of the research so that if they are in
any way distressed they can leave the study and so this can minimise any physical or psychological harm.
Debriefing
Debriefing can be used to reduce any distress that might have been caused by the experiment. Participants
should leave the study in the same state of mind as when they arrived. Do you think this is always
achieved?
Evaluation of the BPS ethical guidelines
 Provide clear guidance. The strength of the BPS Code of Ethics is that they
provide clear guidance.
 Has limited force. However, a weakness is that the code is not law and so has
limited force. The penalty (disbarment from the BPS) is perhaps not severe
enough. It also seems that breaches of the code can be justified and that
decisions as to whether research is justifiable can be researcher biased.
 Local ethical committees. This has led to the increasingly widespread
development of local ethical committees. This means the decision is no
longer as researcher biased but the committees are not without bias as they
most include fellow psychologists, who might be biased in favour of research.
Thus, committees should include non-psychologists and non-expert
members of the public.
Cost–benefit analysis
The double-obligation dilemma
The cost–benefit analysis is a safeguard that should precede all research. It involves weighing-up whether
the ends/findings (i.e. the benefits of the research in terms of increased understanding and applications)
justify the means/methods used to gain the data (i.e. the costs, such as harm to participants). A cost–benefit
analysis raises a double-obligation dilemma because researchers have an obligation both to their
participants and to society.
Evaluation of the cost–benefit analysis
 Difficult to predict outcomes. The cost–benefit analysis has a number of
weaknesses, including the fact that it is difficult to predict outcomes (and so
the potential costs and benefits) because the outcomes of research are not
always clear at the outset.
 Difficult to assess. Another problem is that the assessment of costs and
benefits is difficult, as it is hard to quantify costs and benefits, and such
assessments may be open to researcher bias and value judgements. The cost–
benefit analysis is accused of routinely favouring society over the
participants.
So what does this mean?
There are many decisions to be made in the implementing of research, beginning with which research
method is appropriate based on the strengths and weaknesses of the method. Then there are design issues to
do with the implementing of the different methods, issues of sampling, reliability, and validity, and ethical
considerations.
Over to you
Please see the Example Research Question for this chapter.