HOW TO DESIGN AND REPORT EXPERIMENTS
BY ANDY FIELD & GRAHAM HOLE
Chapter 1: Before You Begin ................................................................................................2
Chapter 2: Planning an Experiment ......................................................................................2
Chapter 3: Experimental Designs ..........................................................................................2
Different methods for doing research ......................................................................................3
Observational methods .........................................................................................................3
Quasi-experimental designs ..................................................................................................3
Experimental research / designs ...........................................................................................4
Chapter 4: Descriptive Statistics ...........................................................................................5
Frequency distribution / histogram ..........................................................................................5
Normal distribution ...............................................................................................................5
Positively skewed distribution ...............................................................................................5
Negatively skewed distribution .............................................................................................5
The mode...................................................................................................................................6
Advantages ............................................................................................................................6
Disadvantages .......................................................................................................................6
The median ................................................................................................................................6
Advantages ............................................................................................................................6
Disadvantages .......................................................................................................................6
The mean ...................................................................................................................................6
Chapter 9: A Quick Guide to Writing a Psychology Lab-Report ..............................................6
Things to include in a lab-report ...............................................................................................6
Title and abstract ..................................................................................................................6
Introduction ...........................................................................................................................7
Method ..................................................................................................................................7
Results ...................................................................................................................................7
Discussion ..............................................................................................................................7
References .............................................................................................................................8
Chapter 11: Answering the Question ‘Why?’ The Introduction Section .................................8
Chapter 12: Answering the Question ‘How?’ The Method Section ........................................8
The Design section .................................................................................................................8
The Participant section ..........................................................................................................8
The Apparatus section ...........................................................................................................9
The Procedure section ...........................................................................................................9
Chapter 13: Answering the Question ‘What Did I Find?’ The Results Section .........................9
The results section ....................................................................................................................9
Descriptive statistics ..............................................................................................................9
Inferential statistics ...............................................................................................................9
Some commonly used statistical symbols and abbrevations .................................................10
Chapter 14: Answering the Question ‘So What?’ The Discussion Section ............................. 10
Chapter 15: Title, Abstract, References and Formatting ...................................................... 10
The Title ...................................................................................................................................10
1
HOW TO DESIGN AND REPORT EXPERIMENTS
BY ANDY FIELD & GRAHAM HOLE
Chapter 1: Before You Begin
This chapter has explored the reason why scientists conduct experiments to answer research
questions. It began by looking at why scientists meaasure things – to make research
conclusions comparable across researchers.
In addition, we started to think about the way in which we measure things and the different
levels of measurement. This led to a discussion of why experimentation is valuable for
isolating cause and effect and to discuss some different philosofical ideas about causality.
Ultimately it was discovered that to isolate causal variables it’s necessary to look at situations
in which the supposed cause is present (an experimental group) and compare it against a
situation in which the cause is absent (a control condition).
Finally, these ideas were used to look at how theories develop using Popper’s idea about
falsification and an example of when a control group might not actually act as a control was
considered.
It was also discovered that one could use probability to determine the ability of someone to
detect when milk is added to tea – which was a bonus!
Chapter 2: Planning an Experiment
The chapter has described some of the initial stages in experimental design. This isn’t the
whole story though, it’s just food for thought.
It looked at how to refine a research question by exploring databases of research material and
how it’s possible to narrow these sources down to a specific question.
It’s not possible to answer everything with one experiment so it’s generally needed to
constraining oneself to a very specific question (and this question must be a scientific
statement: something that is testable).
Once a question is defined, one has to think about what variables to manipulate (the
independent variables) and what one wants to measure (the dependent variables).
In considering how to measure the dependent variables it’s necessary to consider what type
of measure to use (physiological, behavioural or self-report) and how best to construct these
measures. If self-report measures are used then it is important to try to ensure that the
measure produces the same results in identical circumstances (reliability), and that it
measures what is wanted to measure (validity).
Chapter 3: Experimental Designs
There are many issues that have to be considered in designing an experiment, but if one of
the standard experimental designs described in this chapter is used, one should be well on the
way to producing a study which will provide reliable and valid results.
There is no one perfect design, because the optimal design will depend on the particular
circumstances.
However:
2
HOW TO DESIGN AND REPORT EXPERIMENTS
BY ANDY FIELD & GRAHAM HOLE
1. Define precisely what it is you want to measure.
2. Before running the study, think carefully about what statistics you are going to use.
3. Sample the appropriate population to obtain suitable participants. Sometimes you can
get away with using students, sometimes you can’t. You may want to use a particualr
population.
4. Decide which design is most appropriate for your particular circumstances. Keep the
design as simple as you can.
5. Having obtained a result, you have to assess its reliability and its validity. One way to
do this is by using multiple converging operations: investigate the same phenomenon
from different angles by using different techniques, participant groups, dependent
variables etc., to ensure that your results are not specific to your particular original
study.
6. You must behave ethically when you conduct research. Participants must give
informed consent to participating in your study. Avoid deceiving them or exposing
them to physical or mental harm. Once they have finished the experiment, give them
an explanation of what is was all about, and make sure they leave in as good a state of
mnd as they were in when they arrived!
Different methods for doing research
Observational methods



enables the researcher to get a good idea of how people normally/really behave
(which may differ from how they behave in an experiment).
more time-consuming than experiments.
it’s more difficult to identify cause and effect
Quasi-experimental designs

the experimenter doesn’t have complete control over the independent variable (e.g.
age).
Types of quasi-experimental design:
One group post-test design
Treatment
Measurement
One group pre-test / post-test design
Measurement
Treatment
Measurement
Interrupted time-series design
M
M
M
T
M
M
M
time
3
HOW TO DESIGN AND REPORT EXPERIMENTS
BY ANDY FIELD & GRAHAM HOLE
The static group comparison design (the participants are not assigned to the two
conditions randomly)
Group A:
Treatment
Experimental group
Measurement
Group B:
No treatment
Control group
Measurement
Experimental research / designs

randomization is essential (to avoid bias)
Between-groups (or ‘independent measures’) designs:
 use separate groups of participants for each of the different conditions in the
experiment. Each participant is tested once only.
Post-test only / control group design
Group A:
Treatment
Experimental group
Measurement
Group B:
No treatment
Control group
Measurement
Random
allocation
Pre-test/post –test / control group design
Group A:
M
Treatment
Experimental group
M
Group B:
M
No treatment
Control group
M
Random
allocation
The Solomon four-groups design
Group A:
M
Treatment
M
Group B:
M
No treatment
M
Treatment
M
No treatment
M
RA
Group C:
Group D:
An experiment with multiple levels of one independent variable
4
HOW TO DESIGN AND REPORT EXPERIMENTS
BY ANDY FIELD & GRAHAM HOLE
Group A:
M
Treatment
Level A
M
Group B:
M
Treatment
Level B
M
Group C:
M
Treatment
Level C
M
Group D:
M
No treatment
Control group
M
RA
Within-subjects (or ‘repeated measures’) designs:
 each participant is exposed to all of the conditions of the experiment
Basic two-condition repeated measures design
Treatment
M
No treatment
M
No treatment
M
Treatment
M
RA
Chapter 4: Descriptive Statistics
Frequency distribution / histogram

a graph of how many times each score occurs
Normal distribution

a bell-like shaped curve
Positively skewed distribution

the frequent scores are clustered at the lower end
Negatively skewed distribution

the frequent scores are clustered at the higher end
Platykurtic distribution

many scores in the tails and looks flat – the scores are far from the mean
Leptokurtic distribution

thin in the tails and looks pointy – the scores are close to the mean
5
HOW TO DESIGN AND REPORT EXPERIMENTS
BY ANDY FIELD & GRAHAM HOLE
The mode
= the most common score / the score with the highest frequency
 if a data set has two modes it is known as bimodal
 if it has several modes it is called multimodal
A bimodal distribution:
Advantages

can be used with nominal data
Disadvantages

even a single score can dramatically alter the value of the mode
The median
= the middle score of a distribution of scores when they are ranked in order of magnitude
1. Arrange the scores into ascending or descending order
2. Position of middle score is found by Error! (n = the number of scores)
3. If there’s an even number of scores, the median is half way between two, and it is then
found by averaging the two scores Error! x = score
Advantages
 it’s relatively unaffected by extreme scores at either end of the scale (outliers)
 if the distribution is skewed, the median is less affected by this than the mean
 can be used with ordinal, interval and ratio data
Disadvantages


cannot be used with nominal data
large samples are more reliable
The mean
= the sum of all scores divided by the number of scores
 = Error!
Chapter 9: A Quick Guide to Writing a Psychology Lab-Report
Things to include in a lab-report
Title and abstract


a clear and informative title (10-15 words)
the abstract is a clear summary of the study’s aims, methods, findings and conclusions
(max. 150 words)
6
HOW TO DESIGN AND REPORT EXPERIMENTS
BY ANDY FIELD & GRAHAM HOLE
Introduction



summarize relevant experimental findings and theories which relate to the aims of
your experiment. Use this information to provide a justification for why your
experiment is worth doing.
outline your proposed experiment
make specific predictions about the outcome of the experiment, on the basis of the
literature you have reviewed.
Method





include sub-sections on: design, participants, apparatus and procedure
make sure there is enough relevant detail for the reader to be able to repeat the
experiment purely by reading the method section
Design section: identify the independent and dependent variables, and say whether
you used independent measures, repeated measures or a mixed design
give relevant background characteristics of the sample of participants, as well as saying
how many took part
write the apparatus and procedure section in proper English, and not like a recipe
Results








make sure you have clearly described the results and explained whether the evidence
(in general) supports the hypothesis under considerations. Describe them, but leave
interpretations (in terms of relationship to theories and previous experimental work)
until the discussion section.
if you have a fair amount of numerical data, put it in a table or graph
number the tables and figures so that you can refer to them in the text. Figures and
tables are numbered independently of each other
make sure each table and figure is clearly labelled and has a self-explanatory title
make sure tables and graphs are intelligible without reference to the text, and vice
versa
where inferential statistics are used, indicate the statistic that was used (e.g. t, F, etc.).
Give the value of the statistic used, the number of degrees of freedom, the level of
significance reached and whether the test was one-tailed or two-tailed.
put raw data and statistical calculations in an appendix, not in the main text
remember to include means and standard deviations (or medians and ranges or semiinterquartile ranges, if these are more appropriate).
Discussion





summarize the main results
provide some interpretation of what your results mean, in theoretical terms
indicate clearly whether or not your initial hypothesis has been accepted
discuss your own data with reference to other experimental findings and theories in
the area, particularly those summarized in the introduction
identify potential problems with your study, but don’t produce a litany of trivial
criticisms. Make intelligent suggestions for future studies.
7
HOW TO DESIGN AND REPORT EXPERIMENTS
BY ANDY FIELD & GRAHAM HOLE
References


give only the surname(s) of the author(s) and the date of the publication in the text,
unless it is a direct quote – in which case give the number of the page on which the
quote can be found
if a source you have used (e.g. Smith, 1991) cites an author to whom you wish to refer
(e.g. Bloggs, 1950), it must appear in the text as follows: ‘Bloggs (1950, cited in Smith,
1991)’. Smith (1991) should be the reference which appears in the list at the end of
your report, not Bloggs.
Chapter 11: Answering the Question ‘Why?’ The Introduction Section





the introduction’s main purpose is to explain to the reader the reasons why you did
your study. What has been done in this area before? Why is your study worth doing?
introductions normally have three main parts: a description of relevant previous
research and theories; a brief outline of your experiment; and an informal summary of
what you expect to find and what your findings might imply theoretically.
reasons for doing a study include all or some of the following: to test a theory;
replicate or extend previous findings; or to try to resolve anomalous or contradictory
findings from previous work.
in referring to previous research, make sure you stick to studies which are directly
relevant to your own, and avoid going into too much detail about them.
the introduction ends with a statement of what you expect to find
Chapter 12: Answering the Question ‘How?’ The Method Section
The ‘Method’ section sub-divides into the ‘Design’, ‘Participants’, ‘Apparatus’ and ‘Procedure’
sections. Together they give the reader enough details for them to replicate the study should
they wish to do so.
The Design section



outlines the formal structure of the study
identifies the experiment’s independent and dependent variables
tells the reader whether you used independent-measures, repeated-measures or
mixed design
The Participant section

gives relevant details of the people who took part in the experiment
8
HOW TO DESIGN AND REPORT EXPERIMENTS
BY ANDY FIELD & GRAHAM HOLE
The Apparatus section

provides information on what equipment you used to rum your experiment.
(‘Equipment’ may include questionnaires, pencil and paper tests, computers etc.)
The Procedure section

describes how you actually carried out the study in practice
Chapter 13: Answering the Question ‘What Did I Find?’ The Results Section
The results section




this section tells the reader what you found, but leaves the interpretation of your
results for the next section (the Discussion).
you may need to ‘tidy’ your data before analyzing them, but this has to be done in an
honest way – you shouldn’t discard participants data just because they didn’t fit in
with what you expected to find!
this section will normally consist of descriptive statistics and inferential statistics.
statistical calculations (whether by hand or computer) and raw data should not appear
in the results section: if they must be included, put them in an appendix.
Descriptive statistics

may be presented within the text of the results section, in a table, or in a graph. If you
use tables or graphs, they should be labelled clearly enough for them to be
understandable without reference to text of the report.
Inferential statistics

are usually (but not invariably) presented within the text of the results section. They
should be presented in a standard way, and accompanied by a brief explanation of
what they show.
9
HOW TO DESIGN AND REPORT EXPERIMENTS
BY ANDY FIELD & GRAHAM HOLE
Some commonly used statistical symbols and abbrevations
ANCOVA
df
H
LSD
MANOVA
MS
N
ns
R
r2
SD
SS
U

<
X2
Analysis of Covariance
Degrees of freedom
Kruskal-Wallis test statistic
Fischer’s Least Significant Difference
test
Multivariate Analysis of Variance
Mean Square
Total number in a sample /
Numbers of members in an entire
sample
Non-significant
Multiple correlation
Coefficient of Determination
(Pearson’s correlation squared)
Sample standard deviation
Sum of squares
Mann-Whitney test statistic
Alpha: probability of a Type 1 error
less than
Chi-squared test statistic
ANOVA
F
HSD
M
Analysis of Variance
F-ratio (ANOVA test statistic)
Tukey’s Honestly Significant
Difference test
Mean
Mdn
MSE
n
Median
Mean Square Error
Numbers of members in a subset
taken from en entire sample
p
r
rS
Probability
Pearson’s correlation coefficient
Spearman’s correlation (‘rho’)
SE / SEM
T


>
Standard error (of measurement)
Wilcoxon’s test statistic
Sigma: Population standard
deviation
Beta: probability of a Type 2 error
greater than
Chapter 14: Answering the Question ‘So What?’ The Discussion Section
The purpose of the Discussion is to interpret the results that you have found, in relation to
previous research: how do your findings fit in with what’s already known on the topic in
question?
The Discussion has three main parts:
1. A brief recapitulation of your main findings
2. A discussion of how these results fit in with previous research in this area
3. A discussion of possible problems and limitations with your study, perhaps with
intelligent suggestions for future research.
As with the Introduction, make sure that the research and theories that you describe are
relevant to your own study.
Chapter 15: Title, Abstract, References and Formatting
The Title


10 to 12 words long
include a mention of the independent variables and the dependent variables in the
study
10
HOW TO DESIGN AND REPORT EXPERIMENTS
BY ANDY FIELD & GRAHAM HOLE
11