Designing Experiments
Diverse applications, common
principles
The story of an experiment
• Aim to find out of ‘Boreproof’ is
more resistant to stem borer than
M512
• D1: plant a field with Boreproof.
• D2: plant a field of Boreproof and
another of M512
• Observed damage:
50% (M512), 20% (Boreproof)
But we know damage varies widely
Objective
Comparison
Treatments
Experimental
units
• D3: 4 fields of Boreproof
and 4 of M512
• Observe:
Replication
– M512:
50, 60, 40, 35
– Boreproof: 20, 10, 40, 30
Precision
• D4: Unit = 10x10 plot. One
pair in each field. M512 on
the lefthand plot in each,
Boreproof on the right
Blocking
Allocation
• Observe
• Consistent
differences. But
effect may not
be treatment!
• D5: as D4 but
with random
allocation of
treatments to
units
Field
M512
Bore
proof
1
50
20
2
20
10
3
30
20
4
60
30
5
60
40
6
20
5
7
0
0
8
40
10
Consistency
Confounding
Randomisation
Same principles
•
•
•
•
•
Field experiments
On-farm participatory experiments
Lab experiments
Social and institutional experiments
….
getting it right
• Understand and use experimental design
principles
• Look at papers and reports describing how
others have done it
• Refer to experienced researchers working on
similar topics or methods (different region)
• Observe and critique other experiments
• Do a pilot experiment
• Use imagination!
Then…
•
•
•
•
Think!
Use all available sources of help
Show design to others and get comments
Envisage the data and the way you will use it to
reach conclusions.
– Draw empty (or expected) tables and graphs
• Look at practical implications
• Iterate
• Think!
Concepts and mistakes
Objectives
•Vague
•‘Objective is to compare
treatments’
•Too many conflicting objectives in
one trial
–eg biophyisical and farmer
assessment
Treatments
•Contrasts
•Controls
•Factorial structure
•Quantitative
levels
Units
•Size and shape
•Design to
measure
•Interference
•Multiple units
•Extra treatments ‘because
they might be interesting’
•Omitting suitable control or
baseline treatments
•Too many levels of
quantitative factors
•Plots too small for realistic
application and measurement
of treatments
•Non-independent responses
•Over-use of split plots
Replication
•Estimating precision
•Controlling precision
•Insurance
•Extending range of
results
•Using the ‘usual’
number of reps
•Forgetting hidden
replication
•Insisting on equal
replication for all
treatments
•Forgetting the rules
apply to all levels of
units
•Assuming sub-samples
are replicates
Site(s)
•Single site
•Multiple sites
•Using the default site
•Ignoring requirements
of objectives
Blocking
•Assuming only useful in
•Increasing precision by field experiments
controlling variation
•Limited use of
incomplete block
designs
Randomisation
•Assuming only applied
to field experiments
•Omitting to randomise
at some levels of design
Management
•Every aspect of
preparing, implementing,
measuring…
•Management not
appropriate for
objectives (eg level of
inputs)
•Management
confounded with
treatments
•Failure to maximise
precision through
uniform management
The protocol
written plan of the experiment
The protocol
• Written plan of the experiment
• A protocol should be:
– prepared for every experiment, however small
– written, not in your head
– shared with others who can help improve it
• experience from similar problems, methods, species,
ecozones,...
– detailed enough for someone else to take over the
trial
– kept up to date
• plans change during the execution
• a record of what actually done
– archived with the data
• kept up to date
– plans change during the execution
– a record of what actually done
• archived with the data
The protocol contains...
•
•
•
•
•
•
•
•
•
Identification (name of trial, people,...)
Justification
Objectives
Treatments
Field layout (sites, blocks, plots)
Management
Measurements
Analysis methods
Using the results
Check list for planning on-station
Agroforestry experiments
- rather statistical!
Experiments with Farmers:
Checklist for Preparing Protocols
- better!
Good practice in AF field
experiments
Three changes in research
priorities
1. Landscape effects
2. Change processes
3. Participation