Center for Biofilm Engineering
Ruggedness Assessment
and Experimental Design
in the Biofilm Laboratory
Al Parker
Statistician and Research Engineer
Montana State University
CBE workshop – July 2009
Acknowledgments
 Colleagues in the CBE; esp., the SBML team
 Funding
 US EPA
 Montana Board of Research,
Commercialization and Technology
 Industrial Associates of the CBE
 Big Sky Statistical Analysts LLC
Statistical thinking
 Data
 Ruggedness Testing
 2p Factorial versus One-at-a-time design
 Uncertainty assessment
Statistical thinking
 Data
 Ruggedness Testing
 2p Factorial versus One-at-a-time design
 Uncertainty assessment
Grow: CDC biofilm reactor
Sampling
1. Rod is removed
from reactor
2. Coupon is removed
from the rod
3. Rinse
Treat: disinfecting an established biofilm
Sample: harvest from coupon, disaggregate
K. Moll 2008
harvest biofilm by
scraping with a wooden
applicator stick
K. Moll 2008
homogenize to
disaggregate clumps
Sampling and analysis
Treated coupon
Dilution series; plate in
duplicate or triplicate
Biofilm is scraped & rinsed
into the dilution tube; the
suspension is disaggregated
Drop plate: Viable
cell density (cfu/cm2)
A. Hilyard, 2008
Statistical thinking
 Data
 Ruggedness Testing
 2p Factorial versus One-at-a-time design
 Uncertainty assessment
Ruggedness
A standard laboratory method is said to be rugged
if the outcome is unaffected by slight departures
from the protocol.
Parameters in the protocol
 Disaggregation: sonicated or homogenized
 Nutrient (TSB, continuous flow): 50, 100, 200 mg/l
 Rotation (stir plate): 125, 225, 325 rpm
 Temperature: 20, 23, 26 oC
 Time in batch mode: 3, 18, 24 hr
Ruggedness
Is efficacy testing using the CDC reactor rugged
with respect to changes in the batch time over
which the biofilm was grown?
Viable cell density
(log scale)
Ruggedness with respect to batch time > 18 hours
107
105
103
0
18
Time (h)
© 2002 CBE
Performing a ruggedness test
1. Conduct a minimal number of experiments to:
 Identify unimportant parameters: Is the biofilm
significantly influenced by all 5 parameters?
 Check for interactions among parameters
2. Conduct another series of experiments using only the
influential parameters and interactions.
Ruggedness
Is efficacy testing using the CDC reactor rugged
with respect to changes in stir plate rotation and
temperature?
Full factorial design: 2 factors, each at 3 levels
Temperature (o C)
26
23
20
125
225
Stir plate rotation speed (rpm)
325
© 2002 CBE
Statistical thinking
 Data
 Ruggedness Testing
 2p Factorial versus One-at-a-time design
 Uncertainty assessment
One-at-a-time: Study temperature
Temperature (o C)
26
23
20
125
225
Stir plate rotation speed (rpm)
325
© 2002 CBE
One-at-a-time: Study rpm
Temperature (o C)
26
23
20
125
225
Stir plate rotation speed (rpm)
325
© 2002 CBE
One-at-a-time design for 2 factors
Temperature (o C)
26
23
20
125
225
Stir plate rotation speed (rpm)
325
© 2002 CBE
22 factorial design
Temperature (o C)
26
23
20
125
225
Stir plate rotation speed (rpm)
325
© 2002 CBE
A factorial design is superior to a one-at-atime design:
 Factorial design can detect an important
interaction between the two factors;
the one-at-a-time design can’t
 Factorial design has greater precision when
estimating the main effects of each factor than
the one-at-a-time design
True Mean log(cfu / cm^2)
True mean log(density) increases
with temperature; slope depends on RPM
8.5
RPM
125
8.0
225
325
7.5
20
23
Temperature
26
© 2002 CBE
True Mean log(cfu / cm^2)
One-at-a-time design can estimate these
four points only
8.5
RPM
125
8.0
225
325
7.5
20
23
Temperature
26
© 2002 CBE
True Mean log(cfu / cm^2)
One-at-a-time design cannot detect the
fact that the slope depends on RPM
8.5
RPM
125
8.0
225
325
7.5
20
23
Temperature
26
© 2002 CBE
True Mean log(cfu / cm^2)
Factorial design can estimate these four
points only
8.5
RPM
125
8.0
225
325
7.5
20
23
Temperature
26
© 2002 CBE
True Mean log(cfu / cm^2)
Factorial design can detect the fact that
the slope depends on RPM
8.5
RPM
125
8.0
325
7.5
20
23
Temperature
26
© 2002 CBE
However … the factorial approach requires
more experimental effort
Batch time
One-at-a-time
© 2002 CBE
Temperature
However … the factorial approach requires
more experimental effort
Batch time
One-at-a-time
© 2002 CBE
Temperature
Factorial
Batch time
Can use fewer factorial runs; however, can’t
estimate all interactions
© 2002 CBE
Temperature
½ Fraction of
23 Factorial
For Five factors, instead of three:
how many experimental runs?
25 factorial design: 25 = 32
One-at-a-time design: 2*5 = 10
¼ fraction of the 25 factorial design: 32* ¼ = 8
Statistical thinking
 Data
 Ruggedness Testing
 2p Factorial versus One-at-a-time design
 Uncertainty assessment
Differences among experiments
is the major source of variation
 67% attributable to between experiments
 33% attributable to within experiments
Formula for the SE of the mean LR,
averaged over experiments
2
Sc = within-experiment variance of control coupon LD
Sd2 = within-experiment variance of disinfected coupon LD
SE2 = between-experiments variance of LR
nc = number of control coupons
nd = number of disinfected coupons
m = number of experiments
SE of mean LR =
2
Sc
nc • m
+
2
Sd
nd • m
+
2
SE
m
Where to invest effort to get the most precision?
 2 experiments; 6 coupons each;
SE of mean log density (12 coupons) = 0.24
 4 experiments; 2 coupons each;
SE of mean log density (8 coupons) = 0.19
 The precision is increased by running more
experiments with less effort per experiment
Summary
 It is important to do an arm-chair experiment
first
 Use 2p (fractional) factorial design to determine
ruggedness of the protocol to changes in
parameters
 Rely on multiple, independent
experiments, each with few samples, in contrast
to one experiment with many samples
Fin
Reliable laboratory methods