Cleanroom Method
CS 415, Software Engineering II
Mark Ardis, Rose-Hulman Institute
March 20, 2003

Outline
1.
Harlan Mills
2.
Cleanroom method
3.
Industrial use of cleanroom
2

Harlan Mills
1919 - 1996
3

Mathematics and Programming
Roman accounting
"to go from programming as an instinctive, intuitive process to a more systematic, constructive process"
4

Cleanroom Method
Incremental (spiral)
Box structure specification and design
Design verification
No debugging
Statistical testing
5

Box Structures
Black boxes: behavior only
State Boxes: behavior + state
Clear boxes: procedures
6

Black Boxes
S
1
S
2
...S
n stimulus history
 response
R
7

State Boxes
State Data
S R stimulus, old state
 response, new state
8

Clear Boxes
S
State Data
Procedures
R stimulus, old state
 response, new state
9

Box Description Language (BDL)
Invocation: use <type> <name> <args>
Sequence: do B
1
; B
2 od
Alternation: if <cond> then B
1 else B
2 fi
Iteration: while <cond> do B od
10

Box Structure Hierarchy
BB
SB
CB
BB
SB
CB
BB
SB
CB
BB
SB
CB
11

Cartoon of the Day (1/3)
12

Cartoon of the Day (2/3)
13

Cartoon of the Day (3/3)
14

Design Verification
Procedures in BDL are checked for correctness with their higher-level descriptions
All boxes (and all procedures) describe functions
Formal proofs of correctness can be performed (but often informal proofs are done, instead)
15

Verification of Sequence
Given a high-level function [f] for statement: do [g]; [h] od
Does [g] followed by [h] compute the same function as [f] ?
Example:
[f](x) = 2 * x + 7
[g](x) = 2 * x
[h](x) = x + 7
16

Verification of Selection
Given a high-level function [f] for statement: if <cond> then [g] else [h] fi
1.
Whenever <cond> is true, does [g] compute the same function as [f] ?
2.
Whenever <cond> is false, does [h] compute the same function as [f] ?
17

Verification of Iteration
Given a high-level function [f] for statement: while <cond> do [g] od
1.
Whenever <cond> is true, does [g] followed by [f] compute the same function as [f] ?
2.
Does the loop always terminate?
3.
Whenever <cond> is false, does the empty function compute the same function as
[f] ?
18

Usage Testing
Develop an operational profile of use
Generate random tests that fit the probabilities
19

Example
Function
Update
Usage
Probability
Distribution
Interval
32% 0-31
Delete
Query
Print
14%
46%
8%
32-45
46-91
92-99
20

Test Generation
1
2
3
4
Test
Random
Numbers Test Cases
29, 11, 47,
52, 26, 94
U, U, Q,
Q, U, P
62, 98, 39,
78, 82, 65
83, 32, 58,
41, 36, 17
Q, P, D,
Q, Q, Q
Q, D, Q,
D, D, U
36, 49, 96,
82, 20, 77
D, Q, P,
Q, U, Q
21

Industrial Use
Used in a few areas of IBM
Used by some military contractors
Tried at NASA
22

Software Engineering
Laboratory (SEL)
Joint program of NASA Goddard Space
Center, Computer Sciences
Corporation, and the University of
Maryland
Conduct experiments and case studies on new software technology
23

SEL Experience
First trial at University of Maryland
 controlled experiment (10 experiment teams,
5 control teams

FORTRAN

1.5 KLOC
3 case studies at Goddard
 flight-dynamics ground support systems

FORTRAN

40 KLOC, 22 KLOC, 160 KLOC
24

SEL Results –
University Experiment
Cleanroom teams
 use fewer computer resources
 satisfy requirements more successfully
 make higher percentage of scheduled deliveries
25

SEL Results – Goddard
More effort spent in design
Better reliability of final product
Smaller projects achieve higher productivity, but large project just average
26

Summary
Cleanroom may be an effective method for achieving higher reliability
Requires some culture change (no debugging)
Still being investigated by researchers and practitioners
27

References
Victor Basili and Scott Green, "Software process evolution at the SEL", IEEE
Software 11(4) , 58-66, July 1994.
28