Chapter 26
Cleanroom Software Engineering
1
Cleanroom
• Developed in early 80’s by Harlan Mills
• Reported very good results
– reliable, high-quality software
– inexpensively and quickly produced
• Not used very much
• Moderate use of formal methods
329-27
2
Moderate?
• Extreme use of formal methods
– prove theorems in a formal language
– a program checks all proofs
• Moderate use of formal methods
– prove theorems on a whiteboard
– a group of people talk about the proof until
all are satisfied
329-27
3
Cleanroom increment
•
•
•
•
•
•
•
•
Requirements gathering
Box structure specification
Formal design
Correctness verification (proofs)
Code generation
Code inspection
Statistical use testing
Certification
329-27
4
Testing
• Purpose is to estimate quality
• Purpose is NOT to improve quality
– if there are a significant number of bugs, do
it over
• Tests are generated based on
– what users actually do
– probability that event will occur
329-27
5
Statistical use testing
• Make model of how the system will be
used
• List the set of stimuli that cause the
software to change its behavior
• Estimate the probability of each stimuli
• Generate tests based on probability
329-27
6
The formal part
• Box structure specification
• Formal design
• Correctness verification (proofs)
• Goal: produce code that matches
specification
329-27
7
Box Specification
• BB - black box
– sequence of stimuli (input events)
– response
– rules that map stimuli to response
329-27
8
Black box bank account
• Stimuli
– deposit x, withdraw y, check-balance
• Results
– OK, BOUNCE, BALANCE z
• BankAccount is a function
BankAccount(
stimulusHistory: Seq of Stimuli,
stimulus: Stimuli) -> Results
329-27
9
Black box bank account
Define function balance(Seq of Stimuli)
• balance({}) = 0
• balance(SS+S) =
– if (S = withdraw X) and X <= balance(SS)
then balance(SS) - X
– if (S = deposit X) then balance(SS)+X
– ottherwise, balance(SS)
329-27
10
Black box bank account
BankAccount(stimH, s)
if s = balance then BALANCE stim(H)
else if s = deposit X then OK
else “s = withdrawal X”
if X <= balance(stimH)
then OK
else BOUNCE
329-27
11
Box specification
• SB - state box
–
–
–
–
single stimulus (input event)
response
state
rules that map stimulus and old state to
response and new state
329-27
12
State box bank account
Bank account has one variable: balance
BankAccount(s)
if s = balance then BALANCE balance
else if s = deposit X then balance’ =
balance + X and OK else
if x = withdraw X then if X <= balance
then balance’ = balance - X and OK
else BOUNCE
329-27
13
Boxes
• BB: S, T => R where S is a sequence of
stimuli, T is a stimulus, and R is a result
• SB: S, P => R, Q where S is a stimulus, P
and Q are states, and R is a result.
• CB: Clear box can use any code to specify
the function from stimuli to responses.
329-27
14
Design
• Design is the step of converting a Black
Box or State Box into a Clear Box.
• Clear Box is usually described by
pseudocode.
• For each step of the design, the designers
prove that the step is correct.
• Each kind of step has a rule for proving it
correct.
329-27
15
Code generation
• Once a design is expressed only as Clear
Boxes, it is easy to translate into a
programming language like C or Java.
• The programmers translate the design
into code.
329-27
16
Advantages of Cleanroom
•
•
•
•
•
•
Verification becomes a finite process
Improves quality
Can verify every line of design and code
It results in a near zero defect level
It scales up
It produces better code than unit testing
329-27
17
Near Zero Defect Level?
KLOC,error/KLOC
Ericsson OS-32: 3501
1.7
improvement
HP
3.5 1.4
IBM
107 2.6 486 LOC/PM
IBM
86 1.2
US Army
75 0.8 4.8 improvement
329-27
18
Summary
• If reliability is very important,
Cleanroom techniques should be
considered
• Reasonably efficient of programmer time
• Works for groups of 70 programmers
• Not popular, and there are probably
reasons
329-27
19