Evaluating and Tuning a Static
Analysis to Find Null Pointer Bugs
David Hovemeyer, Jaime Spacco, and William Pugh
Presented by Nathaniel Ayewah
CMSC838P
11/16/2006
Why Simple
• Programmers make simple
mistakes
// org.eclipse.jdt.internal.ui.compare.JavaStructureDiffViewer
Control c = getControl();
if (c == null && c.isDisposed())
return;
• Low False Positive Rate 
• Cannot find all bugs 
Findbugs
Findbugs
INPUT
PROCESSING
• Set of “.class”
files containing
byte-code
OUTPUT
• Bug Pattern
Code
• Source Line
Number
• Configurations
• Descriptive
Message
Detectors
Findbugs Detectors
• Independent of each other
PROCESSING
• May share some resources
• GOAL: Low false positives
• Each detector is driven by a
set of heuristics
Know Your
Bug Patterns
Output
HIGH
SEVERE RISK OF
PROGRAM FAILURE
MEDIUM
ELEVATED RISK OF
PROGRAM FAILURE
LOW
LOW RISK OF
PROGRAM FAILURE
Source: US Department of Program Security
Findbugs Detectors
PROCESSING
Null Pointer Analysis
Null Pointer Analysis
• Forward intra-procedural
• Build Control Flow graph for
each method
Slot
Method parmeter,
local variable, or
stack operand
Null
NonNull
Data-flow Frame
PROCESSING
Simple Analysis
Detector foo = null;
foo.execute();
HIGH
SEVERE RISK OF
PROGRAM FAILURE
Detector foo = new Detector(…);
foo.execute();

Dereferencing
Null
Dereferencing
NonNull
If only it were that simple…
• Is a method’s parameter null?
void foo(Object obj) {
int x = obj.hashcode();
…
}
• Infeasible Paths
Infeasible Paths
• Guard indirectly connected to
null check
boolean b;
if (p != null)
b = true;
else
b = false;
if (b)
p.f()
Infeasible Paths
• Assertions
p = null;
...
// throws exception if p null:
checkAssertion(p != null);
p.f(); // safe
Infeasible Paths
• Checked Exceptions that are
never thrown
Foo dup = null;
try {
dup = super.clone();
} catch (CloneNotSupportedException e) {
// Can’t happen
}
dup.contents = ...
Solution
• Null and NonNull are not enough
Checked
NonNull
No Kaboom
NonNull
if (b) {
} else {
NonNull
}
C?
Null-E
Null
NCP
NSP-E
NSP
A
B
Solution
• Dereferencing a variable that has value
Null, NSP, …
MEDIUM
ELEVATED RISK OF
PROGRAM FAILURE
…
Null-E
HIGH
Null
NSP-E
SEVERE RISK OF
PROGRAM FAILURE
LOW
LOW RISK OF
PROGRAM FAILURE
NSP
MEDIUM
ELEVATED RISK OF
PROGRAM FAILURE
Solution
• Choosing a value for a variable
after each statement:
Statement
p = null
p = this
p = new ...
p = "string"
p = Foo.class
p = q.x
p = a[i]
p = f()
Value of p
Null
NonNull
NonNull
NonNull
NonNull
NCP
NCP
NCP
Solution: Infeasible Paths
p = null;
Null
Null
p = new …
Null
NonNull
NSP
NCP
NCP
Solution: Infeasible Paths
Null or NSP
checkAssertion(p != null)
NCP
p.f()
Solution: Infeasible Paths
try {
Null
NSP
Null-E
NSP-E
}
catch(Exception e) {
}
Comparing a Value to null
foo.execute();
if (foo != null) {
...
}
HIGH
SEVERE RISK OF
PROGRAM FAILURE
Comparing
No-Kaboom to
null
Comparing a value to null
R.I.P
R.I.P
Detector foo = null;
if (foo != null) {
foo.execute();
}
if (foo != null) {
...
if (foo == null) {
foo = new ...
}
}
MEDIUM
ELEVATED RISK OF
PROGRAM FAILURE
MEDIUM
ELEVATED RISK OF
PROGRAM FAILURE
Comparing Null
to null
Comparing
Checked
NonNull to null
Other Solutions
• Check for methods that
unconditionally dereference
parameters
• Annotations
– @NotNull: parameter/return
value must not be null
– @CheckForNull: check the
parameter/return value before
dereferencing it
Experiments: Student Code
• With Annotations
Project
NPE
Warning
FN %
Search Tree
Web Spider
71
162
38
127
46
21
Project
Warning
NPE
FP %
Search Tree
Web Spider
40
129
36
101
10
21
Experiments: Student Code
• Without Annotations
Project
NPE
Warning
FN %
Search Tree
Web Spider
71
162
1
47
98
70
Project
Warning
NPE
FP %
Search Tree
Web Spider
2
77
2
75
0
2
Experiments: Production Code
• Cannot calc. false negatives! 
Warning type
Serious
False
FP %
Null dereference
No-Kaboom RCN
Other RCN
73
33
15
16
15
17
18
31
53
Eclipse 3.0.1
Conclusion
• More inter-procedural
techniques could find more
bugs
• But often finding simple bugs
with low FP rate is effective