University of Southern California
Center for Systems and Software Engineering
Requirements Elaboration
Ali Afzal Malik, Barry Boehm, USC-CSSE
Annual Research Review
March 18, 2008
3/18/2008
©USC-CSSE
1
University of Southern California
Center for Systems and Software Engineering
Outline
•
•
•
•
•
•
•
Motivation
Empirical study
Methodology
Results
Findings
Future work
References
3/18/2008
©USC-CSSE
2
University of Southern California
Center for Systems and Software Engineering
Motivation
• Cost estimation - GIGO
Cost Model
3/18/2008
©USC-CSSE
3
University of Southern California
Center for Systems and Software Engineering
Research Objectives
• Early estimation
– Use goals
• Phenomenon of Requirements Elaboration
– High-level Goals  Low-level Requirements
• Characterize cause of variation in
elaboration ratios
3/18/2008
©USC-CSSE
4
University of Southern California
Center for Systems and Software Engineering
Requirements Elaboration Levels
Use Case Level of Detail Scale
(Cockburn, 2001)
3/18/2008
©USC-CSSE
5
University of Southern California
Center for Systems and Software Engineering
Research Approach
• Related previous work
– Agent-based approach (Letier and van
Lamsweerde, 2002)
– GBRAM (Antón, 1996)
• How is our work different?
– Focus on the process itself
• Obtain detailed project data
• Analyze causes of variation
3/18/2008
©USC-CSSE
6
University of Southern California
Center for Systems and Software Engineering
Empirical Study*
•
•
•
•
•
SE I (Fall) and SE II (Spring)
2004 – 2006
20 real-client, MS-student, team projects
Process: MBASE/RUP
Project selection
– Custom development
– Complete documentation
*A. Malik, B. Boehm, “An Empirical Study of Requirements Elaboration”, Submitted in 16th
IEEE International Requirements Engineering Conference.
3/18/2008
©USC-CSSE
7
University of Southern California
Center for Systems and Software Engineering
Empirical Study (2)
S#
Year
Project
Type
1
2004
Online Bibliographies on Chinese Religions in Western Languages
Web-based database
2
2004
Data Mining of Digital Library Usage Data
Data mining
3
2004
Data Mining from Report Files
Data mining
4
2005
Data Mining PubMed Results
Data mining
5
2005
USC Football Recruiting Database
Web-based database
6
2005
Code Generator – Template based
Stand-alone application
7
2005
Develop a Web Based XML Editing Tool
Web-based application
8
2005
EBay Notification System
Stand-alone application
9
2005
Rule-based Editor
GUI
10
2005
CodeCount™ Product Line with XML and C++
Code Counter Tool
11
2006
California Science Center Newsletter System
Web-based database
12
2006
California Science Center Event RSVP System
Web-based database
13
2006
USC Diploma Order/ Tracking Database System
Web-based database
14
2006
USC Civic and Community Relations (CCR) web application
Web-based database
15
2006
Student's academic progress web application
Web-based database
16
2006
New Economics for Woman (NEW)
Web-based database
17
2006
Web Portal for USC Electronic Resources
Web-based GUI
18
2006
Early Medieval East Asian Tombs
Web-based database
19
2006
USC CONIPMO
Cost model
20
2006
An Eclipse Plug-in for Use Case Authoring
Stand-alone application
3/18/2008
©USC-CSSE
8
University of Southern California
Center for Systems and Software Engineering
3/18/2008
©USC-CSSE
Project Process
9
University of Southern California
Center for Systems and Software Engineering
Methodology
• Relationship between capability goals and
capability requirements
• Documents and milestones considered
– LCO-OCD*: Capability goals at the end of the
Inception phase
– IOC-SSRD#: Capability requirements at the end
of the Construction phase
• 7 metrics
– 4 direct
– 3 derived
* Life Cycle Objectives – Operational Concept Description
# Initial Operational Capability – System and Software Requirements Definition
3/18/2008
©USC-CSSE
10
University of Southern California
Center for Systems and Software Engineering
Metrics
3/18/2008
S#
Metric
Description
1
NCGI
Number of initial capability goals
2
NCGR
Number of capability goals removed
3
NCRD
Number of delivered capability requirements
4
NCRN
Number of new capability requirements
5
NCGA
Number of adjusted capability goals
6
NCRA
Number of adjusted capability requirements
7
EF
Elaboration Factor
©USC-CSSE
11
University of Southern California
Center for Systems and Software Engineering
Derived Metrics
• NCGA = NCGI – NCGR
• NCRA = NCRD – NCRN
• EF = NCRA / NCGA
3/18/2008
©USC-CSSE
12
University of Southern California
Center for Systems and Software Engineering
Results
S#
NCGI
NCGR
NCRD
NCRN
NCGA
NCRA
EF
Group
10
14
2
10
1
12
9
0.75
19
8
1
8
2
7
6
0.86
3
3
1
7
5
2
2
1
16
5
2
3
0
3
3
1
7
10
5
6
1
5
5
1
1
12
3
12
2
9
10
1.11
8
10
2
12
2
8
10
1.25
9
7
4
8
4
3
4
1.33
2
3
0
9
4
3
5
1.67
20
5
2
7
2
3
5
1.67
17
7
1
21
10
6
11
1.83
6
4
1
7
1
3
6
2
4
5
1
14
6
4
8
2
15
5
1
11
3
4
8
2
14
3
0
10
3
3
7
2.33
18
6
0
20
5
6
15
2.5
5
4
1
12
3
3
9
3
13
2
0
11
3
2
8
4
12
6
2
19
2
4
17
4.25
11
8
5
16
3
3
13
4.33
Outliers
WUA
ICA
VUA
3/18/2008
©USC-CSSE
13
University of Southern California
Center for Systems and Software Engineering
Results (2)
18
16
14
12
NCR
A
10
(2)
8
6
(2)
4
(2): 2 data points
2
0
0
1
2
3
4
5
6
7
8
9
10
NCGA
3/18/2008
©USC-CSSE
14
University of Southern California
Center for Systems and Software Engineering
Findings
y = 3.1446x
20
R2 = 0.3262
WUA
ICA
18
VUA
16
14
y = 1.8737x
R2 = 0.9447
12
y = 1.1458x
NCR
A
R2 = 0.9688
10
(2)
8
6
(2)
4
-WUA: Well-understood applications
-ICA: Intermediate clarity applications
2
-VUA: Vaguely understood applications
0
0
1
2
3
4
5
6
7
8
9
10
NCGA
3/18/2008
©USC-CSSE
15
University of Southern California
Center for Systems and Software Engineering
Findings (2)
• Project groups based on EF ranges
– WUA: Well-understood applications
– ICA: Intermediate clarity applications
– VUA: Vaguely-understood applications
Outliers
0
WUA
1
ICA
1.5
VUA
2
EF
EF ranges defining groups
3/18/2008
©USC-CSSE
16
University of Southern California
Center for Systems and Software Engineering
Findings (3)
• No one-size-fits-all formula for
requirements elaboration
• Knowledge of a project’s type gives a hint
about its EF value e.g. all projects in the
VUA group are of type “Web-based
Database”
• Benefits of early determination of a
project’s group
– better estimates
– save time and effort
3/18/2008
©USC-CSSE
17
University of Southern California
Center for Systems and Software Engineering
Future Work
• Relationship between LOS goals and LOS
requirements
• Relationship between current metrics and
those provided by architectural documents
• Relationship between current metrics and
software size metrics
• Analysis of industrial data
3/18/2008
©USC-CSSE
18
University of Southern California
Center for Systems and Software Engineering
• Books
References
– Cockburn, A., Writing Effective Use Cases, Addison-Wesley, 2001.
– Kruchten, P., The Rational Unified Process: An Introduction, AddisonWesley, 2003.
• Conference papers
– A. I. Antón, “Goal-based requirements analysis”, Proc. of the IEEE Int.
Req. Eng. Conf. (RE), 1996, pp. 136–144.
– B. Boehm, “Anchoring the Software Process”, IEEE Software 13(4),
1996, pp. 73-82.
– E. Letier and A. van Lamsweerde. “Agent-based tactics for goal-oriented
requirements elaboration”, Proc. of the IEEE Int. Conf. on Soft. Eng.
(ICSE), 2002, pp. 83–93.
• Miscellaneous
– Boehm, B., Klappholz, D., Colbert, E., et al., “Guidelines for Lean ModelBased (System) Architecting and Software Engineering (LeanMBASE)”,
Center for Software Engineering, University of Southern California, 2005.
– http://sunset.usc.edu/csse/courseroot/course_list.html#577a
– http://sunset.usc.edu/csse/courseroot/course_list.html#577b
3/18/2008
©USC-CSSE
19