An Empirical Study on
MBASE and LeanMBASE
Supannika Koolmanojwong
Center for Systems and Software Engineering
CSSE- Annual Research Review
February 13, 2007
2/13/07
(c) USC-CSSE
1
Outline
•
•
•
•
Introduction
Research Questions
MBASE & its problem
LeanMBASE
– Content Comparison
– Performance Comparison
• Conclusion
• Future Work
2/13/07
(c) USC-CSSE
2
Introduction
• Light weighted Software Engineering Projects
– CSCI577ab: Software Engineering Course
Sequence for graduate students in Computer
Science
– 12 weeks in Fall semester, 12 weeks in Spring
semester
– 6 on-campus students, 2 off-campus students
– From 1998-2005, use MBASE as methodology to
develop real-client projects.
• Problems
– MBASE is too heavy for software engineering
classes.
2/13/07
(c) USC-CSSE
3
Research Question
• With small-sized, limited schedule, real
client project, how can we provide the
best opportunity for students to learn
good software engineering approaches
and apply them to the real software
project development?
2/13/07
(c) USC-CSSE
4
MBASE
Model-Based (Systems) Architecting and
Software Engineering
•
Approach
–
–
–
–
–
–
•
MBASE/ RUP Activity / Process Model
WinWin Spiral model
Risk-driven strategy
Iterative refinement
Stakeholder commitment at anchor-point milestones
Emphasis of system and life-cycle issues.
The MBASE Guidelines provide content, format and templates for project
artifacts
– Project Artifacts
•
•
•
Operational Concept, System Requirements, System Architecture, Lifecycle Plan, Feasibility Studies
Plans and reports for constructions and Transition phase
Software tools
–
–
–
–
–
The Easy WinWin negotiation tool
Effort reporting tool
Risk identification tool
USC COCOMO II
COCOTS
2/13/07
(c) USC-CSSE
5
Problems in MBASE
• Problems with MBASE
– MBASE is designed for large industry projects
– Hugh Amount of effort in documentation
– Less time to focus on project implementation
• Strategies
– Analyze data from our project repositories
– Shape up MBASE  leanMBASE
2/13/07
(c) USC-CSSE
6
Solutions in LeanMBASE
• Solutions in LeanMBASE
– Minimum-essential documentation
– Tailor up if proper
– More time in project implementation
• Fall 2005,
– the 260 pages of MBASE Inception and
Elaboration phase Guidelines were
replaced with 90 pages of LeanMBASE
Guidelines.
2/13/07
(c) USC-CSSE
7
LeanMBASE
• Same approach, same documentation set,
same tool but light-weight
• Identify high-value activities, balance the
workload of a development
• Off-campus V&V  IIV&V (Integrated
Independent) involve more to the team
• In Fall 2006, add one new document (SID:
Supporting Information Document) as
package header; combined all similar
information.
2/13/07
(c) USC-CSSE
8
Outline
•
•
•
•
Introduction
Research Questions
MBASE & its problem
LeanMBASE
– Content Comparison
– Performance Comparison
• Effort Comparison
• Client Evaluation Comparison
• Conclusion
• Future Work
2/13/07
(c) USC-CSSE
9
Content Comparison I
Inception & Elaboration Document Size Comparison:
LeanMBASE: smaller size; except SSAD in fall 2005
Fall 03 - Spring 04 : MBASE
Comparison of each I&E document
Fall 04 - Spring 05 : MBASE
Fall 05 - Spring 06 : LeanMBASE
Number of pages
250.00
Inception
Elaboration
Fall 06 - Spring 07 : LeanMBASE
Construction
Transition
200.00
150.00
100.00
50.00
O
LC
LC
O
O
C
D
SS
LC RD
O
SS
A
D
LC
O
LC
LC P
O
FR
D
LC
O
SI
D
LC
AO
C
D
LC
AS
S
LC RD
AS
SA
D
LC
AL
CP
LC
AF
R
D
LC
AS
ID
IO
C
O
C
D
IO
C
SS
R
D
IO
C
SS
A
D
IO
C
LC
P
IO
C
FR
D
AB
O
C
AB D
SS
R
AB D
SS
A
D
AB
LC
P
AB
FR
D
0.00
Inception and Elaboration documents
Operational Concept Definition (OCD); System and Software Requirements Definition (SSRD); System and Software
Architecture Description (SSAD); Life Cycle Plan (LCP); Feasibility Rationale Document (FRD)
2/13/07
(c) USC-CSSE
10
Content Comparison II
Construction and Transition
Document Size Comparison;
Spring 2004 : MBASE
Comparison of size of CTS documents
Spring 2005 : MBASE
Spring 2006 : LeanMBASE
45
Average Document Size (Page)
40
Not much different in CTS
documents;
Room for improvement
35
30
25
20
15
10
5
0
PRP
PRR
IP
IAR
QMP
TP
TDR
RD
TrP
SP
PTP
RTP
UM
CTS Docum ents
2/13/07
(c) USC-CSSE
Plans and Reports in Construction and Transition Phase
•Peer Review Plan (PRP)
•Peer Review Report (PRP)
•Iteration Plan (IP)
•Iteration Assessment Report (IAR)
•Quality Management Report (QMP)
•Test Plan (TP)
•Test Description and Result (TDR)
•Release Description (RD)
•Transition Plan (TRP)
•Support Plan (SP)
•Packaged Tools and Procedures (PTP)
•Regression Test Package (RTP)
•User Manual (UM)
11
Effort Comparison
Comparison of Effort in generating I&E document set
Fall 03 : MBASE
Fall2003 : MBASE
Number of hour / page in generating I&E document set
Fall 04 : MBASE
Fall2005 : LeanMBASE
1.4
Fall 05 : LeanMBASE
160
Fall2004 : MBASE
Fall2006 : LeanMBASE
1.2
140
120
number of hours / page
Average Effort per Team (hours)
Fall 06 : LeanMBASE
100
80
60
40
20
1
0.8
0.6
0.4
0.2
0
OCD
SSRD
SSAD
LCP
FRD
0
OCD
Inception and Elaboration documents
Average number of hours
spent for documentation:
Less Effort, except SSAD in
Fall 2005
2/13/07
SSRD
SSAD
LCP
FRD
Inception and Elaboration documents
Average number of hour/page
in documentation:
Less number of hours per
page; except SSRD in Fall
2006
(c) USC-CSSE
12
Client Evaluation
Uniformly high; relatively close
Average Score of Customer
evaluation
Semester
2/13/07
Fall
Spring
Total
(20 pts)
(20 pts)
(40 pts)
Fall 03 – Sp 04 : MBASE
18.4
18.22
36.7
Fall 04 – Sp 05 : MBASE
17.9
18.29
36.2
Fall 05 – Sp 06 : LeanMBASE
17.5
19.5
37.0
Fall 06 – Sp 07 : LeanMBASE
17.9
n/a
n/a
Average
17.9
18.67056
36.6
(c) USC-CSSE
13
Conclusion
• With LeanMBASE
–
–
–
–
Smaller document size
Less time in documenting
Comparably satisfied with the project result from Clients
Off-campus students or IIV&V personnel play more roles
• At least for small real-client student-team projects,
the LeanMBASE more efficiently supports students in
– developing software projects,
– learning software development
– providing a win-win result to all key stakeholders.
2/13/07
(c) USC-CSSE
14
Future Work
• Additional Comparison
– Risk Comparison
– Defect List Comparison
• Process Enhancement
– Electronic Process Guideline (LeanMBASE
Plug-Ins on EPF OpenUp Framework)
– WikiWinWin
– Mentor Program
2/13/07
(c) USC-CSSE
15