Seminar on CBSE (component-based
software engineering):
An industrial survey in Norway, Germany and Italy
Simula Research Lab., Oslo, 4 Feb. 2005
http://www.idi.ntnu.no/grupper/su/cbse-survey/
Reidar Conradi
Dept. Computer and Information Science (IDI)
NTNU, NO-7491 Trondheim
conradi@idi.ntnu.no, Tel +47 73.593444, Fax +47 73.594466
CBSE seminar, Simula Research Lab., 4 Feb. 2005
1
Agenda for CBSE-seminar
1130-1200 LUNCH and REFRESHMENTS, buffet syle
1200-1215 Reidar Conradi, NTNU: general intro, why is CBSE important?
1215-1230 Christian Bunse, Fraunhofer IESE: CBSE: a short overview.
1230-1245 Maurizio Morisio, Politecnico di Torino: COTS and OSS: what is it?
1245-1300 Preliminary discussion
1300-1400 Jingyue Li et al., NTNU: Main results from survey
Section 1: COTS vs. OSS
Section 2: Risk management in OTS based development
Section 3: Process improvement for OTS based development
1400-1430 Preliminary discussion
1430-1435 Reidar Conradi, NTNU: lottery award ceremony for one respondent
1435-1500 COFFEE BREAK
1500-1530 Additional and more qualitative comments from three companies:
- Per Spilling, Objectware
- Eivind Tagseth, Abeo (ex-ConsultIT)
- Rune Rysstad, WM-data
1530-1545 Odd Petter Slyngstad, NTNU: Method issues in the survey
1545-1615 Final discussion
CBSE seminar, Simula Research Lab., 4 Feb. 2005
2
Int’l CBSE survey, 2003-2005
• This is the promised seminar for the survey respondents!
• What's in it for you:
–
–
–
–
learn how to practise CBSE more efficiently
identify the major risks and challenges in using CBSE
learn from other industrial projects
exchange CBSE experiences with industrial colleagues
• Target audience:
– software professionals and companies that have practiced or want to
practice CBSE
– researchers that want to learn about issues and results
– journalists that want to convey main results back to the ICT community
CBSE seminar, Simula Research Lab., 4 Feb. 2005
3
Motivation for CBSE
Software reuse:
• Any software artifact: rqmts, UML designs, Java code, …
• Development ”FOR reuse”: making software parts for
later reuse. Investing, need domain insight.
• Development ”WITH reuse” (CBSE): composing a
software system with pre-made parts. Harvesting.
• Component = executable software part, open/closed source:
– internal
– external: COTS (commercial-Off-The-Shelf)
– external: OSS (open source software)
100,000s of external ”OTS”-components, available on web.
CBSE seminar, Simula Research Lab., 4 Feb. 2005
4
Component-based system:
Application + components + glue/addware
CBSE seminar, Simula Research Lab., 4 Feb. 2005
5
Reuse/CBSE – key technology for:
• Shorter time-to-market (months)
• Increased productivity / lower cost (pays off in
3rd project, in 1st with CBSE)
• Better quality (well-tested components)
• Promoting standards (domain, architecture,
platform)
• ...
• But may have to adjust the software process
CBSE seminar, Simula Research Lab., 4 Feb. 2005
6
Many CBSE challenges:
• OTS components used by 30% of Norwegian software
projects, 75% are successful. Need guidance.
• How to manage risks at different levels – project,
requirements, architecture, components, ...?
• Process changes: overall planning, increments,
requirements management, ”acquire vs. develop”,
customer contact, component decisions, …
• How to evaluate, integrate and test components?
• What is the impact on future maintenance?
• How to manage external providers?
• …
CBSE seminar, Simula Research Lab., 4 Feb. 2005
7
Need empirical base for what works or not
• Case studies / post-mortems:
– Many reported, also in books [Karlsson95].
– At Ericsson-Grimstad: reused components had less
defects, more stable than non-reused ones (Ex.1).
– Top 10 COTS issues (Ex.2) [Basili01].
• Controlled experiments:
– Hard to apply here, due to lack of realism.
• Surveys w/ questionnaires or interviews:
– A few studies (Ex.3), including our own.
– OSS survey in Italy [Bonaccorsi03].
CBSE seminar, Simula Research Lab., 4 Feb. 2005
8
Ex.1 GPRS/UMTS system at Ericsson-Grimstad
Application A
Application B
Application-specific
components
Business Specific
Middleware
(& Component Framework)
System Platform
Reused components
in PhD work
[Mohagheghi04]
Reused, but
considered as
COTS and OSS here
CBSE seminar, Simula Research Lab., 4 Feb. 2005
9
Ex.2 10 hypotheses on COTS issues [Basili01].
H1: More than 99% of all executing instructions come from COTS
components.
H2: More than half the features in large COTS components go
unused.
H3: Average COTS component has new release every 8-9 months.
H4: Dev. and post-deployment costs go by (#COTS comp.)**2.
H5: Post-deployment costs exceed development costs.
H6: Glue-code less than half dev. effort, costing 3x more per LOC.
H7: Non-development costs, e.g. licensing fees, are significant.
H8: Component assessment and tailoring efforts vary greatly by
COTS product classes.
H9: Personnel capability and experience are dominant cost factors.
H10: COTS-based development is currently high-risk, although many
have used it successfully.
CBSE seminar, Simula Research Lab., 4 Feb. 2005
10
Ex.3 COTS usage contradicts “common wisdom”
Structured interviews of 7 Norwegian and Italian SMEs:
Thesis T1: Open-source software is often used as closed source.
Thesis T2: Integration problems result primarily from lack of
compliance with standards; not architectural mismatches.
Thesis T3: Custom code is mainly devoted to add functionalities.
Thesis T4: Formal selection seldom used; rather familiarity with
product or generic architecture.
Thesis T5: Architecture more important than requirements to
select components.
Thesis T6: Tendency to increase level of control over vendor
whenever possible.
See [Torchiano04].
CBSE seminar, Simula Research Lab., 4 Feb. 2005
11
Joint CBSE survey in 3 countries
• By NTNU/Simula Research Lab,
Fraunhofer IESE, and Politecnico di Torino.
• Pre-study of 16 projects in Norway in 2003.
• Questionnaire in native language + English,
using SESE web tool at Simula.
• Asking 800-900 companies in 3 countries,
aim for 150 reponses (40% of 30%?).
• Have ca. 80 responses, 50 from Norway.
• Main study: May 2004 - March 2005.
CBSE seminar, Simula Research Lab., 4 Feb. 2005
12
Main results
• 30% of the IT companies had applied CBSE.
• 3/4 of projects in these had success with CBSE.
• Most projects searched the Internet for components,
then tried out a few, before making a decision.
• An incremental process is advisable to allow
(re)negotiation of requirements.
• Component quality is not always a major concern.
• Local expert on the actual components is a big asset.
CBSE seminar, Simula Research Lab., 4 Feb. 2005
13
Other results
• OSS components (open source) might be
preferred, as they allow local changes.
• COTS providers gave better technial support than
OSS ones.
• COTS components generally followed market
trends.
• Underestimation of integration and maintenance
effort was a problem.
CBSE seminar, Simula Research Lab., 4 Feb. 2005
14
Conclusion
• First representative, int’l CBSE survey
• Many interesting results, some surprising (webbased evaluation process)
• Later supplement by qualitative interviews and
real case studies, also look at WebServices
• Re-do survey in UK and USA?
• Surveys (as any other studies) are hard work!
CBSE seminar, Simula Research Lab., 4 Feb. 2005
15
Literature list (1)
•
•
•
•
•
•
•
•
•
[Basili01] Victor R. Basili and Barry Boehm, “COTS-Based Systems Top 10 List”, IEEE
Computer, 34(5):91-93, May/June 2001.
[Boehm99] Barry W. Boehm and Chris Abts, “COTS integration: Plug and Pray?”, IEEE
Computer, 32(1):135-138, Jan. 1999.
[Boehm03b] Donald J. Reifer, Victor R. Basili, Barry W. Boehm, and Betsy Clark, “Eight
Lessons Learned in COTS-based Systems Maintenance”, IEEE Software, 20(5):94-96, Sept./Oct.
2003.
[Bonaccorsi03] Andrea Bonaccorsi and Cristina Rossi, "Why Open Source software may
succeed”, Research Policy, 32(7):1243–1258, 2003.
[COCOTS00] Chris Abts, Barry W. Boehm, and Elizabeth Bailey Clark, “COCOTS: A COTS
Software Integration Lifecycle Cost Model- Model Overview and Preliminary Data Collection
Findings”. Technical report: USC-CSE-2000-501,
http://sunset.usc.edu/publications/TECHRPTS/2000/usccse2000-501/usccse2000-501.pdf.
[Comella-Dorda02] Santiago Comella-Dorda, John C. Dean, Edwin Morris, and Patricia
Oberndorf, “A Process for COTS Software Product Evaluation”, First Int’l Conference on COTSBased Software Systems (ICCBSS’02), Orlando, FL, USA, February 4-6, 2002, Springer Verlag
LNCS 2255, ISBN 3-540-43100-4, pp. 176-187.
[Heineman00] George T. Heineman, William T. Councill, Janet S. Flynt, Alok Mehta, John R.
Speed, and Mary Shaw, “Component-based software engineering and the issue of trust”, 22nd Int’l
Conference on Software Engineering (ICSE’2000), Limerick, Ireland, IEEE-CS Press, 4-11 June
2000, pp. 661-664.
[Frakes95] William B. Frakes and Christopher J. Fox, “Sixteen Questions About Software
Reuse”, CACM, 38(6):75-87, June 1995.
[Karlsson95] Even-André Karlsson (Ed.), "Software Reuse: A Holistic Approach" (The REBOOT
Methodology Handbook), Wiley Series in Software Based Systems. John Wiley. 510 p., 1995. ISBN
0-471-95819-0.
CBSE seminar, Simula Research Lab., 4 Feb. 2005
16
Literature list (2)
•
•
•
•
•
•
•
•
[Li04] Jingyue Li, Finn Olav Bjørnson, Reidar Conradi, and Vigdis By Kampenes, "An Empirical
Study of Variations in COTS-based Software Development Processes in Norwegian IT Industry", 10th
IEEE International Metrics Symposium (Metrics'04), Sept. 14-16, 2004, Chicago, p. 72-83. Recommended
for journal publication in Empirical Software Engineering. This contains results from the pre-study in
Norway in 2003.
[Li05] Jingyue Li, Reidar Conradi, Odd Petter N. Slyngstad, Christian Bunse, Marco Torchiano,
and Maurizio Morisio, "Preliminary Results of a State-of-the-Practice Survey on Motivations of Using
Off-The-Shelf Components", Submitted to 6th International Conference on Product Focused Software
Process Improvement (PROFES'2005), 13-16 June, 2005, Oulu, Finland, 17 p. This serves as the technical
report of the main study in 2004, the OTS survey.
[Mohagheghi04] Parastoo Mohagheghi, Reidar Conradi, Ole M. Killi, and Henrik Schwarz, "An
Empirical Study of Software Reuse vs. Defect Density and Stability", 26th Int’l Conference on Software
Engineering (ICSE'2004), 23-28 May 2004, Edinburgh, Scotland, IEEE-CS Press, pp. 282-292 (given one
of the five Distinguished Paper Awards at ICSE’04).
[Morisio00] Maurizio Morisio, Carolyn B. Seaman, Amy T. Parra, Victor R. Basili, Steve E. Kraft,
and Steven E. Condon, “Investigating and Improving a COTS-Based Software Development Process”,
22nd Int’l Conference on Software Engineering (ICSE’2000), Limerick, Ireland, IEEE-CS Press, 4-11
June 2000, pp. 31-40.
[Morisio02] Maurizio Morisio, Michel Ezran, and Colin Tully, “Success and Failure Factors in
Software Reuse”, IEEE Trans. on Software Engineering, 28(4):340-357, April 2002.
[Ropponen00] Janne Ropponen and Kalle Lyytinen, “Components of Software Development Risk: How
to Address”, IEEE Trans. on Software Engineering, 26(2):98-111, Feb. 2000.
[Szyperski03] Clemens A. Szyperski, “Component Technology - What, Where, and How?”, 25th Int’l
Conference on Software Engineering (ICSE’2003), Portland, Oregon, USA, IEEE-CS Press, 3-10 May
2003, pp. 684-693.
[Torchiano04] Marco Torchiano and Maurizio Morisio, "Overlooked Facts on COTS-based
Development", IEEE Software, 21(2):88-93, March/April, 2004.
CBSE seminar, Simula Research Lab., 4 Feb. 2005
17