A Method for Compatible COTS components Selection
Jesal Bhuta and Barry Boehm
Centre for Software Engineering, University of Southern California
Los Angeles, California 90089, USA
{jesal, boehm}@cse.usc.edu
Assignment 3
Name: Sven Elmar Fortuin
Group number: 2
Student number: 3496465
Utrecht University
s.e.fortuin@students.uu.nl
February 22, 2013
Introduction
Bhuta and Boehm (2005) introduce a method which can be used to select compatible Commercial OfThe-Shelf (COTS) components in order to (successfully) create a COTS Based Application (CBA).
The authors define a CBA as “a system for which 30% of the end-user functionality is provided by
COTS components, and at least 10% of the development effort is devoted to COTS considerations”.
Although multiple methods already exist for selecting different COTS components (Alves and Castro,
2001; Kontio et al., 1995; Mohamed, Ruhe and Eberlein, 2007), the described method specifically
focusses on the compatibility between the components. The authors experienced that projects which
took the compatibility into account at an early stage were more successful during integration phases
than those who didn’t. Therefor they found the need to create a model that specifically focusses on
selecting compatible component rather than selecting the best individual components. The model is
part of the USC CBA development process framework (Boehm, Port, Yang, Bhuta and Abts, 2003)
and contains the following activities:
0: Entry conditions.
The method begins with establishing objectives, constraints and priorities (OC&Ps) of the system.
1: Identify candidate legacy and third-party components to be integrated.
During this activity a series of third-party and legacy components needs to be identified that together,
integrated in a certain combination, meet the OC&P’s.
2: Classify the components into function groups
In this step the identified components from activity one are classified into function groups. A function
group is a group of components that share the same functionality.
3: Evaluate alternatives with respect to functional and non-functional OC&Ps.
In this activity the COTS candidates are evaluated against functional and non-functional capabilities in
order to decide whether they are suitable for the CBA.
4: Buy information to reduce risk
This step supports step five. In this step information is acquired about the different COTS candidates.
This acquirement is realised by market analysis, assessment of vendor supportability, evaluation of
prototypes and buying study reports.
1
5: Filter out unacceptable alternatives
In this phase the component alternatives that fail to meet the functional and non-functional capabilities
are removed and will no longer be considered for the CBA.
6: Evaluate available component combinations
During this activity the developers must consider different component combinations that will meet the
OC&Ps. This evaluation is realised through the use of the component compatibility evaluation
framework.
7: Prototype to reduce risk
This step supports step six. In this step prototypes are built in order to gain a better understanding of
the integration of components, which results in a more reliable evaluation.
8: Preserve options to maintain trade-space
During this step different component combinations are stored. This can be done in a database or as text
in a document.
9: Exit to decision-making and integration
In this step a choice is made for the definite combination of COTS components. This choice is based
on the evaluation of combinations against functional and non-functional capabilities, as well as on
compatibility.
10: Re-negotiate with project stakeholders and adjust the OC&Ps
This step is only done when a function group contains zero components after filtering components.
During this activity the OC&Ps are re-negotiated with the stakeholders because one or more function
groups don’t contain components.
11: Develop a custom component for the specific functionality
This step is only done when the OC&P’s cannot be adjusted after re-negotiating. In this step a custom
component is created for the function group(s) that don’t contain components.
2
Example
In the example below a watch vendor wants to broaden his market and has therefor decided to start an
online shop next to his regular shop. The vendor has decided he wants to use COTS components in
order to save time and money buying a custom made system. To create this CBA he has hired a
developer who has some experience with COTS components.
0: Entry conditions
The vendor has a clear image of what the shop should be able to do. After thoroughly looking at other
online shops he has come to the following OC&P’s:
 Web-based inventory management
 Online shopping cart with a secure credit card payment supporting all acknowledged banks in
the Netherlands (Rabobank, ABN amro, ING and SNS) and iDeal.
 Automated order processing
 Automatic generation of invoices
 Sending order and shipping confirmation
 Automatic inventory update
 Access administration capabilities
1: Identify candidate legacy and third-party components to be integrated.
Based on past experience and by conducting research on the internet the developer has identified
multiple components that seem to meet the OC&P’s. This research was done by searching for
information on the functionalities of the components trough vendor websites and specialised fora. The
following components were identified:
 Cart 32
 MSSQL
 Danise Cart
 Microsoft IIS
 Apache-CGI
 X-Hub Enterprise Cart
 MySQL
 Apache-Tomcat
2: Classify the components into function groups
Due to the experience of the developer the classification based on functions is easily done. The results
can be seen in table 1.
Function group:
COTS component:
COTS component:
COTS component:
Table 1.
Database Applications
MySQL
MS-Access
MSSQL
Shopping Carts
Cart 32
Danise Cart
X-Hub Enterprise Cart
Application Servers
Microsoft IIS
Apache-CGI
Apache Tomcat
Classified COTS components per function group
3: Evaluate alternatives with respect to functional and non-functional OC&Ps.
The developer has no experience with the components ‘Danise cart’, ‘X-Hub Enterprise Cart’, Apache
Tomcat’ and ‘Microsoft IIS’. Due to this lack of experience in these products more information is
required in order to properly evaluate them. (Step four is conducted)
With the acquired information from step four it can be concluded that ‘X-Hub Enterprise Cart’ and
‘Apache Tomcat’ do not meet the OC&Ps because they do not support secure credit card payments.
4: Buy information to reduce risk
The developer has called the vendor supportability for all the products on which more information was
required. These phone calls gave him enough information for the components ‘X-Hub Enterprise Cart’
and ‘Apache Tomcat’. However, the vendor supportability of ‘Danise cart’ and ‘Microsoft IIS’ did not
3
provide the necessary information. For these components online available study reports were acquired
in the form of performed case studies that involved the concerning component. In these case studies
the developer found the information he was looking for (step 3 is further conducted).
5: Filter out unacceptable alternatives
Based on the evaluation it was decided that X-Hub Enterprise Cart and Apache Tomcat didn’t meet
the OC&Ps. For this reason they are filtered out and will no longer be considered for the CBA. The
remaining components can be seen in Table 2.
Function group:
COTS component:
COTS component:
COTS component:
Table 2.
Database Applications
MySQL
MS-Access
MSSQL
Shopping Carts
Cart 32
Danise Cart
-
Application Servers
Microsoft IIS
Apache-CGI
-
Classified COTS components per function group after filtering
As can be seen, all the function groups still contain components and therefor step ten and eleven can
be left out.
6: Evaluate available component combinations.
To make evaluation successful the developer maps the components in a component combination
evaluation framework (see Figure 1). The component graphically shows which components can easily
be combined (so without much glueware) due to underlying architectures. (The mapping takes place
based on the information gathered in steps 3 and 4). To gain more insight in the underlying
architectures and interoperability prototypes will be created (step seven is conducted).
7: Prototype to reduce risk
The developer created prototypes for all three combinations. Developing these prototypes has given
the developer the insight that the combination MSSQL – Cart 32 – Microsoft IIS will need the least
amount of glueware.
8: Preserve options to maintain trade-space
The other combinations (MySQL – Denise Cart – Apache-CGI and MS-Access – Cart 32 – Microsoft
IIS) will be preserved.
Database Applications
Application Servers
MySQL
Microsoft IIS
MS-Access
Apache-CGI
MSSQL
Cart 32
Denise Cart
Shopping Carts
Figure 1.
Component Combination Evaluation Framework for an online watch vendor CBA
4
9: Exit to decision-making and integration
All the combinations meet the functional and non-functional requirements. However, the combination
MSSQL – Cart 32 – Microsoft IIS will require the least amount of glueware and is therefor favoured.
The developer and vendor decide that the components MSSQL – Cart 32 – Microsoft IIS forms the
best combination. The developer can now integrate the chosen components.
(Steps 10 and 11 are left out because each function group contained one or more components after step
5)
Process-Deliverable Diagram (PDD)
The PDD (as seen in Figure 2) was created according to the guidelines given in the paper ‘MetaModeling for Situational Analysis and Design Methods’ by Weerd and Brinkkemper (2009).
Descriptions of the activities can be seen in the activity table (Table 3). Finally, the concept table can
be seen in Table 4.
On the left side of the PDD (Figure 2) processes of the method are shown. On the right side the
deliverables of the processes are presented. The PDD contains four important activities namely: create
OC&P’s, component analysis, component combination analysis and select combination. Evaluate
OC&P’s and develop component are left out because they are only performed in a specific context.
Preserve combinations is left out because this is a simple activity and not relevant for the selection of
the final combination.

Create OC&P’s
The creation of OC&P’s corresponds to the entry conditions given in the introduction. The
OC&P’s are needed to evaluate the different components and component combinations. The
process cannot be properly started without established OC&P’s.

Component analysis
During the component analysis all the components are identified and eventually filtered. The
main purpose of this activity (and its sub activities) is to reduce the number of components
that are being considered for the eventual CBA.

Component combination analysis
Next to the identification of possible combinations, a thorough understanding of these
combinations will also be the result of the component combination analysis. The main purpose
of this activity is to identify a combination of components that will require the least amount of
work to integrate.

Select combination
This is the final activity of the model, and will result in the final decision of the component
combination that integrated will lead to the CBA.
The activities explained above all result in multiple deliverables (concepts). The most important
concepts are: OC&P’S, EVALUATION CRITERIA, COMPONENT COMPATIBILITY
EVALUATION FRAMEWORK and COMPONENT COMBINATION.

OC&P’s
The OC&P’s are the basis of this model. The OC&P’s describe the requirements of the CBA
and without them evaluation and filtering of COMPONENTS would be impossible.
5

EVALUATION CRITERIA
The EVALUATION CRITERIA are based on the OC&P’s and are very important because
each component is individually evaluated against these criteria. If a COMPONENT fails to
meet the EVALUATION CRITERIA it will no longer be considered for the CBA. The
EVALUATION CRITERIA are based on the OC&P’s. (As can be seen in the PDD)

COMPONENT COMPATIBILITY EVALUATION FRAMEWORK
Different COMPONENT COMBINATIONS are identified trough the use of the
COMPONENT COMPATIBILITY EVALUATION FRAMEWORK. This concept is the core
of the method since it combines COMPONENTS and FUNCTION GROUPS and ultimately
provides possible CBA’s. A template of this framework can be found in Appendix A, an
example can be seen in Figure 1.

COMPONENT COMBINATION
The final deliverables of the method are different COMPONENT COMBINATIONS that
integrated meet the OC&P’s. It is then left to the stakeholder and developer which
combination is ultimately chosen.
All information on the model as well as most concept definitions were extracted or derived from the
original papers (Bhuta and Boehm, 2005; Boehm et al., 2003). A few definitions were derived from
other papers, these papers are listed in the reference list.
6
Create OC&P’s
Establish objectives
OBJECTIVE
Establish constraints
CONSTRAINT
Establish priorities
PRIORITY
Development team,
Stakeholder
Component
analysis
1..*
1..*
1..*
1..*
NON-FUNCTIONAL
CAPABILITY
1..*
FUNCTIONAL
CAPABILITY
1..*
1
is established by
1..*
is established by
1
OC&P
1
1
Identify candidate
components
Classify
components
1
FUNCTION GROUP
2..*
1
Evaluate candidate
components
0..*
EVALUATION CRITERIA
1
1..*
is evaluated by
[else]
[more
information
needed]
Buy information
COMPONENT
Name
Description
Function
2..*
CUSTOM COMPONENT
is met by
Filter components
Development team
is input for
[else]
[function group empty]
Evaluate OC&P’s
Development team, Stakeholder
[OC&P’s adjustable]
[else]
Develop component
Development team
Component
combination
analysis
Evaluate component
combinations
1
COMPONENT COMPATIBILITY
EVALUATION FRAMEWORK
1
[else]
produces
[more certainty
needed]
1..*
Create prototype
Development team
0..1
PROTOTYPE
Rank
Rank
1..*
Preserve combinations
Development team
Select combination
Development team, Stakeholder
Figure 2.
1..*
is evaluated by
1..*
COMPONENT
COMBINATION
PDD of the compatible COTS election method
7
Activity
Create
OC&P
Sub activity
Establish objectives
Establish constraints
Establish priorities
Component
analysis
Identify candidate
components
Classify components
Evaluate candidate
components
Buy information
Filter components
Evaluate
OC&P’s
Develop
component
Component
combination
analysis
Evaluate component
combinations
Create prototype
Preserve
combinations
Select
combination
Table 3.
Description
The development team and stakeholder(s) together establish
the OBJECTIVES of the CBA.
The development team and stakeholder(s) together establish
the CONSTRAINTS of the CBA.
The development team and stakeholder(s) together establish
the PRIORITIES of the CBA.
The development team identifies candidate COMPONENTS
for the CBA.
The development team classifies each COMPONENT into a
FUNCTION GROUP based on their function.
The development team evaluates the candidate components
against EVALUATION CRITERIA.
If there isn’t enough information on a specific
COMPONENT to perform a successful evaluation, the
development team buys information (in terms of money or
schedule) for that specific COMPONENT.
The development team removes each COMPONENT that
fails to meet the OC&P’S.
If one or more FUNCTION GROUPS contains zero
COMPONENTS, the development team tries to renegotiate
the OC&P’S with the stakeholder(s).
If renegotiation with the stakeholder(s) is not possible, the
development team will develop a CUSTOM COMPONENT
for the empty FUNCTION GROUP(S).
The development team considers the possible combinations
in which COMPONENTS can be integrated to meet the
OC&P’S through a COMPONENT COMPATIBILITY
EVALUATION FRAMEWORK.
If there’s uncertainty regarding the COMPONENT
COMBINATIONS, the development team creates
PROTOTYPES to gain more insight in the different
COMPONENT COMBINATIONS.
The development team preserves the different
COMPONENT COMBINATIONS by storing these. (e.g. in
a database or document)
Together with the stakeholder(s), the development team
selects a final COMPONENT COMBINATION.
Activity table
8
Concept
OBJECTIVE
CONSTRAINT
PRIORITY
OC&P
COMPONENT
FUNCTION
GROUP
EVALUATION
CRITERIA
FUNCTIONAL
CAPABILITY
NONFUNCTIONAL
CAPABILITY
CUSTOM
COMPONENT
COMPONENT
COMPATIBILITY
EVALUATION
FRAMEWORK
PROTOTYPE
COMPONENT
COMBINATION
Table 4.
Description
An OBJECTIVE is a goal the system should be able to reach.
A CONSTRAINT is a boundary that restricts, limits and regulates the
system.
PRIORITY is the relative importance of an object
List that contains OBJECTIVES, CONSTRAINTS and PRIORITIES for a
specific system (Bhuta and Boehm, 2005).
A COMPONENT is a piece of software that can be integrated with other
parts in order to create a full software system.
A FUNCTION GROUP classifies components based on the function they
perform. Components in the same group must satisfy a similar set of system
capabilities (Bhuta and Boehm, 2005).
The EVALUATION CRITERIA is a list of FUNCTIONAL CAPABILITIES
and NON-FUNCTIONAL CAPABILITIES the component must possess in
order to be considered for the system (Bhuta and Boehm, 2005)
FUNCTIONAL CAPABILITIES are the required activities the system
should be able to perform.
A NON-FUNCTIONAL CAPABILITY is a capability that is not specifically
concerned with the functionality of a system. It places restrictions on the
product being developed and the development process, and it specifies
external constraints that the product must meet. (Chung and Prado Leite,
2009)
A CUSTOM COMPONENT is a COMPONENT that is developed for a
specific system.
A COMPONENT COMPATIBILITY EVALUATION FRAMEWORK is a
technique that visually shows which components can be integrated in order to
meet the OC&P’S (Bhuta and Boehm, 2005).
A PROTOTYPE is an early sample that is built to test the system.
Combination of COMPONENTS that, when integrated, meet the OC&P’s
(Bhuta and Boehm, 2005).
Concept table
Criticism
The model is mainly useful for smaller projects, and lacks the capabilities to be used in bigger
software projects. This is all due to the component combination evaluation framework. The maximum
number of function groups that can be integrated into this framework is three. When four or more
function groups are used the framework loses its functionality because the geometrical shape that will
be needed to combine the components is not optimal. Next to the limit on the number of function
groups, there’s also a limitation on the number of components that can be used. Many components in
the framework will result in many lines. Too many lines can result in dysfunction because it will make
the framework cluttered.
9
Related literature
First it should be noticed that the presented method is part of USC CBA development process
framework described by Boehm et al. (2003). This framework can be used in the development of
CBA’s and consists of more parts than just selecting compatible COTS components.
The need for methods that focus on component compatibility derived from incompatibility.
Incompatibility problems between different components have several reasons (Abd-Allah, 1996;
Garlan, Allen and Ockerbloom, 1995). Most of those reasons include: architectural mismatch,
interface conflicts, functional mismatch and non-functional mismatch (Gacek, 1998; Gacek and
Boehm, 1998; Yakimovich, Travassos and Basili, 1999) . However, Fox, Lantner and Marcom state
that not systems gaps, but system overlapping result in computability problems. Another issue when
integrating different components is the amount of glueware needed. Researches Basili and Boehm
(2001) argue that one line of glue(ware) code requires approximately three times the effort per line of
developed application code.
There is a rich history of methods and frameworks that focus on the development and selection of
COTS software, an overview of those models is given by Mohamed et al. (2007) and Land, Blankers,
Chaudron and Crnković (2008).
One of the more popular methods in this area is the COTS based requirement engineering (CRE)
method. The CRE method selects COTS components based on four criteria namely: domain coverage,
time restriction, costs rating and vendor guaranties (Alves and Castro, 2001) rather than compatibility.
However, the discussed method is not the only method that focusses on compatibility. Mariani,
Papagiannakis and Pezzè (2007) argue that some components may be compatible on a syntactic level,
but lack computability on a system level. The solution they provide for this problem also focusses on
compatibility between different components.
Solutions for compatibility issues are very important as is illustrated in a project by Garlan et al.
(1995). During this project the incompatibility between four COTS components caused the project to
quadruple in time estimation and made the costs five times as much. To prevent such multifold in time
and cost more insight is needed in the creation of CBA’s and the compatibility between different
COTS components.
Although the presented method isn’t widely used, development methods that take COTS components
into consideration are still being used and researched (Megas, Beli, Frakes, Urbano and Anguswamy
2013; Hart, Hager, Barinaga and Duckworth, 2013).
10
References
Abd-Allah, A. A. E. S. (1996). Composing heterogeneous software architectures. California, Los
Angeles: University of Southern California.
Alves, C., & Castro, J. (2001). CRE: A systematic method for COTS components selection.
Proceedings of the XV Brazilian Symposium on Software Engineering (SBES). Rio de Janeiro, Brazil,
193-207.
Basili, V. R., & Boehm, B. (2001). COTS-based systems top 10 list. Computer, 34(5), 91-95.
Bhuta, J., & Boehm, B. (2005). A method for compatible cots component selection. COTS-Based
Software Systems, 3412, 132-143. doi:10.1007/978-3-540-30587-3_23
Boehm, B., Port, D., Yang, Y., & Bhuta, J. (2003). Composable process elements for developing
COTS-based applications. Proceedings of the International Symposium on Empirical Software
Engineering, Los Angeles, California, USA, 8-17.
Chung, L., & do Prado Leite, J. (2009). On non-functional requirements in software engineering. In
A.T. Borgida, V.K. Chaudhri, P. Giorgini & E.S. Yu (Eds.), Conceptual modeling: Foundations and
applications, (pp. 363-379). Berlin: Springer. doi:10.1007/978-3-642-02463-4_19
Fox, G., Lantner, K., & Marcom, S. (1997). A software development process for COTS-based
information system infrastructure. Proceedings of the Fifth International Symposium on Assessment of
Software Tools and Technologies, Redondo Beach, California, 133-142.
doi:10.1109/AST.1997.599923
Gacek, C. (1998). Detecting architectural mismatches during systems composition. California, Los
Angeles: University of Southern California.
Garlan, D., Allen, R., & Ockerbloom, J. (1995). Architectural mismatch: Why reuse is so hard. IEEE
Software, 12(6), 17-26. doi:10.1109/MS.2009.86
Hart, G. L., Hager, G. J., Barinaga, C. J., & Duckworth, D. C. (2013). Market Research Survey of
Commercial Off-The-Shelf (COTS) Portable MS Systems for IAEA Safeguards Applications (No.
PNNL-22237). Pacific Northwest National Laboratory (PNNL), Richland, WA (US).
Kontio, J., Chen, S. F., Limperos, K., Tesoriero, R., Caldiera, G., & Deutsch, M. (1995). A COTS
selection method and experiences of its use. Proceedings of the 20th Annual Software Engineering
Workshop, Greenbelt, Maryland, USA, 1-16.
Land, R., Blankers, L., Chaudron, M., & Crnković, I. (2008). Cots selection best practices in literature
and in industry. High Confidence Software Reuse in Large Systems, 5030, 100-111. doi:10.1007/9783-540-68073-4_9
Mariani, L., Papagiannakis, S., & Pezze, M. (2007). Compatibility and regression testing of COTScomponent-based software. Proceedingsof the 29th International Conference on Software Engineering,
Minneapolis, Minnesota, USA, 85-95. doi:10.1109/ICSE.2007.26
Megas, K., Belli, G., Frakes, W. B., Urbano, J., & Anguswamy, R. (2013). A Study of COTS Integration
Projects: Product Characteristics, Organization, and Life Cycle Models.
11
Mohamed, A., Ruhe, G., & Eberlein, A. (2007). COTS Selection: Past, Present, and Future.
Proceedings of the 14th Annual IEEE International Conference and Workshops on the Engineering of
Computer-Based Systems, Washington, DC, USA, 103-114. Doi:10.1109/ECBS.2007.28
Weerd, I., & Brinkkemper, S. (2009). Meta-modeling for situational analysis and design methods. In
M. Syed, & S. Syed (Eds.), Handbook of Research on Modern Systems Analysis and Design
Technologies and Applications (pp. 38-58). Hershey: Idea Group Publishing. doi:10.4018/978-159904-887-1.ch003
Yakimovich, D., Travassos, G. H., & Basili, V. (1999). A classification of software components
incompatibilities for COTS integration. Findings of the 24th Software Engineering Workshop.
Greenbelt, Maryland, USA, 1-9.
12
Appendix – A
Function group 2
Function group 1
A1
B1
A2
B2
A3
B3
A(..)
B(..)
C1
C2
C3
C(..)
Function group 3
Figure 3.
Template for the component combination evaluation framework.
Function group:
Component 1:
Component 2:
Component 3:
Component (..):
Table 5.
1
A1
A2
A3
A(..)
2
B1
B2
B3
B(..)
3
C1
C2
C3
C(..)
Template for function groups and corresponding components
Explanation: Identified components are classified into function groups according to their
functionality. The identified components and function groups should be filled in in the table above
(Table 5). Table 5 shows that function group one contains three components (A1, A2 and A3), the
A(..) indicates that more components within function group one are possible. This means that a
function group could contain any number of components.
After classification the components should be mapped into the component combinations evaluation
framework according to their function group. Function group 1 in the table corresponds to function
group 1 in the component combination evaluation framework. This correspondence also applies to the
different components (A1, A2, B1, C1 etc.).
When the mapping of the components and function groups into the component combination evaluation
framework is complete, lines should be drawn between components that are easily integrated. An
example of this is shown in Figure 1. When all the lines are drawn possible combinations will be
indicated by a perfect triangle (as can be seen in Figure 1).
13