Radar, EW and Intelligence Systems Division
A Lightweight Automated Test Framework based on
UML 2.0 Testing Profile for Component Testing
Mustafa DURSUN
Workshop on Real-time, Embedded
and Enterprise-Scale Time-Critical Systems
May 24-26, 2010, Washington, DC, USA
aselsan is an affiliated company of Turkish Armed Forces Foundation.
Presentation Outline
Radar, EW and Intelligence Systems Division
• Motivation
• Component-Based Software Development
• Component-Model
and
Development Practice
ASELSAN’s
Component-Based
• UML Testing Profile
• Proposed Automated Test Framework for black-box testing of
Embedded Components
• Summary
aselsan is an affiliated company of Turkish Armed Forces Foundation.
Motivation
Radar, EW and Intelligence Systems Division
•
Component-Based software development decreases software development time for
many variants of products by using reusable components as building blocks.
•
Model-based development of reusable components promotes cost-efficient
development of component-based software systems.
•
Validating reusable components with black-box testing much earlier in the
development lifecycle increases reliability of reusable software components.
•
Model-driven testing makes testing easier by use of a common language with
application developers
•
UML Testing Profile enables the use of a common language for model-driven
testing throughout the development process.
•
Automated test framework based on UML Testing Profile can significantly improves
productivity by reducing manual model inspection.
aselsan is an affiliated company of Turkish Armed Forces Foundation.
Component-Based Software Development
Radar, EW and Intelligence Systems Division
• Component-Based Development is an approach to software
development in which all artifacts can be built by assembling,
adapting, and binding - wiring together existing components into
a variety of configurations.
• The main motivation of Component – Based Software
Development (CBSD) is decreasing software development time
for many variants of products
aselsan is an affiliated company of Turkish Armed Forces Foundation.
Component-Based Software Development
Radar, EW and Intelligence Systems Division
§ A component should be independent from the other part of a
system.
§ A component should be used as a black-box.
§ A component should provide and require interfaces.
- Provided interfaces of component determines the external
behavior and features of the component and allows the
component to be used as a black box.
- Required interfaces are necessary to specify contracts of
components
and
independent
development
of
components in object-oriented programming languages.
§ A component should be composed with other components
and replaceable in a composition.
aselsan is an affiliated company of Turkish Armed Forces Foundation.
Radar, EW and Intelligence Systems Division
Composition and Binding Time of
Embedded Components
• It is possible to mention three types of binding time:
– Compile-time
– Link-time
– Run-time
• In general, earlier binding times enable better static analysis and
system optimization, whereas later binding times enable
configuration by users and post deployment updates.
• Earlier binding is often appropriate for constrained systems such
as embedded systems.
aselsan is an affiliated company of Turkish Armed Forces Foundation.
Component Model for Embedded Systems
Radar, EW and Intelligence Systems Division
• Component model restrict composition to link time.
• Defines 3 type of components
– Reusable Component
– Connector
– System Component
aselsan is an affiliated company of Turkish Armed Forces Foundation.
Component Model
Reusable Components
Radar, EW and Intelligence Systems Division
• Reusable components are hierarchical
• Reusable components are not distributed
«ReusableComponent»
Detection
IDetection1
IDetection1
1
pMonitor[2]
1
DetectionFacade
RuleEngine
pMonitor[2]
IMonitor
IMonitor
pINS
pDiagnosis
IINS
IDetection2
IDiagnosis
pINS
1
IINS
pDiagnosis
IDetection2
IDiagnosis
aselsan is an affiliated company of Turkish Armed Forces Foundation.
Database
Component Model
Connector
Radar, EW and Intelligence Systems Division
• Connectors documented as reusable components
• Connectors implements proxy
between distributed components
pattern
for
communication
aselsan is an affiliated company of Turkish Armed Forces Foundation.
Component Model
System Component
Radar, EW and Intelligence Systems Division
• System components are distributed
«SystemComponent»
FaultDetection
1
«ReusableComponent»
InterfaceMonitoring
IDetection1
IDetection1
pDetection
pMonitor[2]
«Connector»
ResourceMonitoring
«ReusableComponent»
Detection
IMonitor
pINS
IMonitor
1
1
IINS
IDetection1
pDiagnosis
pDetection
IDetection2 IDiagnosis
<MessageChannel>
IDetection2 IDiagnosis
IMonitor
pDetection
1
1
«Connector»
Diagnosis
«Connector»
INS
pINS
IINS
<TCP/IP>
aselsan is an affiliated company of Turkish Armed Forces Foundation.
<CORBA>
Radar, EW and Intelligence Systems Division
ASELSAN’s Component-Based Development
Practice
Agile development of components:
1. Stories for an iteration are identified by customer.
2. Component developers is organized to develop software
components with pair programming.
3. The component software is continuously rebuilt and unit tested.
4. Customer makes functional black-box component testing.
5. At the end of each iteration, baseline is created for the
component.
aselsan is an affiliated company of Turkish Armed Forces Foundation.
UML Testing Profile
Radar, EW and Intelligence Systems Division
• The UML Testing Profile defines a language for designing,
visualizing, specifying, analyzing, constructing, and documenting
the artifacts of test systems.
• The UML Testing Profile is organized in four logical groups of
concepts:
– Test architecture defining concepts related to test structure
and test configuration.
– Test data defining concepts for test data used in test
procedures.
– Test behavior defining concepts related to the dynamic
aspects of test procedures.
– Test time defining concepts for a time quantified definition of
test procedures.
aselsan is an affiliated company of Turkish Armed Forces Foundation.
Test Architecture and Test Behavior
Radar, EW and Intelligence Systems Division
aselsan is an affiliated company of Turkish Armed Forces Foundation.
Test Architecture
Radar, EW and Intelligence Systems Division
• Test Context, Test Configuration
• System Under Test (SUT)
• Test Component
• Arbiter
• Scheduler
aselsan is an affiliated company of Turkish Armed Forces Foundation.
Test Data
Radar, EW and Intelligence Systems Division
• Data Pool
• Data Partition
• Data Selector
• Coding Rule
aselsan is an affiliated company of Turkish Armed Forces Foundation.
Test Behavior
Radar, EW and Intelligence Systems Division
• Test objective
• Test case
• Defaults
• Verdict
• Validation action
• Finish Action
aselsan is an affiliated company of Turkish Armed Forces Foundation.
Test Time
Radar, EW and Intelligence Systems Division
• Timers
• Time zones
aselsan is an affiliated company of Turkish Armed Forces Foundation.
Automated Test Framework (ATF)
Radar, EW and Intelligence Systems Division
• Automated Test Framework provides systematic test execution
control based on UML Testing Profile to run tests and validate
test results
• Automated Test Framework provides solution for Message
Sequence, and User-oriented component testing
• Test automation focus on a component model to support its
related testing activities for different projects.
aselsan is an affiliated company of Turkish Armed Forces Foundation.
ATF Test Architecture Elements
Radar, EW and Intelligence Systems Division
• SUT is the reusable component that is documented as a
structured class with well defined interfaces.
• Test Components are defined as objects. Each test component
has a port with complement interfaces of SUT.
• Test context is a system component whose parts are the SUT
and test components.
• Scheduler and arbiter are implemented by Test context.
aselsan is an affiliated company of Turkish Armed Forces Foundation.
ATF Test Data Elements
Radar, EW and Intelligence Systems Division
• Data Partitions are classes that are to store the test data defined
in test context for the each messages defined in the SUT
interfaces.
• Data Partitions keep
– Stimuli Data: Stimuli message arguments data values and
return data values of required interface messages.
– Expected Data values: return data value of Stimuli message
and arguments data values of required interface messages.
• Expected Data Partition defines:
– Expected messages
– Parameter values of the messages
– The number of calls of a message
– The order of the messages
aselsan is an affiliated company of Turkish Armed Forces Foundation.
ATF Test Data Elements
Radar, EW and Intelligence Systems Division
• Data Pool contains data partitions that are associated with a test
context and its test cases.
• Data Selector is a operation that is defined in Data Pool. Test
components reaches the test data using data selectors.
aselsan is an affiliated company of Turkish Armed Forces Foundation.
ATF Test Behavior
Test Context and Test Cases
Radar, EW and Intelligence Systems Division
• Test Cases define:
– Stimuli message
– Expected
message
Message
Sequence
reaction
to
the
stimuli
– Test Data Values; Stimuli, Expectation Data values
• Test Context creates data partitions for each test data values in
the data pool.
aselsan is an affiliated company of Turkish Armed Forces Foundation.
ATF Test Behavior
Radar, EW and Intelligence Systems Division
Test Component
• Test components responsible to stimulate SUT.
• Test components responsible for validation action by validating:
– the value returned from the SUT stimuli message
– SUT calls messages with expected parameter values
– SUT calls messages in expected order
– SUT calls messages in expected number
– SUT calls messages in expected time
• Test components set verdict of the arbiter after validation
actions.
• Test components responsible to perform finish action
aselsan is an affiliated company of Turkish Armed Forces Foundation.
ATF Test Behavior
Radar, EW and Intelligence Systems Division
•
Validation Action
A test component performs validation action
–
At the instant of a message call from SUT
•
•
•
–
by comparing observed return value with the expected return
value of the stimulus message call.
by comparing the observed order of message call with the
expected order of message call.
by comparing the observed message information that is
gained from the message call made by SUT in run-time, with
the expectation message information that is defined in the test
case.
At the end of time-out
•
by comparing observed number of calls of a message in the
expected time with the expected number of calls of the
message.
aselsan is an affiliated company of Turkish Armed Forces Foundation.
ATF Test Behavior
Radar, EW and Intelligence Systems Division
Validation Action
• Observability of a message: It is not always possible to know the
correct order/number of message call made by SUT. In these
cases the message should not be observable. By this way a test
component does not compare the order or number of message
calls.
aselsan is an affiliated company of Turkish Armed Forces Foundation.
ATF Test Behavior
Radar, EW and Intelligence Systems Division
Finish Action
• A test component performs finish action after
– Instantaneous validation action if validation action fails.
– Time-out validation action regardless of the validation action
value.
aselsan is an affiliated company of Turkish Armed Forces Foundation.
ATF Automated Elements
Radar, EW and Intelligence Systems Division
aselsan is an affiliated company of Turkish Armed Forces Foundation.
ATF Automated elements
Radar, EW and Intelligence Systems Division
Test Architecture
• For each port of SUT a test component is created.
• Test context implements IScheduler, and IArbiter.
• Connections of SUT, test components, IScheduler, and IArbiter are created
in test context
aselsan is an affiliated company of Turkish Armed Forces Foundation.
ATF Automated Elements
Radar, EW and Intelligence Systems Division
Test Data
• A data pool is created for each test context.
• A data partition is created for each message of an interface.
• A data selector is created as an operation of data pool for a data
partition.
• Test Context creates data partition objects for each test cases.
aselsan is an affiliated company of Turkish Armed Forces Foundation.
ATF Automated Elements
Test Case
Radar, EW and Intelligence Systems Division
:InterfaceMonitoring
:ResourceMonitoring
:INS
:Detection
:Diagnosis
setMonitoringData(pDataType=1, pDataValue=3)
getMonitoringData(pDataType = 1)
readINS(pINS)
setDiagnosisData(pDataType = 1, pDataValue = 12, pINSInformation = pINS)
aselsan is an affiliated company of Turkish Armed Forces Foundation.
ATF Automated Elements
Radar, EW and Intelligence Systems Division
Test Behavior
aselsan is an affiliated company of Turkish Armed Forces Foundation.
Summary
Radar, EW and Intelligence Systems Division
• ATF allows model-driven functional black-box testing compatible
with UML Testing Profile for reusable embedded components.
• ATF allows validating reusable components for application
developers by using common modeling language.
• ATF increases testability of reusable components by abstracting
testers from implementation details of ports and structure
classes.
• ATF improves productivity by reusing same framework for
different projects developed with the same component-model.
aselsan is an affiliated company of Turkish Armed Forces Foundation.
Radar, EW and Intelligence Systems Division
Thanks for your attention!
Sorular?
(Questions?)
aselsan is an affiliated company of Turkish Armed Forces Foundation.