MBSE for System Testing
Managing the development of system testing using the
principles of Model Based System Engineering
Ron Kratzke, Vitech Corporation
rkratzke@vitechcorp.com
Context Diagram - Verification Process
(Ref: INCOSE Handbook Ver 3.2.2 Oct 2011)
Controls
- Applicable Laws and Regulations
- Industry Standards
- Agreements
- Project Procedures and Standards
-Project Directives
Inputs
- System Requirements
- Verification Criteria
- Specification Tree
- Updated RVTM
- Interface Requirements
- Integrated System
- Interface Control Documents
- Integration Report
Outputs
Activities
- Plan Verification
- Perform Verification
- Verification Strategy
- Verification Enabling System
Requirements
- Verification Constraints on Design
- Verification Procedures
-Final RVTM
- Verified System
- Verification Report
Enablers
- Organization / Enterprise Policies,
Practices, and Procedures
- Organization / Enterprise Infrastructure
- Project Infrastructure
- Verification Enabling Systems
Vitech Corporation
2
Model-Based Systems Engineering
• Model-based systems engineering (MBSE) is
the formalized application of modeling to
support system requirements, design,
analysis, verification and validation activities
beginning in the conceptual design phase and
continuing throughout development and later
life cycle phases.
–
From “INCOSE Systems Engineering Vision” 2020 INCOSE-TP-2004-004-02 September, 2007
Vitech Corporation
3
MODELS
Vitech Corporation
4
Everyone has a model …
VITECH CORPORATION
5
System Testing has to account for a
number of items…
Components
(Physical Architecture)
Functions
Testing
(Behavior Architecture)
Requirements
Vitech Corporation
6
MBSE Verification Process Model
Must …
• Provides insight into the problem and
potential solutions
• Provide order and traceability
• Possess integrity and consistency
• Have ability demonstrate and explain the test
program
Vitech Corporation
7
Is The VCRM Enough???
Number
R.1.1.1
R.2.1.1.1.1
R.2.1.1.1.4
R.2.1.1.1.5
R.2.2.2
R.2.3.1
R.2.3.1
R.2.3.1
FN.1.1
FN.1.1
FN.2.1
FN.2.2
FN.2.3
FN.2.4
Name
Availability
Media of Requests: Hardcopy
Forms
Media of Requests: Telephonic
Electronic File
Media of Requests: Web Services
Provide Products
Description
The system shall be unavailable no more than a total of ten minutes
per month.
The system shall accept requests via hardcopy forms.
method
Analysis
The system shall accept requests via telephonic electronic file.
Test
The system shall accept requests via via a Web service.
The system shall provides previously collected images/products, if
appropriate.
Control Multiple Collectors
The system shall control multiple image collectors.
Control Multiple Collectors
The system shall control multiple image collectors.
Control Multiple Collectors
The system shall control multiple image collectors.
Accept And Format Request
The system shall accept the requests for information, verify that the
requester is a valid customer of the system, and format the request
into a form and media that the system can use.
Accept And Format Request The system shall accept the requests for information and format the
Without Certification
request into a form and media that the system can use.
Check Product Inventory
The system shall check the imagery product inventory to determine if
the customer's request for imagery products is available from current
imagery product inventory.
Prioritize Request
The system shall organize customer orders using a priority based
approach. Priorities shall be assigned based upon delivery date,
customer need, current collection assignments, and system resources.
Determine Collector Mix
The system shall evaluate the customer's order to establish which set
of collectors shall best provide imagery products answering the
customer's need.
Notify User Of Estimated Schedule The system shall prepare and send to the customer a notification
estimating the delivery date of the customer's order.
Vitech Corporation
Demonstration
Test
Demonstration
Test
Test
Demonstration
Test
Test
Test
Test
Test
Test
8
Additional Information Needed
• In addition to the method, shouldn’t we also
know?
– Objective
– Environment
– Special Conditions
– Success Criteria
Ref: A Day in the Life of a Verification Requirement Tutorial, Northrop Grumman, Steve Scukanec, 27 th Annual T&E Conference, March 14, 2011
Vitech Corporation
9
Status of Completing the Verification
• Wouldn’t we also want to track?
– Method (which is in the VCRM)
– Planning Status
– Completion Status
– Where / When the Verification Event will take
place
Vitech Corporation
10
Verification Requirement Statement
Verification Requirement Description statement provides the
details on HOW the verification will be accomplished
Vitech Corporation
11
Relating Requirements to Verification
Function,
Requirement
verified by
(verifies)
Verification
Requirement
Verification Requirement
attributes:
- Description
- Method
- Status
Add two more
attributes
- Level
Vitech Corporation
12
Table Output from Model …
Number
R.1.1.1
R.2.1.1.1.1
VR No.
V.R.1.1
V.R.2.1.1.1
R.2.2.2
Name
Availability
Media of Requests: Hardcopy
Forms
Media of Requests: Telephonic
Electronic File
Media of Requests: Web
Services
Provide Products
R.2.3.1
Control Multiple Collectors
V.R.2.3.1.a
R.2.3.1
Control Multiple Collectors
R.2.3.1
Control Multiple Collectors
FN.1.1
Accept And Format Request
FN.1.1
R.2.1.1.1.4
R.2.1.1.1.5
V.R.2.1.1.4
V.R.2.1.1.5
V.R.2.2.2
VR Name
Continuous Support
Media of Requests: Hardcopy
Forms
Media of Requests: Telephone
Electronic File
Media of Requests: Web-based
Electronic File
Provide Products
method
Analysis
Demonstration
Status
Planned
Planned
Level of Test
System
Subsystem
Test
In Progress
Subsystem
Test
Not Yet
Planned
Not Yet
Planned
Planned
Subsystem
Demonstration
Control Multiple Collectors Laboratory
V.R.2.3.1.b Control Multiple Collectors Ground Testing
V.R.2.3.1.c Control Multiple Collectors Flight Testing
V.R.2.5
Provide Feedback
Test
V.R.2.5
Provide Feedback
Test
FN.2.1
Accept And Format Request Without Certification
Check Product Inventory
V.R.2.5
Provide Feedback
Test
FN.2.2
Prioritize Request
V.R.2.5
Provide Feedback
Test
FN.2.3
Determine Collector Mix
V.R.2.5
Provide Feedback
Test
FN.2.4
Notify User Of Estimated
Schedule
V.R.2.5
Provide Feedback
Test
Test
Demonstration
Test
System
Subsystem
Completed Subsystem
Unsatisfactory
In Progress
Subsystem
Completed Satisfactory
Completed Satisfactory
Completed Satisfactory
Completed Satisfactory
Completed Satisfactory
Completed Satisfactory
System
System
System
System
System
System
Provides the data from the model to provide overall status
of verification planning… But what about the test events?
Vitech Corporation
13
What about the actual Testing?
Function,
Requirement
verified by
(verifies)
Verification
Requirement
What test events are planned?
Which Verification Requirements will be verified in which test event?
Who is doing the testing? What Organization is responsible?
What configuration is going to be used in testing?
Which Test Procedures will be used during any particular Test Event?
Vitech Corporation
14
What about the actual Testing?
Function,
Requirement
verified by
(verifies)
Verification
Requirement
fulfilled by
(fulfills)
Verification
Event
Add “Verification Event” to specify and describe a Test Event
which will be used to verify a subset of Verification
Requirements (for example, Limited User Test, Laboratory
Bench Test, Operational Test, etc.)
Vitech Corporation
15
What about the actual Testing?
Function,
Requirement
Organization
verified by
(verifies)
Verification
Requirement
assigned to
(responsible for)
fulfilled by
(fulfills)
Relate the Verification Event to an
Organization, Test Configuration,
and Test Procedure
Verification
Event
employs
(employed by)
Test
Configuration
Vitech Corporation
employs
(employed by)
Test
Procedure
16
Provide a complete picture…
Vitech Corporation
17
Verification Event Information
Verification Event
Limited System Functional Test
Test Organization
Acme Test Labs
Logistic Test Event
Logistics Analysts, Inc
Media Usage Testing
DataBase Testers R Us
Multiple Collector Testing
Collector Image Management, LLC
Test Config/Procedure
TestConfiguration
TestProcedure
TestProcedure
TestConfiguration
TestConfiguration
TestProcedure
TestConfiguration
TestConfiguration
TestProcedure
TestConfiguration
TestConfiguration
TestProcedure
Vitech Corporation
Name
Test Database
Data Base Access
Data Entry Procedure
Forward Operation Configuration
Permanent Cmd Ctr Configuration
30 Day System Operation Model
Forward Operation Configuration
Test Database
Media Usage Test Procedure
Global Hawk UAV Test Interface
Micro UAV Test Interface
Two Collector Tasking Procedure
18
What about time and resources for
the Verification Requirement?
Function,
Requirement
Organization
verified by
(verifies)
Verification
Requirement
executed by
(executes)
assigned to
(responsible for)
fulfilled by
(fulfills)
Verification
Event
employs
(employed by)
Test Activity
Test
Configuration
Vitech Corporation
employs
(employed by)
Test
Procedure
19
Test Activity – Detailed Test Planning
Vitech Corporation
20
Produce a Test Plan from the Model
Vitech Corporation
21
Summary …
• System Testing model
– Extends traditional VCRM to provide added insight
– Plan Verification Events including support resources
– Manage Testing Activity sequence and timing
– Produce Test Plans
•
•
•
•
Traceability
Completeness
Consistency
Document Generation
Vitech Corporation
22
MBSE for System Testing
Questions ???
Ron Kratzke, Vitech Corporation
rkratzke@vitechcorp.com