Opentest Architecture
•
Table of Content
– The Design
• Basic Components
• High-Level Test Architecture
• Test Flow
– Services provided by each Layer
• Test Mgt System
• Test Master Controller
• Service-Providers layer
– Interfaces between layers
• Test Mgt System <-> Test Master Controller
• Test Master Controller <-> Service-Providers layer
– User Interaction w/ the System
1
Test Architecture’s Basic Components
3-Layer Architecture:
1) Test Management System. 2) Test Master Controller. 3) Service Providers
Test Management System
1
Front-End
Test Projects
n
Custom
Fields*
n
Test Cases
TestLink (GPL) + Staf
* Custom fields= Test Execution Engine,
Test Execution Logic, Required HW assets,
Required SW assets and Test Parameters.
Test Reports
Test Requirements
STAF Request + XML
Test Master Controller
1
Staf (EPL v1.0)
Dispatcher
Monitor
TMS Writer
Resource Manager
STAF Request
STAF Request + XML
Build Execution Engine
Test Execution Engine
Execution Logic
Execution Logic
…
Equipment Drivers
Staf +
Vatf|Stax|iCOTS|LinuxTEE|Other
…
2
High-Level Test Architecture
Site A
Site B
Test Management System
Test Management System
Test Artifacts
(Mysql)
Test Artifacts
(Mysql)
Test Master Controller
IP
Network
Test Master Controller
Test
Execution
Engine
Build
Execution
Engine
Service
Provider
Test
Execution
Engine
Build
Execution
Engine
Service
Provider
Test
Execution
Engine
Test
Execution
Engine
Service
Provider
Test
Execution
Engine
Test
Execution
Engine
Service
Provider
• Test Artifacts Databases must be synchronized on a periodic basis
• The 3rd layer is the service-providers layer. TEE and BEE are just special Service Providers.
• It should be possible to reuse Service Providers across sites if/when desired
• Typically 1 TMS : 1 TMC
1 TMC : Many Service Providers
3
Sample TMS-Driven Test Flow
TMS
Front-End
1: Creates Test Plan (optionally)
2: Triggers Test Execution.
Specifies: Test plan, Platform,
SW assets versions
15: Generate Test Reports
3: Export Test
Plan to XML
Test Plan
(XML)
14: Save test
results in TMS
4: Send Request to TMC
(Pass Test Plan XML file)
TMC
12:Send Request
to save Results
TMS Writer
Dispatcher
13: Send E-mail (optionally)
5,8: Request BEE or TEE
6:Send build job
request to BEE
11: Notify completion
Provide Results
7:Notify Build
completion
9: Send test
job request to TEE
BEE Iface
Res Mgt
10: Optionally request job
to a service provider via
Resource Manager Service
Service Provider
BEE
TEE Iface
TEE
Service Iface
4
Test Management System’s
Services
•
TestLink Basic Definitions:
– Test Project holds together the multiple test
artifacts (test cases, test requirements, test plans,
etc.)
– Test Case describes a testing task and expected
results. Test Cases are the fundamental piece of
TestLink.
– Test Suite is a collection of Test Cases. It
organizes Test Cases to units.
– Test Build is the software to be tested. It is part
of the Test Plan.
– Test Plan is created when you'd like to execute
Test Cases. Test Plans can be made up of the
Test Cases from the current Test Project.
•
Main Services provided by the TMS:
5
Test Master Controller’s
Services
• Dispatch Jobs to appropriate BEE & TEE
• Translate Test parameters between TMS
and BEE/TEE
• Provide TMS-independent test case
representation to TEE
• Provide a mechanism by which TEE can
save test results in TMS
• Manage Service Providers
6
Service Providers’ Services
•
At least there will be two types of service providers:
– BEE: Build Execution Engines to build the
required software assets.
– TEE: Test Execution Engines to run the test
cases. The TEE maybe as simple as a shell
script or as complex as VATF.
•
•
BEE & TEE are the only service providers that the TMC Dispatcher directly
request services from.
Complex TEE, such as VATF, will typically perform these tasks:
– Initialize HW assets (both Device Under Test
and Test Equipment)
– Create Test Parameters representation
– Run Test Execution Logic
• Load Device Under Test (DUT) w/ appropriate SW assets
• Boot DUT
7
TMS <-> Test Controller IFACE
TMS GUI
Trigger Exec
Front-End
TestLink
Test Plan
(Mysql)
Export
Staf Request
Trigger Exec
Staf Request
Trigger Exec
Dispatcher
Read
Test
Cases
Import
Test Plan
(XML)
Write Back
Test Results
Test Master Controller
• XML format provides a TMS-independent Test Data Representation
• Staf provides a generic communication mechanism, both the TestLink
server and the Test Master Controller must have Staf installed
8
Test Controller <-> Service
Providers IFACE
Test Master Controller
Resource
Mgt
E-Mail
Dispatcher
Monitor
Staf Request
Process, FS or
other services
Staf Request
Notify, FS or
other services
StafProc
BEE/TEE Service Providers
Service-Provider
-specific
Actions
• Similar to TMS-TMC iface, Staf provides a generic communication mechanism.
• The Test Controller Dispatcher is a Staf-Enabled Java application
• Service Providers, including BEE & TEE, must have Staf on it (StafProc process)
• The Dispatcher reads BEE/TEE name from Test Plan and dispatch the appropriate job request to the appropriate engine.
For example, it reads SW assets info in the Test Plan and interact w/ a BEE to build them. Similarly, it reads the TEE type
from the test plan and submit a test request to the appropriate TEE. The TEE will adapt or transform the test plan data
according to its needs.
• The Dispatcher may dispatch a Test job that contains multiple test cases to streamline the test execution.
• The Dispatcher adapts TEE responses before writing back to Test Plan database
9
User Interaction w/ the system
Creates/Edits/Deletes
Test Artifacts
Test Management System
Generate Test Reports
Monitor Test Execution
Send E-mail
Triggers Test Execution
Test Master Controller
Send E-mail
Triggers Test Execution
Monitor Test Execution
Test Execution Engine
Triggers Test Execution
10