CVM17 Keymile Test Plan
Document Owner:
Nguyen Duong
Part Number:
681-00169-02
Date of Issue:
06-18-2012
© 2011 GENBAND
The information contained herein is the property of GENBAND and is strictly confidential. Except as expressly authorized in
writing by GENBAND, the holder shall keep all information contained herein confidential, shall disclose the information only to
its employees with a need to know, and shall protect the information from disclosure and dissemination to third parties. Except
as expressly authorized in writing by GENBAND, the holder is granted no rights to use the information contained herein. If you
have received this document in error, please notify the sender and destroy it immediately.
NOTE TO HOLDERS OF COPIES: Check the GENBAND website to verify that this document is still current. Questions concerning
interpretation of this document or suggestions for improvement should be directed to the Template Owner.
CVM17 Keymile Test Plan
Contents
1.
2.
3.
4.
5.
DOCUMENT INFORMATION .................................................................................................................. 3
1.1
Storage Location ........................................................................................................................... 3
1.2
Key Contributors ........................................................................................................................... 3
1.3
Document Approval ...................................................................................................................... 4
1.4
Document Publication History ...................................................................................................... 4
PREFACE ................................................................................................................................................ 4
2.1
Document Goal ............................................................................................................................. 4
2.2
Intended Audience ........................................................................................................................ 4
2.3
Document Organization ................................................................................................................ 5
Test Activity Scope ................................................................................................................................ 5
3.1
Overview ....................................................................................................................................... 5
3.2
Test Activity Release Requirements.............................................................................................. 5
3.3
Test Activity Entrance and Exit Criteria......................................................................................... 5
3.4
Test Strategy ................................................................................................................................. 6
3.5
Assumptions .................................................................................................................................. 7
3.6
Dependencies................................................................................................................................ 7
3.7
Exclusions and limitations ............................................................................................................. 7
Resource Requirements ........................................................................................................................ 7
4.1
Test Primes and Accountability..................................................................................................... 7
4.2
Test Activity Timeline .................................................................................................................... 7
4.3
Test Lab and Diagrams .................................................................................................................. 8
4.4
Hardware Requirements ............................................................................................................... 9
4.5
Software Requirements ................................................................................................................ 9
4.6
Firmware Requirements ............................................................................................................... 9
4.7
Test Tool Requirements and Dependencies ............................................................................... 10
4.8
Results Tracking Location............................................................................................................ 10
Test Areas............................................................................................................................................ 10
2
CVM17 Keymile Test Plan
5.1
Call Processing & Services ........................................................................................................... 10
5.2
Traffic and Capacity..................................................................................................................... 11
5.3
Performance ............................................................................................................................... 11
5.4
Robustness .................................................................................................................................. 11
5.5
Clients.......................................................................................................................................... 12
5.6
Interoperability and Solution Integration ................................................................................... 12
5.7
Configuration and Provisioning................................................................................................... 12
5.8
Documentation ........................................................................................................................... 13
5.9
Compliance ................................................................................................................................. 13
5.10
Fault Management & Maintenance ............................................................................................ 13
5.11
Upgrades, Backup/Restore, & Backwards Compatibility ........................................................... 13
5.12
Security and Vulnerability ........................................................................................................... 13
5.13
User interface.............................................................................................................................. 13
5.14
Installation and Media Validation ............................................................................................... 13
6.
Test Cases............................................................................................................................................ 13
1. DOCUMENT INFORMATION
1.1
Storage Location
CVM17 End Office Regression Test Plan can be found at:
http://sharepoint.genband.com/sites/Dev/SRL/SV/CVM17_SV/Docs/Forms/AllItems.Aspx
1.2
Key Contributors
3
CVM17 Keymile Test Plan
Name
Role
Dept
Nguyen Duong
Keymile Prime
VV35
Thu Minh
Keymile Prime
VV35
My Hoang
Keymile Prime
VV35
Hai Nguyen
Keymile Prime
VV35
Huy Thai
Keymile Prime
VV35
1.3
Document Approval
Function
Name
Mandatory/Optional
Product/Solution Verification
Release Prime
Viet Nguyen
Genband Test Activity Prime
Viet Nguyen
M
Test Activity Prime
Viet Nguyen
M
Keymile PLM
Nguyen Duong
M
1.4
Document Publication History
Issue
0.01
M
Status
Draft
Change Summary
Initial draft version of the document preface.
Author(s)
Keymile team
Date
2012-07-31
2. PREFACE
2.1
Document Goal
The goal of this document will provide test coverage for Keymile in the CVM17 release. Test plan
document details high level verification plan for Keymile during CVM17 release to ensure full test
overage against approved requirements.
2.2
Intended Audience
The intended audience for this document includes all Systems, Design, Product Test, System Test, GNPS
and VO Team members that are involved in the test planning, execution, and/or customer deployment
4
CVM17 Keymile Test Plan
of this Product/Solution release. In addition, any of our customers may request visibility to the
Product/Solution Verification Test Plan.
2.3
Document Organization
This document is organized as follows:
Section 2: Preface. This section introduces the reader to the purpose, organization and flow of the
document.
Section 3: Test Activity Scope. This section describes the scope of the test activity, requirements,
exceptions and key assumptions.
Section 4: Resource Requirements. This section describes the resources, equipment and timeline
required to achieve the test coverage.
Section 5: Test Areas This section lists the various test areas that need to be considered when
planning test coverage for the product or solution.
Section 6: Test cases. This section contains the test cases that will be executed for each test area.
Section 7: References and Glossary This section identifies the reference documents used to
develop this system verification test plan.
3. TEST ACTIVITY SCOPE
3.1
Overview
This document will identify the detailed plan for Keymile activity in CVM17 that contains
H.248/MEGACO,MCST.
The plan includes the verification test deliverables, roles, responsibilities, quality targets, resources,
dependencies, areas that will be covered in this activity.H248 Keymile is being continued with the new
load released for CVM17.
3.2
Test Activity Release Requirements
3.3
Test Activity Entrance and Exit Criteria
TABLE 1 TEST ACTIVITY ENTRANCE CRITERIA
Type
General
Entrance Criteria
Required
N/A
Activity Test Plan reviewed and approved
5
CVM17 Keymile Test Plan
General
All Activity resources (personnel, captive lab equipment, test
tools) in place
General
Sanitized software load available
Product
Regression
All entry criteria for Design Activity/Feature Integration have been
met and Feature Integration has begun.
Product
Regression
All props from previous releases are in place.
Solution
Integration
Each product has met the s/w and feature integration targets
defined in the Solution Strategy Document
Product/Solution
Verification
All product integration/feature integration test activities have been
completed successfully: all test cases 100% executed with pass
rate of 95% or higher and no OPEN P1 CRs
Product/Solution
Verification
IT CRs evaluated and all System Verification impacting CRs
identified and resolved or mitigated.
Product/Solution
Verification
Drafts of all customer visible documents to be reviewed are
available
Product/Solution
Verification
All DP3 Milestone criteria have been met and the milestone has
been declared.
TABLE 2 TEST ACTIVITY EXIT CRITERIA
Type
Exit Criteria
General
TD (Test Done) milestone achieved
General
All replied and fixed CRs retested and closed
General
Test cases to be executed during Final Assessment identified at TFP (Test First Pass)
3.4
Test Strategy
The test strategy of the End Office is a part of the one of CVM17 Consumer and Business Voice SV
Strategy and can be found at:
http://sharepoint.genband.com/sites/Dev/SRL/SV/CVM17_SV/Docs/Forms/AllItems.Aspx
6
CVM17 Keymile Test Plan
3.5
Assumptions
3.6
Dependencies
3.7
Exclusions and limitations
-
Traffic testing is not covered in this test plan and should be done by Large System testing.
For line interworkings, the End Office Solution functionality will be tested for the agents exist in
the Ottawa and TMA labs.
Only Keymile will be regressed in UAIP, MG9K should be used for interworking purpose only.
4. RESOURCE REQUIREMENTS
4.1
Test Primes and Accountability
Title/Function
Name
Location
Contact
Keymile Architect
Viet Nguyen
Genband
nquocviet@tma.com.vn
TMA Verification Prime
Viet Nguyen
TMA
nquocviet@tma.com.vn
Verification Lab Prime
Viet Nguyen
TMA
nquocviet@tma.com.vn
Keymile Prime
Nguyen Duong
TMA
dhnguyen-batch3.5@tma.com.vn
4.2
Test Activity Timeline
Verification Milestone Start Week
SD (Strategy Done)
Complete
Week
Target Lab (for Execution)
18-July-2012 18-Aug-2012
7
CVM17 Keymile Test Plan
TP (Regression)
18-July-2012 18-Aug-2012
LR (Regression)
18-July-2012 18-Aug-2012
4.3
TMA2
Test Lab and Diagrams
The test will be performed on the 3 different platforms: XACore, Compact PCI and AdvancedTCA. All of the test
cases will be executed at TMA site (TMA2).
TMA2 is an International Hybrid Compact Lab. There is a large number of TCs (Keymile) need to be
tested in this lab as an Intl Solution. Several Cable TCs which were executed in CS2K5 before, are also
defined to run in this lab.
Network Manager
TMA 2: Int’l Solution Lab
USP
ISSG
SS7
Network
SAM21-1/2 GWC/CICM
CS2000
SAM21-3/4 GWC/CICM
Int’l Int’l
cCS2k
cCS2K
Hybrid
MS
IOC IOC
MG EMS
Storm
(L HP)
NOC
ENET
MTM PDTC
PLGCI
AMS 2010
(240)
IW-SPM
HA
SYSTEMS
CBM
(N240)
StreamWide
VM
MPEM
CICM EM
(7147)
DS1
TDM
Network
AS5200
Other Lab CS2Ks
LCME
Passport IGIGE, ATM)
15K
Camiant PS (Infra)
VSP2
VSP3
OC3
VSP3o
Loop
VSP4e
ISUP, PTS,
PRI Trks
RAD MUX
CS-LAN
Routers 0&1
(IP Network)
SIP Trunking
OM3500
TN-1X Express
MP
7200
HT Langley
MDM
(N240)
CICM
CICM
7147
7147
SST
SST
(HT
(HT
Langley
Langley
METRO
SBC
Note: Currently
Cc3310
MG3200
H.248
E1
STM1
MG3200
H.248
MG3500 Loop
H.248
(VLAN)
MG9k
ITP/ATM
ABI
POTS
DSL
SAA-12 Port
DS1 16 Port
MG3500
H.248
(Non VLAN)
SSL
SSL
(HT
(HT
Langley
Langley
PLGCI
LCME
DS1
ONT
E1
E1
SIP-PBX
TDM Network
H.323 Enterprise
Networks
PSTN
OCS R1/R2
KEYMILE
(2 UMUX 1500,
1 MileGate)
E1
E1
CMT/IEMS
(N240)
MGSITE
Loop
Loop
9KEM
(N240)
CICM
CICM
5385
5385
BCM50
BCM400
D-Link
ACME
CCM
Westell
CISCO 3600
CISCO 3700
CISCO 7200
CS1K
One200 Access
CICM/IAD Enterprise
Networks
I2001
I2002
I2004
I2007
11xx
2033
m6350
221x
61xx
Askey 30 (2)
Askey 4 (5)
Foxcon 1 (5)
Foxconn 32 (2)
CMTS
Cable HFC
Arriis 402/502
Arris 512
Moto 5120
BCM-50
M1/CS1K
Cisco CM
Session Server
Lines
1120
CISO 7960
1140
Telab ONT
LG68XX LG88XX
LG1534
Moto 5121
Moto 5220
Component HW not yet identified
8
CVM17 Keymile Test Plan
4.4
Hardware Requirements
Beside the general devices for C20 (Core, GWC, SST, SSL …), we need the below items for our EO testing:

This activity will be executed in TMA and Ottawa labs required for CVM17

Keymile Milegate2500 shelf equipped with IPSM2, SUPM3 and SUIQ1 cards
Keymile Umux1500 shelf equipped with IPSMG, ISBUQ and SUBH3 cards.

4.5
Software Requirements
TABLE 3
SOFTWARE REQUIREMENT TABLE
NE
S/W Level
SST
SST_16
GWC
A2E SSL
Media Server
2010
(MS2010)
(IP)
Keymile
Umux1500
Keymile
Milegate2500
4.6
Release Status
NE
S/W Level
15.0.0
GC160
MCP 8.0
20.62
SAM21
Platform
CS2M
SESM
Core
IPSMG_R5AH2
Arris TM602
6.1.101
ipsm2_r7a23
Arris TM512
6.1.101
Release
Status
CS2M16
SESM16
NA:CSNNC13
Intl:CSNWC13
Firmware Requirements
TABLE 4
FIRMWARE REQUIREMENTS TABLE
Component
F/W Version
Release
Status
Component
F/W Version
Release
Status
9
CVM17 Keymile Test Plan
4.7
Test Tool Requirements and Dependencies
- Simtool – SCP simulator for AIN testing
- Jira, PR will be used to keep track the bugs
- Mercury Quality Center
Mercury Quality Center Test case attributes / results database will be used to track
passed and failed CR’s and will hold this test plan and test case descriptions
- GWC Trace Tools
GWC Trace tools will be used for each test case to determine the protocols used for
voice and fax transmissions
- GWC PMDEBUG
- MAPCI
- LOGUTIL
- Unix server for using soft phone
- CATT server for automation testcases execution
- Serdec server for monitoring LI messages
4.8
Results Tracking Location
All execution results will be stored in Quality Center under Domain: Carrier Project: CVM17_SV
5. TEST AREAS
5.1
Call Processing & Services
This area testing will be executed in 3 different platforms: XACore, Compact PCI and AdvancedTCA. This part also
covers all scenarios which are related to major issued areas from previous releases: Hold/retrieve; call transfer, LI,
Conference…
1. Keymile
10
CVM17 Keymile Test Plan
This release, Keymile GW has a new improved load. The improvements compared with the old
load include:
- Fax handling
- Items related to ISDN-BA
- Improved stability.
So we will collect the test cases based on those areas as well as some raised CRs in the previous
release.
Besides that, this testing area is used to make sure all general functions and features introduced
for UAIP work properly in CVM17. Line GWs inter-opt and services inter-opt testing will be
covered under Inter-opt & Inter-work area. However, this coverage will be reduced due to low
find rate.
5.2
Traffic and Capacity
5.3
Performance
5.4
Robustness
1. Keymile
Robustness & Recovery testing will be executed on the gateway to ensure that the components
perform as expected under failure scenarios.
The testing will include focus within the following areas:
– Network impairments/isolations (e.g. Ethernet connectivity loss)
– GWC Warm/Cold SWACT.
– GWC dead shelf recovery
– GW power cycle
– Active Call failovers
– Core restarts and swact
11
CVM17 Keymile Test Plan
5.5
Clients
5.6
Interoperability and Solution Integration
TABLE 5 INTEROPERABILITY TESTING SCOPE
NCS
SSL
SIPPBX H.323 MGCP
CICM MG9K Keymile
NCS
x
SSL
x
x
SIPPBX
x
x
x
H.323
x
x
x
x
MGCP
x
x
x
x
x
CICM
x
x
x
x
x
x
MG9K
x
x
x
x
x
x
x
Keymile
x
x
x
x
x
x
x
x
SST
x
x
x
x
x
x
x
x
PVG trunk
x
x
x
x
x
x
x
x
G2 trunk
x
x
x
x
x
x
x
x
G6 trunk
x
x
x
x
x
x
x
x
MG3200
x
x
x
x
x
x
x
x
G9
x
x
x
x
x
x
x
x
VMail
x
x
x
x
x
x
x
x
SST
PVG
G6
G2 trunk
MG3200
trunk
trunk
G9
VMail
> CallP and services will be tested based on this interoperability matrix to increase the test coverage in
CVM17
5.7
Configuration and Provisioning
1. Keymile
12
CVM17 Keymile Test Plan
- Defining end points.
- Successful Creation of endpoints for a fully equipped Keymile GW.
- Configure Keymile GW as H248 line GW
- Configure Keymile GW (UMUX) as V5.2 line GW (AN)
5.8
Documentation
All customer documents that change will be verified that will be identified by DP3 milestone
5.9
Compliance
5.10
Fault Management & Maintenance
5.11
Upgrades, Backup/Restore, & Backwards Compatibility
5.12
Security and Vulnerability
5.13
User interface
5.14
Installation and Media Validation
6. TEST C ASES
13
CVM17 Keymile Test Plan
QC Domain: Carrier
QC Project: CVM17_SV
QC Test Plan Folder/Path: TBD
QC Test Lab Folder/Path: TBD
User Interface
TC Title A sample test case
TC ID
QC_TCID
Type:
{Automated | Manual}
Priority:{High | Medium | Low}
Requirement: {optional, reference to the requirement covered by this test case}
Test Case Description
1. Test Objective
This section defines the test objective of the test case.
2. Test Configuration
This section describes the hardware required to execute this test case, and a reference to the Network
Configuration, which may be a figure in this document.
3. Test Procedure
This section provides a list of step-by-step instructions to follow to execute the test case.
14
CVM17 Keymile Test Plan
4. Expected Results
This section provides a list of outcomes which must occur in order for the test case to meet its
objectives. If these outcomes are not observed, then the test case is considered failed.
5. Notes
This section describes a list of additional information that may be relevant to the execution of the test
case.
Cable Offsite Manual
No
Title
1 CFU - Verify that service is working properly.
2 ACRJ - Verify that service is working properly.
3 CNDB - Verify that service is working properly.
4 CFB - Verify that service is working properly.
5 CNAMD - Verify that service is working properly.
6 CNAB - Verify that service is working properly.
7 CPU - Verify that service is working properly.
8 NAME - Verify that service is working properly.
9 CND - Verify that service is working properly.
10 CWT/CWI – Verify that service is working properly.
15
CVM17 Keymile Test Plan
16