PROPRIETARY INFORMATION
INTERNAL USE ONLY
THIS DOCUMENT AND THE DATA DISCLOSED HEREIN OR HEREWITH IS PROPRIETARY AND IS NOT TO BE REPRODUCED, USED OR DISCLOSED IN
WHOLE OR IN PART TO ANYONE WITHOUT THE WRITTEN PERMISSION OF ORACLE. COPYRIGHT, © ORACLE 2022. ALL RIGHTS RESERVED.
Title:
R12.1 Reference Architecture B - Performance Test Plan (draft)
Doc Number:
TP10213
Revision #: 1.0
Policy Management
Test Plan
Reference Architecture B –
Performance Test Plan for R12.1
Release 12.1
P. Piacentini
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 1 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
1
CHANGE HISTORY
4
2
REFERENCES
5
3
PURPOSE AND SCOPE
5
4
TERMINOLOGY
6
5
TOPOLOGIES
7
5.1
Reference B-1 Topology
5.1.1
Reference B-1 Performance metrics
7
7
5.2
Reference B-2 Topology
5.2.1
Reference B-2 Performance Metrics
8
8
5.3
Reference B-3 Topology
5.3.1
Wireless Outbound Roaming Section of Topology B-3
5.3.2
Wireless Inbound Roaming Section of Topology B-3
5.3.3
Reference B-3 Performance Metrics
9
9
10
10
5.4
Reference C-1 Topology
5.4.1
Reference C-1 Performance Metrics
10
11
6
SERVER AND SYSTEM CAPACITIES
12
7
CALL FLOWS
13
7.1
Call1 - LTE + Subscriber Quota(2 PDN Connections)
14
7.2
Call2 - Subscriber Quota(2 PDN Connections) Allot Variation
15
7.3
Call3 - LTE No Quota 2 PDN connection +Sy
16
7.4
Call4 - GPRS(2 PDN Connections) + Pooled Quota + Regular Subscriber Quota
17
7.5
Call5 - GPRS(2 PDN Connections) with secondary PDP + Secondary Gx Subscriber Quota
18
7.6
Call6 - IMS VoLTE Call (VoLTE calls will contain both IMSI and E164 Sub ID)
19
7.7
Call7 - Application Monitoring on Sd
20
7.8
Call8 - LTE + Quota on Sd
21
7.9
Variation1 - OCS Triggered Throttling
21
7.10
Variation2 - Sh Triggered Throttling
22
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 2 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
7.11
VoLTE plus S9 OutBound Roaming
22
7.12
VoLTE plus S9 InBound Roaming
22
8
GENERAL CONFIGURATION SETUP
22
9
EXAMPLE POLICIES
23
10
GENERAL GOALS OF THE TESTS
23
10.1
Latency Goals
24
10.2
Stability Goals
24
10
TEST SUITE : TP010213_REFERENCE ARCHITECTURE B PERFORMANCE TEST PLAN 25
10.1 Test Suite : Reference Architecture B-1
25
10.1.1
Test Case CAM-94636: Mixed call flow at 80% using IPv4 in call flow
25
10.1.2
Test Case CAM-94637: Mixed call flow at 100% using IPv4 in call flow
26
10.1.3
Test Case CAM-94638: Mixed call flow at 120% using IPv4 in call flow
27
10.1.4
Test Case CAM-94639: Mixed call flow at 150% using IPv4 in call flow
28
10.1.5
Test Case CAM-94640: Mixed call flow at 200% using IPv4 in call flow
29
10.1.6
Test Case CAM-94641: Mixed call flow at 80% using IPv6 in call flow
30
10.1.7
Test Case CAM-94642: Mixed call flow at 100% using IPv6 in call flow
31
10.1.8
Test Case CAM-94643: Mixed call flow at 120% using IPv6 in call flow
31
10.1.9
Test Case CAM-94644: Mixed call flow at 150% using IPv6 in call flow
32
10.1.10
Test Case CAM-94645: Mixed call flow at 200% using IPv6 in call flow
33
10.1.11
Test Case CAM-95599: Mixed call flow at 100% using Dual-Stack IP in call flow
34
10.1.12
Test Case CAM-94647: Mixed call flow at 100% using IPv4 in call flow in all IPv6 topology
35
10.1.13
Test Case CAM-95602: Mixed call flow at 100% using IPv4 in call flow in mixed IPv4/IPv6 topology
36
10.1.14
Test Case CAM-97266: Mixed call flow at 100% using IPv4 in call flow with Topology Hiding
37
10.1.15
Test Case CAM-94648: Mixed call flow at 100% using IPv4 in call flow using SCTP transport
38
10.1.16
Test Case CAM-95432: BL460 MPE with 64GB RAM can Support 6.5 Million Concurrent Sessions in B-1
Topology 39
10.1.17
Test Case CAM-95433: BL460 MRA with 64GB RAM can support up to 52,000 Transactions/second in B-1
Topology 40
10.1.18
Test Case CAM-95619: BL460 MRA with 64GB RAM can support up to 20M bindings in B-1 Topology 41
10.1.19
Test Case CAM-94646: Longevity Test - Mixed call flow at 100% using IPv4 in call flow
42
10.1.20
Test Case CAM-95439: CMP Can Support Managing up to 50 MPEs/MRA Geo-Redundant Clusters
43
10.1.21
Test Case CAM-95437: BL460 MRA with 64GB RAM can support up to 8k Client Connections
44
10.1.22
Test Case CAM-97374: MRA will support 16 Diameter connections from a DRA
46
10.1.23
Test Case CAM-98672: Baseline Step Test - BL460-G8 MPE
46
10.1.24
Test Case CAM-98673: Baseline Step Test - BL460-G8 MRA
47
10.1.25
Test Case CAM-99004: Baseline Step Test - BL460-G9 MPE
48
10.1.26
Test Case CAM-99005: Baseline Step Test - BL460-G9 MRA
49
10.1.27
Test Case CAM-98677: Mixed call flow at 100% loads with over 1k Policies Deployed
50
10.1.28
Test Case CAM-98702: 2-Site MRA at 100% load in Legacy mode
51
10.1.29
Test Case CAM-98703: 2-Site MRA at 100% load in Algov1 mode
51
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 3 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
10.2 Test Suite : Reference Architecture B-2
52
10.2.1
Test Case CAM-95618: BL460 MRA with 64GB RAM can support up to 52,000 Transactions/second in B-2
Topology 52
10.2.2
Test Case CAM-95435: BL460 MRA with 64GB RAM can support up to 20M bindings in B-2 Topology
54
10.3 Test Suite : Reference Architecture B-3
10.3.1
Test Case CAM-94649: Mixed call flow at 80% using IPv4 in call flow
10.3.2
Test Case CAM-94650: Mixed call flow at 100% using IPv4 in call flow
10.3.3
Test Case CAM-94651: Mixed call flow at 120% using IPv4 in call flow
10.3.4
Test Case CAM-94652: Mixed call flow at 150% using IPv4 in call flow
10.3.5
Test Case CAM-94653: Mixed call flow at 200% using IPv4 in call flow
10.3.6
Test Case CAM-94654: Longevity Test - Mixed call flow at 100% using IPv4 in call flow
10.3.7
Test Case CAM-94655: Mixed call flow at 100% using IPv4 in call flow using SCTP transport
10.3.8
Test Case CAM-95434: Oracle X5-2 MRA can support up to 50,000 Transactions/second
10.3.9
Test Case CAM-95431: Oracle X5-2 MPE can Support 10 Million Concurrent Sessions
10.3.10
Test Case CAM-95440: Oracle X5-2 MRA can support up to 40M bindings
10.3.11
Test Case CAM-95438: Oracle X5-2 MRA can support up to 8k Client Connections
10.3.12
Test Case CAM-97382: VoLTE-Plus-Inbound-S9 Roaming call flow at 100%
10.3.13
Test Case CAM-97323: VoLTE-Plus-Outbound-S9 Roaming call flow at 100%
55
55
56
56
57
58
59
60
60
62
63
64
65
66
10.4 Test Suite : Reference Architecture C-1
10.4.1
CAM-94630:RADIUS with Sd and SPR Profile Lookup at 80% load
10.4.2
CAM-94631:RADIUS with Sd and SPR Profile Lookup at 100% load
10.4.3
Test Case CAM-94632: RADIUS with Sd and SPR Profile Lookup at 120% load
10.4.4
Test Case CAM-94633: RADIUS with Sd and SPR Profile Lookup at 150% load
10.4.5
Test Case CAM-94634: RADIUS with Sd and SPR Profile Lookup at 200% load
10.4.6
Test Case CAM-94635: RADIUS with Sd and SPR Profile Lookup Longevity Test
68
68
69
70
71
72
73
10.5 Test Suite : Reference Architecture B Impairment tests
74
10.5.1
Test Case CAM-95400: BL460 C-Class Active MPE Failover to Standby MPE at 100% Load
74
10.5.2
Test Case CAM-99006: BL460-G9 C-Class Active MPE Failover to Standby MPE at 100% Load
76
10.5.3
Test Case CAM-95401: Oracle X5-2 Active MPE Failover to Standby MPE at 100% Load
77
10.5.4
Test Case CAM-95402: BL460 C-Class Geo-Redundant Site1 MPE Failover to Site2 MPE at 100% Load
77
10.5.5
Test Case CAM-95403: Oracle X5-2 Geo-Redundant Site1 MPE Failover to Site2 MPE at 100% Load
78
10.5.6
Test Case CAM-95404: BL460 C-Class Active MRA Failover to Standby MRA at 100% Load
79
10.5.7
Test Case CAM-99007: BL460-G9 C-Class Active MRA Failover to Standby MRA at 100% Load
80
10.5.8
Test Case CAM-95405: Oracle X5-2 MRA Failover to Standby MRA at 100% Load
81
10.5.9
Test Case CAM-95406: BL460 C-Class Geo-Redundant primary site MRA Failover to secondary site MRA at
100% Load
82
10.5.10
Test Case CAM-95407: Oracle X5-2 Geo-Redundant primary site MRA Failover to secondary site MRA at
100%
83
10.5.11
Test Case CAM-95408: Surge of Session Creation Requests
83
10.5.12
Test Case CAM-98453: 100% Load to MRA with 80ms Latency Added Between MRA and MPE
84
10.5.13
Test Case CAM-98454: 100% Load to MRA with 80ms Latency Added Between MPEs and SPR
85
10.5.14
Test Case CAM-99096: 100% Load to 2-site MRAs with dropped DRMA messages between MRAs
87
Appendix A. Review Summaries
89
1
CHANGE HISTORY
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 4 of 94
PROPRIETARY INFORMATION
Date
4/30/2015
5/12/2015
5/28/2015
6/11/2015
6/15/2015
7/13/2015
2
Rev #
0.1
0.2
0.3
0.4
0.5
1.0
INTERNAL USE ONLY
Author
P. Piacentini
P. Piacentini
P. Piacentini
P. Piacentini
P. Piacentini
P. Piacentini
Approved
No
No
No
No
No
Yes
REFERENCES
Title
[1] Product Function Specification (PFS) 12.1
[2] Reference Architecture for Release 12.1
[3]
[4]
[5]
3
Revision Description
1st draft
Changes based on online review comments
Changes based on in-person review comments
Added some test cases for BL460-G9
Added test case where messages dropped between MRAs
Document approved
Number
PF006194
FE007538
revision
1.3
1.1
Author
C. Berlutti
C. Berlutti
Date
04/15/2015
03/24/2015
PURPOSE AND SCOPE
This test plan is based on the configurations and call flows as described in FE007538 - Reference Architecture for Release 12.1. The tests
contained within are intended to verify the requirements that are described in that document.
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 5 of 94
PROPRIETARY INFORMATION
4
INTERNAL USE ONLY
TERMINOLOGY
Binding
A binding between a subscriber identifier (e.g. IMSI, MSISDN, IP Address) and an MPE. The MRA
maintains bindings, and there is one binding per subscriber even if the subscriber has multiple
active sessions.
BRAS/BNG
A BRAS or BNG is where the access session is terminated. It also serves as policy enforcement
point for fixed line networks. In theory, a BRAS supports PPP sessions across ATM, while BNG
supports native IP over Ethernet (using DHCP extensions to define each subscriber) across an
Ethernet WAN. In reality, many current devices support both network types, so the terms are often
used interchangeably. The terms BRAS and BNG are used interchangeably in this document.
Other terms describing this device include edge router and services router.
Policy Solution
The total of all Policy Components (CMPs, MRAs, MPEs) and Policy Systems (see below) that
comprise the policy deployment for a network. A Policy Solution may contain one or more Policy
Systems.
Policy System
All of the Policy Components under the control of a single NW-CMP instance. A single NW-CMP
instance with multiple S-CMP may host multiple MRAs and multiple MPEs behind the MRAs. A
Policy System will generally consist of at least two Sites, since a NW-CMP instance generally hosts
components at two mated, geo-diverse Sites.
PCRF Segment
A partition of the Policy function within an operator’s network
• Policy segments may be defined by functionality, scale, geography, PCRF vendor or other
operator driven partitioning motive
• Intra-segment Policy session correlation may or may not be provided by the PCRF
• Inter-segment Policy Session correlation is NOT provided by the PCRF
For the Oracle PCRF, a Policy Segment is a collection of one or more MPE clusters, fronted by one
or more PFE Clusters and managed by a single CMP Cluster.
An Oracle Policy Segment:
• May span multiple PCRF Nodes
• May span multiple PCRF Sites
• Provides intra-segment Policy session correlation
Session
A Diameter session between the MPE and an external device (e.g. a Gx, Gxa, Gx-Lite, Rx and Sy
session), Sh, as well as non-diameter sessions, are excluded. Subscribers can (and typically do)
maintain multiple sessions at any given time.
Site
All Policy Components physically hosted at a single location.
Transaction
An incoming or outgoing message (such as a Diameter CCR-I or an RAR) and all the associated
internal product processing until the subsequent message is sent/received. Note that the
subsequent message need not be of the same type as the initial message. For example:
- Gx CCR-u incoming Diameter message, internal processing, Gx CCA Diameter response
message = 1 transaction
- Gx CCR-i incoming Diameter message, internal processing, Sh UDR outgoing Diameter HSS
query message, Sh UDA Diameter response message, internal processing, Gx CCA Diameter
response message = 2 Transactions
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 6 of 94
PROPRIETARY INFORMATION


5
INTERNAL USE ONLY
DRMA transactions between MRAs in multi-site deployments, and transactions between MRA
and MPEs, are not included in this rated capacity.
All transactions between the MPE and GWs, AFs, MRAs, SPRs, LDAP Servers, OCSs, and
SMSC Servers are included in this capacity.
TOPOLOGIES
5.1
REFERENCE B-1 TOPOLOGY
Reference B-1 topology consists of all Geo-Redundant MRA and MPE clusters. There will be two MRA sites. The
second MRA site will most likely not be a full chassis of blades, depending on hardware availability. All C-Class
blades are BL460 G8. The RMS is DL380. Subscriber authentication will be performed by LDAP and/or SPR
lookup. Quota and Entity State variables will be managed and stored by the SPR. Both Legacy SPR and R10.0
OCUDR will be used. TDF and OCS functions will be simulated by seagull servers.
Although the DUT will be the MRA and MPEs at Site-1, for every test there will be a small amount of traffic routed
between Site-1 and Site-2
5.1.1 REFERENCE B-1 PERFORMANCE METRICS
Component
Performance Target (per instance)
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
System TPS capability
Title:
System TPS requirement
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 7 of 94
PROPRIETARY INFORMATION
5.2
MRA Per
PCRF
GeoRedundant
Segment(Statefull,
Dynamic)- (1)
MRA cluster
52,000 TPS
40M Bindings
MPE Per
PCRF
GeoRedundant
Segment- (7)
MPE Clusters
6,500 TPS for Gen 8
6,500 TPS for Gen 9
15M Concurrent Sessions Gen 9
15M Concurrent Session/Gen 8
INTERNAL USE ONLY
52,000 (100% rated, no failures)
41,600 TPS@80%


HP Gen 9 & HP Gen 8
BL460 Server Clusters
45,500 TPS@100%

HP Gen 9 & HP Gen 8
BL460 Server Clusters
105M Concurrent
Sessions@100%

HP Gen 9 & HP Gen 8
BL460 Server Clusters
HP Gen 9 & HP Gen 8
BL460 Server Clusters
36,400 TPS@80%

HP Gen 9 & HP Gen 8
BL460 Server Clusters
84M Concurrent
Sessions@80%

HP Gen 9 & HP Gen 8
BL460 Server Clusters
REFERENCE B-2 TOPOLOGY
Topology B-2 consists of a single fully loaded chassis, consisting of an MRA cluster and seven MPE clusters. The
chassis is managed by a rack-mount CMP cluster. All C-Class blades are BL460 G8. The RMS is DL380. Just a
minimal set of tests will be run in this configuration, since it’s a subset of topology B-1 and will inherently be
covered by tests run on that topology.
5.2.1 REFERENCE B-2 PERFORMANCE METRICS
Component
(1) MRA
Active/Standby
Performance Target (per instance)
52,000 TPS
10K PCRF Client connections
40M Bindings
System TPS capability
System TPS requirement
52,000 (100% rated, no failures)
41,600 TPS@80%


6,500 TPS for Gen 8
6,500 TPS for Gen 9
45,500 TPS@100%

HP Gen 9 & HP Gen 8
HP Gen 9 & HP Gen 8
BL460 Server Clusters
HP Gen 9 & HP Gen 8
BL460 Server Clusters
Cluster
(7) MPE
Active/Standby
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
36,400 TPS@80%

HP Gen 9 & HP Gen 8
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 8 of 94
PROPRIETARY INFORMATION
Clusters
5.3
INTERNAL USE ONLY
15M Concurrent Sessions Gen 9
15M Concurrent Session/Gen 8
BL460 Server Clusters
105M Concurrent
Sessions@100%

HP Gen 9 & HP Gen 8
BL460 Server Clusters
BL460 Server Clusters
84M Concurrent
Sessions@80%

HP Gen 9 & HP Gen 8
BL460 Server Clusters
REFERENCE B-3 TOPOLOGY
Topology B-3 consists of all rack-mount servers. In this case the servers will be Oracle X5-2 servers. The setup
consists of an RMS CMP cluster, and RMS MRA cluster, and 1 or more RMS MPE clusters.
5.3.1 WIRELESS OUTBOUND ROAMING SECTION OF TOPOLOGY B-3
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 9 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
5.3.2 WIRELESS INBOUND ROAMING SECTION OF TOPOLOGY B-3
5.3.3 REFERENCE B-3 PERFORMANCE METRICS
Component
(1)
MRA
Performance Target (per instance)
System TPS capability
System TPS requirement
52,000 TPS
40M Bindings
52,000 (100% rated, no failures)
41,600 TPS@80%
6,500 TPS for Gen 8
15M Concurrent Sessions
13,000 TPS@100%
30M Concurrent
Sessions@100%
10,400 TPS@80%
44M Concurrent
Sessions@80%
Active/Standby
Cluster
(2) MPE
Active/Standby
Clusters
5.4
REFERENCE C-1 TOPOLOGY
Topology C-3 consists of all rack-mount servers. The same Oracle X5-2 servers that were used in topology B-3
will be used in this setup. The setup consists of an RMS CMP cluster, and RMS MRA cluster, and 1 or more RMS
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 10 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
MPE clusters. Topology C-3 will focus on the Fixed Mobile Convergence use case, with a mix of RADIUS and
Diameter interfaces.
5.4.1 REFERENCE C-1 PERFORMANCE METRICS
Component
(1)
MRA
Performance Target (per instance)
System TPS capability
System TPS requirement
52,000 TPS
40M Bindings
52,000 (100% rated, no failures)
41,600 TPS@80%
6,500 TPS
15M Concurrent Sessions
13,000 TPS@100%
30M Concurrent
Sessions@100%
10,400 TPS@80%
44M Concurrent
Sessions@80%
Active/Standby
Cluster
(2) MPE
Active/Standby
Clusters
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 11 of 94
PROPRIETARY INFORMATION
6
INTERNAL USE ONLY
SERVER AND SYSTEM CAPACITIES
The following table described the performance capacities of the various hardware combinations that are supported in Release
12.1.
Hardware
TPS
Per-MPE Capacity
(100% Rated Capacity)
Conc.
Sub.
Sessions
Bindings
6.5M
N/A
TPS
Per-MRA Capacity
(100% Rated Capacity)
Conc. Sessions
Sub. Bindings
BL460 G6
4500
25,200
N/A
20M
DL360/380 G6
4500
6.5M
N/A
25,200
N/A
20M
BL460 G8 (64G
RAM)
BL460 G8+
(64G RAM)
BL460 G8 (256G
RAM
DL 380 G8
(64G RAM)
DL 380 G8 +
(64G RAM)
BL460 G9
(256G RAM)
DL 380 G9
(128G RAM)
Oracle Netra X32 RMS
(128G RAM)
Oracle X5-2 RMS
(256G RAM)
6000
6.5M
N/A
50k
N/A
20M*
6000
6.5M
N/A
50k
N/A
20M*
6500
15M
N/A
52K
NA
40M
6000
6.5M
N/A
50k
N/A
20M*
6000
6.5M
N/A
50k
N/A
20M*
6500
15 M
N/A
52k
N/A
40M
6000
10 M
N/A
50k
N/A
40M
6000
10M
N/A
50k
N/A
40M
6500
15M
N/A
52k
N/A
40M
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 12 of 94
PROPRIETARY INFORMATION
7
INTERNAL USE ONLY
CALL FLOWS
Each test will use a mix of call flows. This section descibes each of the call flows used in detail.
The percentage of each call type that will be attempted in the mixed calls traffic model is shown in the table below.
Percentage
of Total
Traffic
Call Flow Name
LTE + Subscriber Quota(2 PDN Connections)
5
LTE + Subscriber Quota(2 PDN Connections) Allot Variation
5
LTE No Quota (2 PDN connections) +Sy
10
GPRS(2 PDN Connections) + Pooled Quota + Regular
Subscriber Quota
10
GPRS(2 PDN Connections) with secondary PDP + Secondary
Gx Subscriber Quota)
10
IMS VoLTE Call + Sy
OCS Triggered Throttling
25
5
Sh Triggered Throttling
Application Monitoring on Sd
LTE + Quota on Sd
VoLTE + S9 Outbound Roaming
VoLTE + S9 InBound Roaming
Total Percentage
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
5
5
10
5
5
100
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 13 of 94
PROPRIETARY INFORMATION
7.1
INTERNAL USE ONLY
Call1 - LTE + Subscriber Quota(2 PDN Connections)
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 14 of 94
PROPRIETARY INFORMATION
7.2
INTERNAL USE ONLY
CALL2 - SUBSCRIBER QUOTA(2 PDN CONNECTIONS) ALLOT VARIATION
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 15 of 94
PROPRIETARY INFORMATION
7.3
INTERNAL USE ONLY
CALL3 - LTE NO QUOTA 2 PDN CONNECTION +SY
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 16 of 94
PROPRIETARY INFORMATION
7.4
INTERNAL USE ONLY
CALL4 - GPRS(2 PDN CONNECTIONS) + POOLED QUOTA + REGULAR SUBSCRIBER QUOTA
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 17 of 94
PROPRIETARY INFORMATION
7.5
INTERNAL USE ONLY
CALL5 - GPRS(2 PDN CONNECTIONS) WITH SECONDARY PDP + SECONDARY GX SUBSCRIBER
QUOTA
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 18 of 94
PROPRIETARY INFORMATION
7.6
INTERNAL USE ONLY
CALL6 - IMS VOLTE CALL (VOLTE CALLS WILL CONTAIN BOTH IMSI AND E164 SUB ID)
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 19 of 94
PROPRIETARY INFORMATION
7.7
INTERNAL USE ONLY
CALL7 - APPLICATION MONITORING ON SD
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 20 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
7.8
CALL8 - LTE + QUOTA ON SD
7.9
VARIATION1 - OCS TRIGGERED THROTTLING
A small percentage of the calls that use an OCS will have an SNR from the OCS trigger an RAR to the Gx interface to the
gateway.
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 21 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
7.10 VARIATION2 - SH TRIGGERED THROTTLING
A small percentage of the calls will have a PNR sent from the SPR, which will triggers an RAR on the Gx interface to the
gateway,
7.11 VOLTE PLUS S9 OUTBOUND ROAMING
This call flow will use the same script and policies used in “Call6 – IMS VoLTE Call”. The Policy Server configuration will be changed to
support the S9 configuration and topology
7.12 VOLTE PLUS S9 INBOUND ROAMING
This call flow will use the same script and policies used in “Call6 – IMS VoLTE Call”. The Policy Server configuration will be changed to
support the S9 configuration and topology
8
GENERAL CONFIGURATION SETUP


The MRAs are configured in stateful mode, as backups of each other
All C-Class blades will have mezzanine cards installed, and the traffic will be separated so that OAM traffic uses bond0, and
signaling traffic will use bond1
All MPEs will be configured with “Validate user” enabled
All MPEs will have Primary and Secondary HSS Data Sources configured, using HSS profilev4
Subscription will be enabled on the HSS Data Sources
A subscription-Id of E164 will be used
MRA and MPEs indexed by E164 and IP, MRA also indexed by Session-ID, and primary indexing on the MRA is E164
Prior to running the performance tests, each of the MPEs will be preloaded with 1M sessions.
Prior to running the tests, all statistics will be cleared
Session cleanup will be disabled so that these sessions stay intact during the tests.
Each test will run for 2 hours, and memory, CPU, latency, and other KPIs will be collected during the test
The CMP will be configured to send SNMP traps to a remote trap receiver
The CMP will be configured to forward syslog messages to a remote syslog server











DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 22 of 94
PROPRIETARY INFORMATION
9
INTERNAL USE ONLY
EXAMPLE POLICIES
Many of the scripts and policies that will be needed to support these test cases have not yet been developed, therefore they
cannot be included in this test plan. Once the testing gets underway, it is assumed that they will be accessable to anyone that
wouild be interested in looking at them or trying to use them.
Unless it proves too difficult to implement, the current plan is to use the APN in the policy conditions to trigger the correct
policies based on the expected call flow. Also, each call flow script will use a unique set of Subscription-Ids so that the calls
can be differentiated.
Below is an example of the policies that could be used for Call #1:
PERF_call1_int_init_save_state
where the request is creating a new session
And where the session is an enforcement session
And where the device type is not DPI
And where the APN matches one of call1_int
And where the network element name matches one of ggsn-1,ggsn-2,ggsn-3,ggsn-4,ggsn-5,ggsn-6,ggsn-7
set the subscriber state variable call1_session_start to now using configured local time and save always
set the subscriber state variable call1_reset to now + 1 minutes rounded down with same granularity using
CONFIGURED LOCAL TIME and save always
continue processing message
PERF_call1_int_grant_subscriber_quota
where the request is creating a new session,modifying an existing session
And where the session is an enforcement session
And where the APN matches one of call1_int
grant total volume to 100 percent used for PERF_call1_subscriber using call1_int
set session revalidation time to 60 seconds
continue processing message
PERF_call1_int_grant_subscriber_quota_on_update
where the request is modifying an existing session
And where the session is an enforcement session
And where the APN matches one of call1_int
grant total volume to 100 percent used for PERF_call1_subscriber using call1_int
continue processing message
PERF_call1_int_term_reset_quota_save_state
where the request is terminating an existing session
And where the session is an enforcement session
And where the device type is not DPI
And where the APN matches one of call1_int
And where the network element name matches one of ggsn-1,ggsn-2,ggsn-3,ggsn-4,ggsn-5,ggsn-6,ggsn-7
set the subscriber state variable call1_session_stop to now using CONFIGURED LOCAL TIME and save always
reset all plan usage
continue processing message
10 GENERAL GOALS OF THE TESTS
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 23 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
10.1 LATENCY GOALS
For each test that sends traffic at 80% and 100% of the rated capacity of the DUT, the following latency values are expected
to be met:
1.
92% of the calls are expected to be achieved with a latency of under 100 ms
2.
99.9% of the calls are expected to be achieved with a latency of under 200 ms
3.
The transaction response time of the Policy system with two sites for 92% of all transactions shall not exceed 167 +
(2 * LL) ms; and
4.
The transaction response time of the Policy system with two sites for 99.9% of all transactions shall not exceed 250
+ (2 * LL) ms, where LL is the latency of the link between the two systems
5.
The latency from the SPR is expected and assumed to be 25ms or less as part of the transaction.
6.
The latency from the OCS simulator is expected and assumed to be 20ms or less as part of the transaction
7.
The latency from the PCEF and AF simulator is expected and assumed to be 20ms or less as part of the transaction.
10.2 STABILITY GOALS
For each test case that sends traffic at 120% or more of the rated capacity of the DUT, the following expectations are to be
met:
1.
There will be no unexpected exceptions seen in the logs of the MRA or MPEs
2.
There will be no unexpected failure or failover of the MRA or MPEs
3.
Some load shedding at increasingly higher percentage is expected as the traffic is increased beyond the rated
capacity
a.
4.
The systems are expected to return no nor,normal serviceabbilty when the traffic is lowered to fall within
it’s rated capacity
The systems are expected to recover after each of the impariment tests are run
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 24 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
10 TEST SUITE : TP010213_REFERENCE ARCHITECTURE B PERFORMANCE TEST
PLAN
This test plan is for R12.1.
10.1 TEST SUITE : REFERENCE ARCHITECTURE B-1
Reference Architecture B-1
Reference Architecture B-1 consists of an RMS CMP, along with a full C-Class chassis, containing 1 MRA blade pair, and 7
MPE blade pairs. It also includes a second C-Class chassis half populated with blades, including an MRA, and 7 MPEs,
which are configured in the topology as a second site to the first chassis, providing Geo-Redundancy.
A second backup MRA site will also be set up, and for each test a small amount of traffic will be routed between the MRAs.
·
·
·
·
·
HP Gen8 servers will be used for these tests. There are a few Gen8Plus blades in the setup as well
The MRAs are configured in stateful mode.
All C-Class blades will have mezzanine cards installed, and the traffic will be separated so that OAM traffic uses
bond0, and signaling traffic will use bond1
Prior to running the performance tests, each of the MPEs will be preloaded with 1M sessions.
Session cleanup will be disabled on most of these tests, so that the preloaded sessions stay intact during the tests.
Session cleanup will be enabled for the longevity tests
10.1.1
TEST CASE CAM-94636: MIXED CALL FLOW AT 80% USING IPV4 IN CALL FLOW
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-1 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 80% of the capacity of the MPEs.
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 80% of the rated load of the MPEs
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.
Expected Results:
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 25 of 94
PROPRIETARY INFORMATION



INTERNAL USE ONLY
The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-130: R-19526026-130 The policy systems described in Sections 2.4.3, 2.4.4,
2.4.5, and 2.4.6 while running
R-19526026-200: R-19526026-200 Reference Architecture B-1 shall be supported with
the following minimum physical har
R-19526026-205: R-19526026-205 The Policy DL460 G8 blade configuration used for
testing MPE and MRA shall include ad
R-19526026-252: R-19526026-252 The general traffic model described in [R-19526026230] shall be repeated on hardware
R-19526026-100: R-19526026-100 The Policy Component capacities shown in Table 2
shall be verified for each server t
R-19526026-600: R-19526026-600 The following SPR/OCUDR Configurations and as
shown in Table 15 shall be verified usi
Keywords:
Performance
ANCHORAGE
10.1.2
TEST CASE CAM-94637: MIXED CALL FLOW AT 100% USING IPV4 IN CALL FLOW
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-1 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 100% of the capacity of the MPEs.
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 100% of the rated load of the MPEs
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.



The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 26 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
Requirements
R-19526026-130: R-19526026-130 The policy systems described in Sections 2.4.3, 2.4.4,
2.4.5, and 2.4.6 while running
R-19526026-200: R-19526026-200 Reference Architecture B-1 shall be supported with
the following minimum physical har
R-19526026-205: R-19526026-205 The Policy DL460 G8 blade configuration used for
testing MPE and MRA shall include ad
R-19526026-252: R-19526026-252 The general traffic model described in [R-19526026230] shall be repeated on hardware
R-19526026-100: R-19526026-100 The Policy Component capacities shown in Table 2
shall be verified for each server t
R-19526026-600: R-19526026-600 The following SPR/OCUDR Configurations and as
shown in Table 15 shall be verified usi
Keywords:
Performance
ANCHORAGE
10.1.3
TEST CASE CAM-94638: MIXED CALL FLOW AT 120% USING IPV4 IN CALL FLOW
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-1 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 120% of the capacity of the MPEs.
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 120% of the rated load of the MPEs
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.
Expected Results:



The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-130: R-19526026-130 The policy systems described in Sections 2.4.3, 2.4.4,
2.4.5, and 2.4.6 while running
R-19526026-200: R-19526026-200 Reference Architecture B-1 shall be supported with
the following minimum physical har
R-19526026-205: R-19526026-205 The Policy DL460 G8 blade configuration used for
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 27 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
testing MPE and MRA shall include ad
R-19526026-252: R-19526026-252 The general traffic model described in [R-19526026230] shall be repeated on hardware
R-19526026-100: R-19526026-100 The Policy Component capacities shown in Table 2
shall be verified for each server t
Keywords:
10.1.4
Performance
ANCHORAGE
TEST CASE CAM-94639: MIXED CALL FLOW AT 150% USING IPV4 IN CALL FLOW
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-1 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 150% of the capacity of the MPEs.
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 150% of the rated load of the MPEs
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.
6. Verify that if load shedding kicks in, the correct levels and discards are used
Expected Results:



The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-130: R-19526026-130 The policy systems described in Sections 2.4.3, 2.4.4,
2.4.5, and 2.4.6 while running
R-19526026-200: R-19526026-200 Reference Architecture B-1 shall be supported with
the following minimum physical har
R-19526026-205: R-19526026-205 The Policy DL460 G8 blade configuration used for
testing MPE and MRA shall include ad
R-19526026-252: R-19526026-252 The general traffic model described in [R-19526026-
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 28 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
230] shall be repeated on hardware
R-19526026-260: R-19526026-260 The mixed call model defined in Section 2.3.14 shall
be repeated on hardware configu
R-19526026-270: R-19526026-270 The mixed call model defined in Section 2.3.14 shall
be repeated on hardware configu
R-19526026-100: R-19526026-100 The Policy Component capacities shown in Table 2
shall be verified for each server t
Keywords:
10.1.5
Performance
ANCHORAGE
TEST CASE CAM-94640: MIXED CALL FLOW AT 200% USING IPV4 IN CALL FLOW
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-1 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 200% of the capacity of the MPEs.
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 200% of the rated load of the MPEs
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.
6. Verify that if load shedding kicks in, the correct levels and discards are used
Expected Results:



The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-130: R-19526026-130 The policy systems described in Sections 2.4.3, 2.4.4,
2.4.5, and 2.4.6 while running
R-19526026-200: R-19526026-200 Reference Architecture B-1 shall be supported with
the following minimum physical har
R-19526026-205: R-19526026-205 The Policy DL460 G8 blade configuration used for
testing MPE and MRA shall include ad
R-19526026-252: R-19526026-252 The general traffic model described in [R-19526026-
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 29 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
230] shall be repeated on hardware
R-19526026-260: R-19526026-260 The mixed call model defined in Section 2.3.14 shall
be repeated on hardware configu
R-19526026-270: R-19526026-270 The mixed call model defined in Section 2.3.14 shall
be repeated on hardware configu
R-19526026-100: R-19526026-100 The Policy Component capacities shown in Table 2
shall be verified for each server t
Keywords:
10.1.6
Performance
ANCHORAGE
TEST CASE CAM-94641: MIXED CALL FLOW AT 80% USING IPV6 IN CALL FLOW
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-1 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 80% of the capacity of the MPEs. The format of the IP
address in the calls will be IPv6
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 80% of the rated load of the MPEs
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.
Expected Results:



The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-130: R-19526026-130 The policy systems described in Sections 2.4.3, 2.4.4,
2.4.5, and 2.4.6 while running
R-19526026-200: R-19526026-200 Reference Architecture B-1 shall be supported with
the following minimum physical har
R-19526026-205: R-19526026-205 The Policy DL460 G8 blade configuration used for
testing MPE and MRA shall include ad
R-19526026-100: R-19526026-100 The Policy Component capacities shown in Table 2
shall be verified for each server t
Keywords:
Performance
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 30 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
ANCHORAGE
10.1.7
TEST CASE CAM-94642: MIXED CALL FLOW AT 100% USING IPV6 IN CALL FLOW
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-1 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 80% of the capacity of the MPEs. The format of the IP
address in the calls will be IPv6
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 100% of the rated load of the MPEs
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.
Expected Results:



The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-130: R-19526026-130 The policy systems described in Sections 2.4.3, 2.4.4,
2.4.5, and 2.4.6 while running
R-19526026-200: R-19526026-200 Reference Architecture B-1 shall be supported with
the following minimum physical har
R-19526026-205: R-19526026-205 The Policy DL460 G8 blade configuration used for
testing MPE and MRA shall include ad
R-19526026-100: R-19526026-100 The Policy Component capacities shown in Table 2
shall be verified for each server t
Keywords:
Performance
ANCHORAGE
10.1.8
Author:
TEST CASE CAM-94643: MIXED CALL FLOW AT 120% USING IPV6 IN CALL FLOW
ppiacentini
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 31 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
Summary:
This test is to verify that the Reference Architecture B-1 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 80% of the capacity of the MPEs. The format of the IP
address in the calls will be IPv6
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 120% of the rated load of the MPEs
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.
Expected Results:



The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-130: R-19526026-130 The policy systems described in Sections 2.4.3, 2.4.4,
2.4.5, and 2.4.6 while running
R-19526026-200: R-19526026-200 Reference Architecture B-1 shall be supported with
the following minimum physical har
R-19526026-205: R-19526026-205 The Policy DL460 G8 blade configuration used for
testing MPE and MRA shall include ad
R-19526026-100: R-19526026-100 The Policy Component capacities shown in Table 2
shall be verified for each server t
Keywords:
Performance
ANCHORAGE
10.1.9
TEST CASE CAM-94644: MIXED CALL FLOW AT 150% USING IPV6 IN CALL FLOW
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-1 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 80% of the capacity of the MPEs. The format of the IP
address in the calls will be IPv6
Steps:
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 32 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 150% of the rated load of the MPEs
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.
6. Verify that if load shedding kicks in, the correct levels and discards are used
Expected Results:



The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-130: R-19526026-130 The policy systems described in Sections 2.4.3, 2.4.4,
2.4.5, and 2.4.6 while running
R-19526026-200: R-19526026-200 Reference Architecture B-1 shall be supported with
the following minimum physical har
R-19526026-205: R-19526026-205 The Policy DL460 G8 blade configuration used for
testing MPE and MRA shall include ad
R-19526026-260: R-19526026-260 The mixed call model defined in Section 2.3.14 shall
be repeated on hardware configu
R-19526026-270: R-19526026-270 The mixed call model defined in Section 2.3.14 shall
be repeated on hardware configu
R-19526026-100: R-19526026-100 The Policy Component capacities shown in Table 2
shall be verified for each server t
Keywords:
Performance
ANCHORAGE
10.1.10 TEST CASE CAM-94645: MIXED CALL FLOW AT 200% USING IPV6 IN CALL FLOW
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-1 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 80% of the capacity of the MPEs. The format of the IP
address in the calls will be IPv6
Steps:
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 33 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 200% of the rated load of the MPEs
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.
6. Verify that if load shedding kicks in, the correct levels and discards are used
Expected Results:



The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-130: R-19526026-130 The policy systems described in Sections 2.4.3, 2.4.4,
2.4.5, and 2.4.6 while running
R-19526026-200: R-19526026-200 Reference Architecture B-1 shall be supported with
the following minimum physical har
R-19526026-205: R-19526026-205 The Policy DL460 G8 blade configuration used for
testing MPE and MRA shall include ad
R-19526026-260: R-19526026-260 The mixed call model defined in Section 2.3.14 shall
be repeated on hardware configu
R-19526026-270: R-19526026-270 The mixed call model defined in Section 2.3.14 shall
be repeated on hardware configu
R-19526026-100: R-19526026-100 The Policy Component capacities shown in Table 2
shall be verified for each server t
Keywords:
Performance
ANCHORAGE
10.1.11 TEST CASE CAM-95599: MIXED CALL FLOW AT 100% USING DUAL-STACK IP IN CALL
FLOW
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-1 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 100% of the capacity of the MPEs. The UE address in the
Gx messages will contain both IPv4 and IPv6 IP addresses
Steps:
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 34 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 100% of the rated load of the MPEs
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.



The MRA and MPE should process 100% of the calls generated by the traffic generator, using dualstack IP addressing.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-130: R-19526026-130 The policy systems described in Sections 2.4.3, 2.4.4,
2.4.5, and 2.4.6 while running
R-19526026-200: R-19526026-200 Reference Architecture B-1 shall be supported with
the following minimum physical har
R-19526026-205: R-19526026-205 The Policy DL460 G8 blade configuration used for
testing MPE and MRA shall include ad
R-19526026-251: R-19526026-251 The general traffic model described in [R-19526026230] shall be repeated on hardware
R-19526026-100: R-19526026-100 The Policy Component capacities shown in Table 2
shall be verified for each server t
Keywords:
Performance
ANCHORAGE
10.1.12 TEST CASE CAM-94647: MIXED CALL FLOW AT 100% USING IPV4 IN CALL FLOW IN
ALL IPV6 TOPOLOGY
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-1 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 100% of the capacity of the MPEs. The topology will be
using all IPv6 addresses for OAM, signaling, and replication.
Steps:
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 35 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 100% of the rated load of the MPEs
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.



The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-130: R-19526026-130 The policy systems described in Sections 2.4.3, 2.4.4,
2.4.5, and 2.4.6 while running
R-19526026-200: R-19526026-200 Reference Architecture B-1 shall be supported with
the following minimum physical har
R-19526026-205: R-19526026-205 The Policy DL460 G8 blade configuration used for
testing MPE and MRA shall include ad
R-19526026-250: R-19526026-250 The general traffic model described in [R-19526026230] shall be repeated on hardware
R-19526026-100: R-19526026-100 The Policy Component capacities shown in Table 2
shall be verified for each server t
Keywords:
Performance
ANCHORAGE
10.1.13 TEST CASE CAM-95602: MIXED CALL FLOW AT 100% USING IPV4 IN CALL FLOW IN
MIXED IPV4/IPV6 TOPOLOGY
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-1 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 100% of the capacity of the MPEs. The topology will be
using a mix of IPv4 and IPv6 addresses for OAM, signaling, and replication.
Steps:
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 36 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 100% of the rated load of the MPEs
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.



The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-130: R-19526026-130 The policy systems described in Sections 2.4.3, 2.4.4,
2.4.5, and 2.4.6 while running
R-19526026-200: R-19526026-200 Reference Architecture B-1 shall be supported with
the following minimum physical har
R-19526026-205: R-19526026-205 The Policy DL460 G8 blade configuration used for
testing MPE and MRA shall include ad
R-19526026-251: R-19526026-251 The general traffic model described in [R-19526026230] shall be repeated on hardware
R-19526026-100: R-19526026-100 The Policy Component capacities shown in Table 2
shall be verified for each server t
Keywords:
None
10.1.14 TEST CASE CAM-97266: MIXED CALL FLOW AT 100% USING IPV4 IN CALL FLOW
WITH TOPOLOGY HIDING
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-1 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 100% of the capacity of the MPEs. For this test case both
the Gx and Rx topology hiding features will be active.
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Enable Gx and Rx topology hiding on the MRA
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 37 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
3. Prior to running the test, reset all counters on the MRA and MPEs
4. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 100% of the rated load of the MPEs
5. The traffic will run for 2 hours
6. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.



The MRA and MPEs should process 100% of the calls generated by the traffic generator, with topology
hiding for Gx and Rx active on the MRA.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-200: R-19526026-200 Reference Architecture B-1 shall be supported with
the following minimum physical har
R-19526026-205: R-19526026-205 The Policy DL460 G8 blade configuration used for
testing MPE and MRA shall include ad
R-19526026-210: R-19526026-210 Reference Architecture B-1 shall support the
following performance figure using BL 46
R-19526026-230: R-19526026-230 The mixed call flow model as defined in Section
2.3.14 will be run on hardware conf
Keywords:
Performance
ANCHORAGE
10.1.15 TEST CASE CAM-94648: MIXED CALL FLOW AT 100% USING IPV4 IN CALL FLOW
USING SCTP TRANSPORT
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-1 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 100% of the capacity of the MPEs. The transport used for
this test will be SCTP.
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 100% of the rated load of the MPEs
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 38 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.



The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-130: R-19526026-130 The policy systems described in Sections 2.4.3, 2.4.4,
2.4.5, and 2.4.6 while running
R-19526026-200: R-19526026-200 Reference Architecture B-1 shall be supported with
the following minimum physical har
R-19526026-205: R-19526026-205 The Policy DL460 G8 blade configuration used for
testing MPE and MRA shall include ad
R-19526026-240: R-19526026-240 The general traffic model described in [R-19526026230] shall be repeated on hardware
R-19526026-252: R-19526026-252 The general traffic model described in [R-19526026230] shall be repeated on hardware
R-19526026-100: R-19526026-100 The Policy Component capacities shown in Table 2
shall be verified for each server t
Keywords:
Performance
ANCHORAGE
10.1.16 TEST CASE CAM-95432: BL460 MPE WITH 64GB RAM CAN SUPPORT 6.5 MILLION
CONCURRENT SESSIONS IN B-1 TOPOLOGY
Author:
ppiacentini
Summary:
This test will verify that the HP BL460 C-Class MPE will be able to support up to 6.5 million active concurrent
sessions and still provide the expected level of service.
Steps:
1. Seagull will be used for the traffic generation. Traffic will be sent directly to an MPE cluster
2. Reboot the MPE cluster that will be used in this chassis to insure a clean setup to start the test
3. Two test drivers will be used for this test
4. The call flow used will be the call flow that was used for the performance tests with an SPR server for profile
lookup
5. Between the 2 test drivers, run seagull scripts to generate Gx, Sd, and Rx sessions on the MPE, and calculate
the hold time such that for a 1 hour period, there are a constant 6.5M active sessions, with the MPE's TPS at the
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 39 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
max rate of 6000
6. While the test is running, capture memory, CPU, and queue statistics from the MPE for future reference
7. While the test is running, monitor the KPI statistics screen for any unusual and unexpected issues on the MPE
Expected Results:


The HP BL460 C-Class MPE with 64GB RAM should be able to run normally with 6.5 million
concurrent Gx, Sd, and Rx sessions.
There should be no exceptions seen in the logs, or no abnormalities as seen in the KPI monitor screens
Requirements
R-19526026-100: R-19526026-100 The Policy Component capacities shown in Table 2
shall be verified for each server t
Keywords:
Performance
ANCHORAGE
10.1.17 TEST CASE CAM-95433: BL460 MRA WITH 64GB RAM CAN SUPPORT UP TO 52,000
TRANSACTIONS/SECOND IN B-1 TOPOLOGY
Author:
ppiacentini
Summary:
This test is to verify that the Oracle MRA running on an HP BL460 C-Class blade can support up to 52,000
transactions per second.when configured in the Reference Architechture B-1 topology.
Test setup: (1) CMP cluster, (2) MRA clusters (2 sites), (10) MPE clusters (some simulated)
1.
The MRA is configured for stateful routing, and should be evenly distributing the messages among the 10
MPES
2.
The MPEs have Validation disabled, and there are no Data Sources enabled
3.
The MPEs are configured to index by E164 address
4.
All logging on all systems is set to WARN level
Steps:
1. Reset all MRAs and MPES to insure a consistent start of test
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 40 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
2. Set up multiple seagull test drivers so that each instance of seagull uses a unique range of subscribers
3. Configure the seagull test drivers to send the following message flow: CCR-Initial, CCR-Update, CCRTerminate.
5. The traffic should run at steady-state rate of 52,000 for two hours
6. The logs should be monitored for any exceptions of unusual errors while the test is running
7. The KPI statistics screen will be monitored for any unexpected issues while the test is running
8. Latency statistics are captured for all of the calls
Expected Results:

The C-Class BL460 MRA should be able to support 52,000 tps total rate for a sustained period of time,
while meeting the goal of 92% of the latency responses under 100ms and 99.9% of the latency
responses under 200ms.
Requirements
R-19526026-210: R-19526026-210 Reference Architecture B-1 shall support the
following performance figure using BL 46
Keywords:
Performance
ANCHORAGE
10.1.18 TEST CASE CAM-95619: BL460 MRA WITH 64GB RAM CAN SUPPORT UP TO 20M
BINDINGS IN B-1 TOPOLOGY
Author:
ppiacentini
Summary:
This test is to verify that the Oracle MRA running on an HP BL460 C-Class blade can support up to 20M
bindings in the Reference Architecture B-1 topology
Test setup: (1) CMP cluster, (2) MRA clusters (2 sites), (10) MPE clusters
1.
The MRA is configured for stateful routing, and should be evenly distributing the messages among the 10
MPES
2.
The MPEs have Validation disabled, and there are no Data Sources enabled
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 41 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
3.
The MPEs are configured to index by E164 address
4.
All logging on all systems is set to WARN level
Steps:
1. Reset all MRAs and MPES to insure a consistent start of test
2. Set up multiple seagull test drivers so that each instance of seagull uses a unique range of subscribers
3. Configure the seagull test drivers to send the following message flow: CCR-Initial, CCR-Update, CCRTerminate.
5. Generate enough sessions, using 20M unique Subscription-Ids, such that the MRA will contain 20M active
bindings
6. Send a moderate amount of steady state traffic to the MRA containing the 20M bindings for a period of one
hour
7. The logs should be monitored for any exceptions of unusual errors while the test is running
8. The KPI statistics screen will be monitored for any unexpected issues while the test is running. CPU and
memory usage will be captured
9. Latency of the calls made during steady state period will be monitored
Expected Results:

The C-Class BL460 MRA with 64GB RAM should be able to support having 20M active bindings
while handling steady state traffic with no system or forwarding problems seen.
Requirements
R-19526026-100: R-19526026-100 The Policy Component capacities shown in Table 2
shall be verified for each server t
Keywords:
None
10.1.19 TEST CASE CAM-94646: LONGEVITY TEST - MIXED CALL FLOW AT 100% USING IPV4
IN CALL FLOW
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-1 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 100% of the capacity of the MPEs over an extended
period of time.
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 42 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Session cleanup will be enabled for the duration of this test
4. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 100% of the rated load of the MPEs
5. The traffic will run for 72 hours without disruption
6. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.



The MRA and MPE should process 100% of the calls generated by the traffic generator for the entire 72
hour period.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-130: R-19526026-130 The policy systems described in Sections 2.4.3, 2.4.4,
2.4.5, and 2.4.6 while running
R-19526026-200: R-19526026-200 Reference Architecture B-1 shall be supported with
the following minimum physical har
R-19526026-205: R-19526026-205 The Policy DL460 G8 blade configuration used for
testing MPE and MRA shall include ad
R-19526026-230: R-19526026-230 The mixed call flow model as defined in Section
2.3.14 will be run on hardware conf
R-19526026-252: R-19526026-252 The general traffic model described in [R-19526026230] shall be repeated on hardware
R-19526026-260: R-19526026-260 The mixed call model defined in Section 2.3.14 shall
be repeated on hardware configu
R-19526026-100: R-19526026-100 The Policy Component capacities shown in Table 2
shall be verified for each server t
Keywords:
Performance
ANCHORAGE
10.1.20 TEST CASE CAM-95439: CMP CAN SUPPORT MANAGING UP TO 50 MPES/MRA GEOREDUNDANT CLUSTERS
Author:
ppiacentini
Summary:
This test case is to verify that the CMP can manage up to 50 MRAs and MPEs. All MRAs and MPEs are GeoRedundant clusters
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 43 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
Steps:
1. On the CMP, configure 50 MRAs and MPEs. This can be a combination of MRA and MPEs
2. Monitor the logs on the CMP for any unexpected errors while adding the MRAs and MPEs
3. In the CMP, go to the Policy Server/Configuration screen and verify that all the MPEs are showing up. If
MRAs are configured verify that they show up in the MRA/Configuration screen
4. In the CMP, go into the statistics screens of several of the MRAs and MPES and verify that the statistics are
updating
5. Go into the KPI Dashboard screen and verify that all MRAs and MPEs are represented, and the CMP shows
statistics for each one
Expected Results:

The CMP can manage up to 50 MRA and MPE Geo-Redundant Clusters
Requirements
R-19526026-120: R-19526026-120 A S-CMP shall be able to manage and monitor a
policy system of up to 50 MPE and MRA
Keywords:
Performance
ANCHORAGE
10.1.21 TEST CASE CAM-95437: BL460 MRA WITH 64GB RAM CAN SUPPORT UP TO 8K CLIENT
CONNECTIONS
Author:
ppiacentini
Summary:
This test is to verify that the Oracle MRA running on an HP C-Class BL460 blade with 64GB RAM can support
up to 8k simultaneous and active Diameter client connections.
Test setup: (1) CMP cluster, (1) MRA cluster (1) SDM cluster (10) MPE clusters
1.
All connections from test drivers will terminate on the MRA
2.
All traffic from the test drivers will be sent to the MRA over the Diameter connections
3.
The MPEs are configured in Gx quota mode
4.
The MPEs have Validation enabled, and has an SPR Data Source
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 44 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
5.
The MPE is configured to index by E164 address
6.
The test driver will use the same set of E164 addresses that are configured in the SPR
7. 10k unique Network Elements are defined in the CMP, and the MPEs are all associated with the 10k Network
Elements
Steps:
1. Set up the MRA so that it routes the Diameter traffic equally to the MPEs
2. Set up a seagull test driver so that it will generate sessions with the following call flow: Gx-Init, GxTerminate. Alternatively, diamcli's test rate could be used:
3. Set up the seagull scenario file so that it will bring up a total of 8k connections to the MRA. This will
probably have to be spread out among several seagull test drivers
4. Set up the seagull script so that each simulated GGSN will send a small amount of traffic, say 10 CCRs/sec
5. Generate the traffic from all 10k clients for a period of 2 hours to the MRA/MPEs
6. Monitor the CPU, memory, and other statistics on the MRA and MPEs while the traffic is running
7. Check the KPI statistics and verify the connection count is correct, and the TPS to each MRA and MPE is
accurate
8. Monitor the logs on the MRA and MPEs for any exceptions or other types of errors
Sample test-rate command
test rate gx -maxtransactions=100 -reportevery=1 -duration=3600 -delay=500 -transactionPool=10 numclients=1 -transactions=1 -peerCount=100 -identity=ggsn-1,ggsn-2,ggsn-3,ggsn-4,ggsn-5,ggsn-6,ggsn7,ggsn-8,ggsn-9,ggsn-10,ggsn-11,ggsn-12,ggsn-13,ggsn-14,ggsn-15,ggsn-16,ggsn-17,ggsn-18,ggsn-19,ggsn20,ggsn-21,ggsn-22,ggsn-23,ggsn-24,ggsn-25,ggsn-26,ggsn-27,ggsn-28,ggsn-29,ggsn-30,ggsn-31,ggsn-32,ggsn33,ggsn-34,ggsn-35,ggsn-36,ggsn-37,ggsn-38,ggsn-39,ggsn-40,ggsn-41,ggsn-42,ggsn-43,ggsn-44,ggsn-45,ggsn46,ggsn-47,ggsn-48,ggsn-49,ggsn-50,ggsn-51,ggsn-52,ggsn-53,ggsn-54,ggsn-55,ggsn-56,ggsn-57,ggsn-58,ggsn59,ggsn-60,ggsn-61,ggsn-62,ggsn-63,ggsn-64,ggsn-65,ggsn-66,ggsn-67,ggsn-68,ggsn-69,ggsn-70,ggsn-71,ggsn72,ggsn-73,ggsn-74,ggsn-75,ggsn-76,ggsn-77,ggsn-78,ggsn-79,ggsn-80,ggsn-81,ggsn-82,ggsn-83,ggsn-84,ggsn85,ggsn-86,ggsn-87,ggsn-88,ggsn-89,ggsn-90,ggsn-91,ggsn-92,ggsn-93,ggsn-94,ggsn-95,ggsn-96,ggsn-97,ggsn98,ggsn-99,ggsn-100 -remoteidentity=mpe26-59.camiant.com -realm=camiant.com -hostname=10.15.25.159 addressrange=1.0.0.0/18 -terminate=true -E164userids=3611000000/1000000 -subidtypes=E164 -ipcantype=0 calledstationid=mycalledstationid
Expected Results:

The BL460 C-Class MRA should be able to handle Diameter messages from up to 8k Diameter client
connections simultaneously
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 45 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
Requirements
R-19526026-510b: R-19526026-510b Up to 8000 Diameter Connections shall be set up to
the MRA (stateful model) without
Keywords:
Performance
ANCHORAGE
10.1.22 TEST CASE CAM-97374: MRA WILL SUPPORT 16 DIAMETER CONNECTIONS FROM A
DRA
Author:
ppiacentini
Summary:
This test case is to verify that the MRA can support 16 connections from a DRA, with traffic being sent across
all 16 connections
Steps:
1. Configure a DRA to front-end an MRA containing 7 MPEs in its pool
2. Set up 16 connections between the DRA and the MRA
3. Run traffic at 100% load in this configuration for a period of two hours
4. The traffic will use the call model described in section 2.3.2 - LTE + Subscriber Quota (2 PDN connections),
and the call model described in section 2.3.8 - OCS Throttling
5. Host-based routing will be used to route traffic bask to the AFs
Expected Results:

The MRA should be able to support traffic coming into it via 16 connections from a DRA.
Requirements
R-19526026-500b: R-19526026-500b Up to 16 Diameter Connections shall be set up
between the DRA and MRA (stateful mode
Keywords:
Performance
ANCHORAGE
10.1.23 TEST CASE CAM-98672: BASELINE STEP TEST - BL460-G8 MPE
Author:
ppiacentini
Summary:
This test is to get a baseline the basic code of the release, using a BL460-G8, with only Gx being used, and no
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 46 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
other Diameter interfaces. There is no validation, user lookup, policies being used. The only configuration on
the MPE will be the Diameter Identity, realm, and Network Element used in the test. This should allow more
direct apples-apples comparison between releases.
The call flow will be simple Gx CCR-Initial, wait 10 secs, Gx CCR-Update, wait 10 secs, Gx CCR-Terminate
Steps:
1. Configure a Diameter Identity and Realm on the MPE
2. Add the Network Elements that will be needed to run the test
3. Set up a seagull script that will send a Gx CCR-I, wait 10 secs, CCR-U, wait 10 secs, CCR-T
4. Send traffic to the MPE at a low level (ie, 200 calls/sec) which will generate 600 tps on the MRA.
5. Traffic will run for 15 minutes, script will capture protocol stats, CPU and Memory usage
6. Continually increase the traffic, running for 15 minutes at each level, until it reaches 200% of the rated load of
the MPE



The BL460-G8 MPE should handle all traffic up to 100% of its rated load without any problems
There may be some load shedding seen as the traffic increases from 100% to 200% of the rated load.
The latency should be within the guidelines specified in [R-19526026-130] for up to 100% load
Requirements
None
Keywords:
Performance
ANCHORAGE
10.1.24 TEST CASE CAM-98673: BASELINE STEP TEST - BL460-G8 MRA
Author:
ppiacentini
Summary:
This test is to get a baseline the basic code of the release, using a BL460-G8, with only Gx being used, and no
other Diameter interfaces. There is no validation, user lookup, policies being used. The only configuration on
the MRA will be the Diameter Identity, realm, and Network Element used in the test. This should allow more
direct apples-apples comparison between releases.
The call flow will be simple Gx CCR-Initial, wait 10 secs, Gx CCR-Update, wait 10 secs, Gx CCR-Terminate
Steps:
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 47 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
1. Configure a Diameter Identity and Realm on the MRA
2. Add the Network Elements that will be needed to run the test
3. The MRA will need at least 10 MPEs in its MPE pool, real or simulated
4. Set up a seagull script that will send a Gx CCR-I, wait 10 secs, CCR-U, wait 10 secs, CCR-T
5. Send traffic to the MPE at a low level (ie, 1500 calls/sec) which will generate 4500 tps on the MRA.
6. Traffic will run for 15 minutes, script will capture protocol stats, CPU and Memory usage
7. Continually increase the traffic and run at each level for 15 minutes until it reaches 200% of the rated load of
the MPE



The BL460-G8 MRA should handle all traffic up to 100% of its rated load without any problems
There may be some load shedding seen as the traffic increases from 100% to 200% of the rated load.
The latency should be within the guidelines specified in [R-19526026-130] for up to 100% load
Requirements
None
Keywords:
Performance
ANCHORAGE
10.1.25 TEST CASE CAM-99004: BASELINE STEP TEST - BL460-G9 MPE
Author:
ppiacentini
Summary:
This test is to get a baseline of the basic code of the release, running on a BL460-G9, with only Gx being used,
and no other Diameter interfaces. There is no validation, user lookup, policies being used. The only
configuration on the MPE will be the Diameter Identity, realm, and Network Element used in the test. This
should allow more direct apples-apples comparison between releases.
The call flow will be simple Gx CCR-Initial, wait 10 secs, Gx CCR-Update, wait 10 secs, Gx CCR-Terminate
Steps:
1. Configure a Diameter Identity and Realm on the MPE
2. Add the Network Elements that will be needed to run the test
3. Set up a seagull script that will send a Gx CCR-I, wait 10 secs, CCR-U, wait 10 secs, CCR-T
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 48 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
4. Send traffic to the MPE at a low level (ie, 200 calls/sec) which will generate 600 tps on the MRA.
5. Traffic will run for 15 minutes, script will capture protocol stats, CPU and Memory usage
6. Continually increase the traffic, running for 15 minutes at each level, until it reaches 200% of the rated load of
the MPE



The G9 MPE should handle all traffic up to 100% of its rated load without any problems
There may be some load shedding seen as the traffic increases from 100% to 200% of the rated load.
The latency should be within the guidelines specified in [R-19526026-130] for up to 100% load
Requirements
None
Keywords:
Performance
ANCHORAGE
10.1.26 TEST CASE CAM-99005: BASELINE STEP TEST - BL460-G9 MRA
Author:
ppiacentini
Summary:
This test is to get a baseline the basic code of the release, using a BL460-G9, with only Gx being used, and no
other Diameter interfaces. There is no validation, user lookup, policies being used. The only configuration on
the MRA will be the Diameter Identity, realm, and Network Element used in the test. This should allow more
direct apples-apples comparison between releases.
The call flow will be simple Gx CCR-Initial, wait 10 secs, Gx CCR-Update, wait 10 secs, Gx CCR-Terminate
Steps:
1. Configure a Diameter Identity and Realm on the MRA
2. Add the Network Elements that will be needed to run the test
3. The MRA will need at least 10 MPEs in its MPE pool, real or simulated
4. Set up a seagull script that will send a Gx CCR-I, wait 10 secs, CCR-U, wait 10 secs, CCR-T
5. Send traffic to the MPE at a low level (ie, 1500 calls/sec) which will generate 4500 tps on the MRA.
6. Traffic will run for 15 minutes, script will capture protocol stats, CPU and Memory usage
7. Continually increase the traffic and run at each level for 15 minutes until it reaches 200% of the rated load of
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 49 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
the MPE



The BL460-G9 MRA should handle all traffic up to 100% of its rated load without any problems
There may be some load shedding seen as the traffic increases from 100% to 200% of the rated load.
The latency should be within the guidelines specified in [R-19526026-130] for up to 100% load
Requirements
None
Keywords:
Performance
ANCHORAGE
10.1.27 TEST CASE CAM-98677: MIXED CALL FLOW AT 100% LOADS WITH OVER 1K
POLICIES DEPLOYED
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-1 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 100% of the capacity of the MPEs, with the addition of at
least 1000 additional policies beying deployed, above the number needed to support the mixed traffic flows.
It is expected that the additional 1000 policies will be deployed, but not triggered.
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. In addition to the policies being used for the mixed call flows, deploy at least 1000 additional policies, taken
from the field if possible
4. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 100% of the rated load of the MPEs
5. The traffic will run for 2 hours
6. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.


The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 50 of 94
PROPRIETARY INFORMATION

INTERNAL USE ONLY
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
None
Keywords:
Performance
ANCHORAGE
10.1.28 TEST CASE CAM-98702: 2-SITE MRA AT 100% LOAD IN LEGACY MODE
Author:
ppiacentini
Summary:
This test is to verify that the 2 MRA sites in the Reference Architecture B-1 system can process loads equal to
100% of their capacity when configured as Primary and Backup in the Legacy mode
Steps:
1. Two MRA sites are set up, with enough real or simulated MPEs in their pool to support a traffic load of 100%
sent to the MRAs
2. The MRA association will be configured in "legacy" mode
3. Send traffic equal to 100% of the rated load to each of the two MRAs. A small amount of traffic should be
routed between them
4. Traffic will run for a period of two hours
5. Monitor the CPU and Memory loads of the two MRAs during the test



The two MRA sites should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
None
Keywords:
Performance
ANCHORAGE
10.1.29 TEST CASE CAM-98703: 2-SITE MRA AT 100% LOAD IN ALGOV1 MODE
Author:
ppiacentini
Summary:
This test is to verify that the 2 MRA sites in the Reference Architecture B-1 system can process loads equal to
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 51 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
100% of their capacity when configured as Primary and Backup in the Algov1 mode
Steps:
1. Two MRA sites are set up, with enough real or simulated MPEs in their pool to support a traffic load of 100%
sent to the MRAs
2. The MRA association will be configured in "Algov1" mode
3. Send traffic equal to 100% of the rated load to each of the two MRAs. A small amount of traffic should be
routed between them
4. Traffic will run for a perod of two hours
5. Monitor the CPU and Memory loads of the two MRAs during the test



The two MRA sites should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
None
Keywords:
Performance
ANCHORAGE
10.2 TEST SUITE : REFERENCE ARCHITECTURE B-2
Reference Architecture B-2 is an RMS based CMP, along with a full C-class chassis, consisting of one MRA cluster, and 7
MPE clusters.
10.2.1
TEST CASE CAM-95618: BL460 MRA WITH 64GB RAM CAN SUPPORT UP TO 52,000
TRANSACTIONS/SECOND IN B-2 TOPOLOGY
Author:
ppiacentini
Summary:
This test is to verify that the Oracle MRA running on an HP BL460 C-Class blade can support up to 52,000
transactions per second.when configured in the Reference Architechture B-2 topology.
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 52 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
Test setup: (1) CMP cluster, (1) MRA cluster1, (10) MPE clusters (some simulated)
1.
The MRA is configured for stateful routing, and should be evenly distributing the messages among the 10
MPES
2.
The MPEs have Validation disabled, and there are no Data Sources enabled
3.
The MPEs are configured to index by E164 address
4.
All logging on all systems is set to WARN level
Steps:
1. Reset all MRAs and MPES to insure a consistent start of test
2. Set up mutliple seagull test drivers so that each instance of seagull uses a unique range of subscribers
3. Configure the seagull test drivers to send the following message flow: CCR-Initial, CCR-Update, CCRTerminate.
4. The traffic should run at steady-state rate of 52,000 for two hours
5. The logs should be monitored for any exceptions of unusual errors while the test is running
6. The KPI statistics screen will be monitored for any unexpected issues while the test is running
7. Latency statistics are captured for all of the calls
Expected Results:

The C-Class BL460 MRA should be able to support 52,000 tps total rate for a sustained period of time,
while meeting the goal of 92% of the latency responses under 100ms and 99.9% of the latency
responses under 200ms.
Requirements
R-19526026-300: R-19526026-300 Reference Architecture B-2 shall be supported with
the following minimum physical har
R-19526026-305: R-19526026-305 The Policy DL460 G8 blade configuration used for
testing MPE and MRA shall include ad
R-19526026-310: R-19526026-310 Reference Architecture B-2 shall support the
following performance figure using BL 4
Keywords:
Performance
ANCHORAGE
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 53 of 94
PROPRIETARY INFORMATION
10.2.2
INTERNAL USE ONLY
TEST CASE CAM-95435: BL460 MRA WITH 64GB RAM CAN SUPPORT UP TO 20M
BINDINGS IN B-2 TOPOLOGY
Author:
ppiacentini
Summary:
This test is to verify that the Oracle MRA running on an HP BL460 C-Class blade can support up to 20M
bindings in the Reference Architecture B-2 topology
Test setup: (1) CMP cluster, (1) MRA cluster , (10) MPE clusters
1.
The MRA is configured for stateful routing, and should be evenly distributing the messages among the 10
MPES
2.
The MPEs have Validation disabled, and there are no Data Sources enabled
3.
The MPEs are configured to index by E164 address
4.
All logging on all systems is set to WARN level
Steps:
1. Reset all MRAs and MPES to insure a consistent start of test
2. Set up mutliple seagull test drivers so that each instance of seagull uses a unique range of subscribers
3. Configure the seagull test drivers to send the following message flow: CCR-Initial, CCR-Update, CCRTerminate.
5. Generate enough sessions, using 20M unique Subscription-Ids, such that the MRA will contain 20M active
bindings
6. Send a moderate amount of steady state traffic to the MRA containing the 20M bindings for a period of one
hour
7. The logs should be monitored for any exceptions of unusual errors while the test is running
8. The KPI statistics screen will be monitored for any unexpected issues while the test is running. CPU and
memory usage will be captured
9. Latency of the calls made during steady state period will be monitored
Expected Results:

The C-Class BL460 MRA with 64GB RAM should be able to support having 20M active bindings
while handling steady state traffic with no system or forwarding problems seen.
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 54 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
Requirements
R-19526026-300: R-19526026-300 Reference Architecture B-2 shall be supported with
the following minimum physical har
R-19526026-305: R-19526026-305 The Policy DL460 G8 blade configuration used for
testing MPE and MRA shall include ad
R-19526026-310: R-19526026-310 Reference Architecture B-2 shall support the
following performance figure using BL 4
Keywords:
Performance
ANCHORAGE
10.3 TEST SUITE : REFERENCE ARCHITECTURE B-3
Reference Architecture B-3 consists of a CMP cluster, an MRA cluster, and 2 MPE clusters. All servers are Sun Netra X5-2
Rackmount servers
10.3.1
TEST CASE CAM-94649: MIXED CALL FLOW AT 80% USING IPV4 IN CALL FLOW
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-3 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 80% of the capacity of the MPEs.
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 80% of the rated load of the MPEs
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.
Expected Results:



The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-400: R-19526026-400 Reference Architecture B-3 shall be supported with
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 55 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
the following minimum physical Co
R-19526026-410: R-19526026-410 Reference Architecture B-3 shall support the
following performance figures for Sun X5
Keywords:
10.3.2
Performance
ANCHORAGE
TEST CASE CAM-94650: MIXED CALL FLOW AT 100% USING IPV4 IN CALL FLOW
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-3 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 100% of the capacity of the MPEs.
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 100% of the rated load of the MPEs
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.



The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-400: R-19526026-400 Reference Architecture B-3 shall be supported with
the following minimum physical Co
R-19526026-410: R-19526026-410 Reference Architecture B-3 shall support the
following performance figures for Sun X5
Keywords:
Performance
ANCHORAGE
10.3.3
TEST CASE CAM-94651: MIXED CALL FLOW AT 120% USING IPV4 IN CALL FLOW
Author:
ppiacentini
Summary:
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 56 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
This test is to verify that the Reference Architecture B-3 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 120% of the capacity of the MPEs.
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 120% of the rated load of the MPEs
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.
Expected Results:



The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-400: R-19526026-400 Reference Architecture B-3 shall be supported with
the following minimum physical Co
R-19526026-410: R-19526026-410 Reference Architecture B-3 shall support the
following performance figures for Sun X5
Keywords:
Performance
ANCHORAGE
10.3.4
TEST CASE CAM-94652: MIXED CALL FLOW AT 150% USING IPV4 IN CALL FLOW
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-3 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 150% of the capacity of the MPEs.
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 150% of the rated load of the MPEs
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 57 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.
6. Verify that if load shedding kicks in, the correct levels and discards are used
Expected Results:



The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-400: R-19526026-400 Reference Architecture B-3 shall be supported with
the following minimum physical Co
R-19526026-410: R-19526026-410 Reference Architecture B-3 shall support the
following performance figures for Sun X5
Keywords:
Performance
ANCHORAGE
10.3.5
TEST CASE CAM-94653: MIXED CALL FLOW AT 200% USING IPV4 IN CALL FLOW
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-3 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 200% of the capacity of the MPEs.
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 200% of the rated load of the MPEs
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.
6. Verify that if load shedding kicks in, the correct levels and discards are used
Expected Results:

The MRA and MPE should process 100% of the calls generated by the traffic generator.
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 58 of 94
PROPRIETARY INFORMATION


INTERNAL USE ONLY
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-400: R-19526026-400 Reference Architecture B-3 shall be supported with
the following minimum physical Co
R-19526026-410: R-19526026-410 Reference Architecture B-3 shall support the
following performance figures for Sun X5
Keywords:
Performance
ANCHORAGE
10.3.6
TEST CASE CAM-94654: LONGEVITY TEST - MIXED CALL FLOW AT 100% USING IPV4
IN CALL FLOW
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-3 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 100% of the capacity of the MPEs over an extended
period of time.
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 100% of the rated load of the MPEs
4. The traffic will run for 72 hours without disruption
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.



The MRA and MPE should process 100% of the calls generated by the traffic generator for the entire 72
hour period.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-400: R-19526026-400 Reference Architecture B-3 shall be supported with
the following minimum physical Co
R-19526026-410: R-19526026-410 Reference Architecture B-3 shall support the
following performance figures for Sun X5
Keywords:
Performance
ANCHORAGE
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 59 of 94
PROPRIETARY INFORMATION
10.3.7
INTERNAL USE ONLY
TEST CASE CAM-94655: MIXED CALL FLOW AT 100% USING IPV4 IN CALL FLOW
USING SCTP TRANSPORT
Author:
ppiacentini
Summary:
This test is to verify that the Reference Architecture B-3 system can process the requests generated by the mixed
call flows as defined in Appendix-A at a load equal to 100% of the capacity of the MPEs. The transport used for
this test will be SCTP.
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the mixed call flows, so that the aggregate load generated by the full call cycle will
equal 100% of the rated load of the MPEs
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.



The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-400: R-19526026-400 Reference Architecture B-3 shall be supported with
the following minimum physical Co
R-19526026-410: R-19526026-410 Reference Architecture B-3 shall support the
following performance figures for Sun X5
Keywords:
Performance
ANCHORAGE
10.3.8
TEST CASE CAM-95434: ORACLE X5-2 MRA CAN SUPPORT UP TO 50,000
TRANSACTIONS/SECOND
Author:
ppiacentini
Summary:
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 60 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
This test is to verify that the Oracle MRA running on an X5-2 can support up to 50,000 transactions per second.
Test setup: (1) CMP cluster, (1) MRA cluster, (10) MPE clusters (some simulated)
1.
The MRA is configured for stateful routing, and should be evenly distributing the messages among the 10
MPES
2.
The MPEs have Validation disabled, and there are no Data Sources enabled
3.
The MPEs are configured to index by E164 address
4.
All logging on all systems is set to WARN level
Steps:
1. Reset all MRAs and MPES to insure a consistent start of test
2. Set up multipleseagull test drivers so that each instance of seagull uses a unique range of subscribers
3. Configure the seagull test drivers to send the following message flow: CCR-Initial, CCR-Update, CCRTerminate.
5. The traffic should run at steady-state rate of 50,000 for an hour
6. The logs should be monitored for any exceptions of unusual errors while the test is running
7. The KPI statistics screen will be monitored for any unexpected issues while the test is running
8. At the conclusion of the test, the data_grinder.sh script will be used to trim off the ramp-up and ramp-down
information, and calculate the latencies and percentiles using only the information from the steady-state period.
The time to trim from the results would be 2 times the hold time (180 seconds)
Expected Results:

The Oracle X5-2 MRA should be able to support 50,000 tps total rate for a sustained period of time,
while meeting the goal of 92% of the latency responses under 100ms and 99.9% of the latency
responses under 200ms.
Requirements
R-19526026-400: R-19526026-400 Reference Architecture B-3 shall be supported with
the following minimum physical Co
R-19526026-410: R-19526026-410 Reference Architecture B-3 shall support the
following performance figures for Sun X5
Keywords:
Performance
ANCHORAGE
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 61 of 94
PROPRIETARY INFORMATION
10.3.9
INTERNAL USE ONLY
TEST CASE CAM-95431: ORACLE X5-2 MPE CAN SUPPORT 10 MILLION CONCURRENT
SESSIONS
Author:
ppiacentini
Summary:
This test will verify that the Oracle X5-2 MPE will be able to support up to 10 million active concurrent sessions
and still provide the expected level of service. The sessions will be a mix of Gx, Rx, and Sd sessions.
Steps:
1. Seagull will be used for the traffic generation. Traffic will be sent directly to an MPE cluster
2. Reboot the MPE cluster that will be used in this chassis to insure a clean setup to start the test
3. Two test drivers will be used for this test
4. The call flow used will be the call flow that was used for the performance tests with an SPR server for profile
lookup
5. Between the 2 test drivers, run seagull scripts to generate Gx, Sd, and Rx sessions on the MPE, and calculate
the hold time such that for a 1 hour period, there are a constant 10M active sessions, with the MPE's TPS at the
max rate of 6000
6. While the test is running, capture memory, CPU, and queue statistics from the MPE in a CSV file for future
reference
7. While the test is running, monitor the KPI statistics screen for any unusual and unexpected issues on the MPE
Expected Results:


The Oracle X5-2 MPE should be able to run normally with 10 million concurrent Gx sessions.
There should be no exceptions seen in the logs, or no abnormalities as seen in the KPI monitor screens
Requirements
R-19526026-400: R-19526026-400 Reference Architecture B-3 shall be supported with
the following minimum physical Co
R-19526026-410: R-19526026-410 Reference Architecture B-3 shall support the
following performance figures for Sun X5
Keywords:
Performance
ANCHORAGE
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 62 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
10.3.10 TEST CASE CAM-95440: ORACLE X5-2 MRA CAN SUPPORT UP TO 40M BINDINGS
Author:
ppiacentini
Summary:
This test is to verify that the Oracle MRA running on a X352 server can support up to 40M bindings
Test setup: (1) CMP cluster, (1) MRA cluster, (10) MPE clusters (some simulated)
1.
The MRA is configured for stateful routing, and should be evenly distributing the messages among the 10
MPES
2.
The MPEs have Validation disabled, and there are no Data Sources enabled
3.
The MPEs are configured to index by E164 address
4.
All logging on all systems is set to WARN level
Steps:
1. Reset all MRAs and MPES to insure a consistent start of test
2. Set up mutliple seagull test drivers so that each instance of seagull uses a unique range of subscribers
3. Configure the seagull test drivers to send the following message flow: CCR-Initial, CCR-Update, CCRTerminate.
5. Generate enough sessions, using 20M unique Subscription-Ids, such that the MRA will contain 20M active
bindings
6. Send a moderate amount of steady state traffic to the MRA containing the 20M bindings for a period of one
hour
7. The logs should be monitored for any exceptions of unusual errors while the test is running
8. The KPI statistics screen will be monitored for any unexpected issues while the test is running. CPU and
memory usage will be captured
9. Latency of the calls made during steady state period will be monitored
Expected Results:

The Oracle X5-2 MRA should be able to support having 40M active bindings while handling steady
state traffic with no system or forwarding problems seen.
Requirements
R-19526026-400: R-19526026-400 Reference Architecture B-3 shall be supported with
the following minimum physical Co
R-19526026-410: R-19526026-410 Reference Architecture B-3 shall support the
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 63 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
following performance figures for Sun X5
Keywords:
Performance
ANCHORAGE
10.3.11 TEST CASE CAM-95438: ORACLE X5-2 MRA CAN SUPPORT UP TO 8K CLIENT
CONNECTIONS
Author:
ppiacentini
Summary:
This test is to verify that the Oracle MRA running on a X5-2 can support up to 8k simultaneous and active
Diameter client connections.
Test setup: (1) CMP cluster, (1) MRA cluster (1) SDM cluster (10) MPE clusters (some simulated)
1.
All connections from test drivers will terminate on the MRA
2.
All traffic from the test drivers will be sent to the MRA over the Diameter connections
3.
The MPEs are configured in Gx quota mode
4.
The MPEs have Validation enabled, and has an SPR Data Source
5.
The MPE is configured to index by E164 address
6.
The test driver will use the same set of E164 addresses that are configured in the SPR
7.
1024 Unique Network Elements are defined in the CMP, and the MPEs are all associated with the 1024
unique Network Elements
Steps:
1. Set up the MRA so that it routes the Diameter traffic equally to the MPEs
2. Set up a seagull test driver so that it will generate sessions with the following call flow: Gx-Init, GxTerminate. Alternatively, diamcli's test rate could be used:
3. Set up the seagull scenario file so that it will bring up a total of 8k connections to the MRA. This will
probably have to be spread out among several seagull test drivers
4. Set up the seagull script so that each simulated GGSN will send a small amount of traffic, say 10 CCRs/sec
5. Generate the traffic from all 10k clients for a period of 2 hours to the MRA/MPEs
6. Monitor the CPU, memory, and other statistics on the MRA and MPEs while the traffic is running
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 64 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
7. Check the KPI statistics and verify the connection count is correct, and the TPS to each MRA and MPE is
accurate
8. Monitor the logs on the MRA and MPEs for any exceptions or other types of errors
Sample test-rate command
test rate gx -maxtransactions=100 -reportevery=1 -duration=3600 -delay=500 -transactionPool=10 numclients=1 -transactions=1 -peerCount=100 -identity=ggsn-1,ggsn-2,ggsn-3,ggsn-4,ggsn-5,ggsn-6,ggsn7,ggsn-8,ggsn-9,ggsn-10,ggsn-11,ggsn-12,ggsn-13,ggsn-14,ggsn-15,ggsn-16,ggsn-17,ggsn-18,ggsn-19,ggsn20,ggsn-21,ggsn-22,ggsn-23,ggsn-24,ggsn-25,ggsn-26,ggsn-27,ggsn-28,ggsn-29,ggsn-30,ggsn-31,ggsn-32,ggsn33,ggsn-34,ggsn-35,ggsn-36,ggsn-37,ggsn-38,ggsn-39,ggsn-40,ggsn-41,ggsn-42,ggsn-43,ggsn-44,ggsn-45,ggsn46,ggsn-47,ggsn-48,ggsn-49,ggsn-50,ggsn-51,ggsn-52,ggsn-53,ggsn-54,ggsn-55,ggsn-56,ggsn-57,ggsn-58,ggsn59,ggsn-60,ggsn-61,ggsn-62,ggsn-63,ggsn-64,ggsn-65,ggsn-66,ggsn-67,ggsn-68,ggsn-69,ggsn-70,ggsn-71,ggsn72,ggsn-73,ggsn-74,ggsn-75,ggsn-76,ggsn-77,ggsn-78,ggsn-79,ggsn-80,ggsn-81,ggsn-82,ggsn-83,ggsn-84,ggsn85,ggsn-86,ggsn-87,ggsn-88,ggsn-89,ggsn-90,ggsn-91,ggsn-92,ggsn-93,ggsn-94,ggsn-95,ggsn-96,ggsn-97,ggsn98,ggsn-99,ggsn-100 -remoteidentity=mpe26-59.camiant.com -realm=camiant.com -hostname=10.15.25.159 addressrange=1.0.0.0/18 -terminate=true -E164userids=3611000000/1000000 -subidtypes=E164 -ipcantype=0 calledstationid=mycalledstationid
Expected Results:

The Oracle X5-2 MRA should be able to handle Diameter messages from up to 8k Diameter client
connections simultaneously
Requirements
R-19526026-400: R-19526026-400 Reference Architecture B-3 shall be supported with
the following minimum physical Co
R-19526026-410: R-19526026-410 Reference Architecture B-3 shall support the
following performance figures for Sun X5
Keywords:
Performance
ANCHORAGE
10.3.12 TEST CASE CAM-97382: VOLTE-PLUS-INBOUND-S9 ROAMING CALL FLOW AT 100%
Author:
ppiacentini
Summary:
Confirm that the Reference Architecture B-3 system can process the requests generated by the VoLTE-plusinbound-S9 call flows as defined in Appendix-A at a load equal to 100% of the capacity of the MPEs.
Prerequisites:


Reference Architecture B-3 Site configured as HPLMN: Global S9 configuration settings; Home and
Visited Match Lists; Roaming Profile; Policy Server S9 Settings; DEA configured for MRA(s).
SPR configured with IMSI-based Subscriber IDs
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 65 of 94
PROPRIETARY INFORMATION


INTERNAL USE ONLY
Another Site configured as VPLMN (originates S9 messages).
Gx CCR-i messages must be contain both IMSI- and MSISDN-based subscriber IDs.
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the VoLTE-plus-S9 call flows, so that the aggregate load generated by the full call
cycle will equal 100% of the rated load of the MPEs. Note the prerequisite that the Gx CCR-i messages contain
IMSI- and MSISDN-based subscriber IDs.
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.
Expected Results:



The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-130: R-19526026-130 The policy systems described in Sections 2.4.3, 2.4.4,
2.4.5, and 2.4.6 while running
R-19526026-200: R-19526026-200 Reference Architecture B-1 shall be supported with
the following minimum physical har
R-19526026-205: R-19526026-205 The Policy DL460 G8 blade configuration used for
testing MPE and MRA shall include ad
R-19526026-252: R-19526026-252 The general traffic model described in [R-19526026230] shall be repeated on hardware
R-19526026-100: R-19526026-100 The Policy Component capacities shown in Table 2
shall be verified for each server t
R-19526026-600: R-19526026-600 The following SPR/OCUDR Configurations and as
shown in Table 15 shall be verified usi
Keywords:
Performance
ANCHORAGE
10.3.13 TEST CASE CAM-97323: VOLTE-PLUS-OUTBOUND-S9 ROAMING CALL FLOW AT 100%
Author:
ppiacentini
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 66 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
Summary:
Confirm that the Reference Architecture B-3 system can process the requests generated by the VoLTE-plusoutbound-S9 call flows as defined in Appendix-A at a load equal to 100% of the capacity of the MPEs.
Prerequisites:



Reference Architecture B-3 Site configured as VPLMN: Global S9 configuration settings; Home and
Visited Match Lists; Roaming Profile; Policy Server S9 Settings; DEA configured for MRA(s).
Another Site configured as HPLMN (recipient for S9 messages).
Gx CCR-i messages must be contain both IMSI- and MSISDN-based subscriber IDs.
Steps:
1. Preload all MPEs with 1M Gx sessions, using a range of E164 Subscription-Ids that will not be used by the
scripts that are run in this test. Session cleanup is disabled
2. Prior to running the test, reset all counters on the MRA and MPEs
3. Start all scripts used for the VoLTE-plus-S9 call flows, so that the aggregate load generated by the full call
cycle will equal 100% of the rated load of the MPEs. Note the prerequisite that the Gx CCR-i messages contain
IMSI- and MSISDN-based subscriber IDs.
4. The traffic will run for 2 hours
5. Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory, call latencies,
average session size, and other KPIs will be recorded during the test.
Expected Results:



The MRA and MPE should process 100% of the calls generated by the traffic generator.
During the time of the test there should be no exceptions seen on the servers, and no unexpected error
or warning messages.
The latency should be within the guidelines specified in [R-19526026-130]
Requirements
R-19526026-130: R-19526026-130 The policy systems described in Sections 2.4.3, 2.4.4,
2.4.5, and 2.4.6 while running
R-19526026-200: R-19526026-200 Reference Architecture B-1 shall be supported with
the following minimum physical har
R-19526026-205: R-19526026-205 The Policy DL460 G8 blade configuration used for
testing MPE and MRA shall include ad
R-19526026-252: R-19526026-252 The general traffic model described in [R-19526026230] shall be repeated on hardware
R-19526026-100: R-19526026-100 The Policy Component capacities shown in Table 2
shall be verified for each server t
R-19526026-600: R-19526026-600 The following SPR/OCUDR Configurations and as
shown in Table 15 shall be verified usi
Keywords:
Performance
ANCHORAGE
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 67 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
10.4 TEST SUITE : REFERENCE ARCHITECTURE C-1
Reference Architecture C-1 will use the same Sun Netra X5-2 topology that is used in Reference Architecture B-3, except it
will be configured for Wireline services and the major protocol being used will be RADIUS.
For this set of tests, RADIUS sessions will be created on the Netra X5-2 MPEs behind a Netra X5-2 MRA.
RADIUS sessions will be used for 90% of the calls, and the remaining 10% will be Gx sessions
For 90% of the RADIUS sessions, the call flow will be Account-Start, wait for a period of 3 minutes, then Account-Stop.
For 10% of the RADIUS calls, there will be a mid-session update triggered by a usage threshold on the Sd session.
10.4.1
CAM-94630:RADIUS WITH SD AND SPR PROFILE LOOKUP AT 80% LOAD
Author:
ppiacentini
Summary:
This test is to insure that the Netra X5-2 MPE will be able to handle a load of 100% of its rated capacity and stay within
the latency and performance guidelines as specified in the Reference Architecture B document. The bulk of the traffic
will be RADIUS sessions with quota management over Sd. A small amount of the traffic will be Gx sessions.
.
Steps:
1. Set up a Netra X5-2 MRA cluster, with at least one Netra X5-2 MPE cluster behind it
2. The MRA and MPE are configured for RADIUS, with a common shared secret
3. The MPE is configured with an SPR Data Source, and E164 will be used for the Subscription-Id
4. 10% of the total traffic will consist of Gx sessions
5. Policy will be in place to create an Sd session for each RADIUS session, and quota will be granted and tracked on the Sd
session
6. Set up seagull test drivers such that they create and terminate sessions to generate an aggregate rate on the MPE that equals
80% of the rated load for this release. 80% of the RADIUS sessions will be Account-Start and Account-Stop, with a 3 minute
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 68 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
hold time. 10% of the sessions will have a mid-session update triggered by a usage threshold on the Sd session, and another
10% of the sessions will have a mid-sessions update due to a profile change on the SPR. Both of these mid-session updates
will trigger a COA request to the BNG to install services.
7. Traffic will run at steady state for two hours
8. Protocol counts, CPU and memory usage will be monitored during the test
Expected Results:


The measured latency should not exceed the established guidelines described in the Reference Architecture B
document.
The MRA and MPE clusters should not have any failovers or exhibit any system or forwarding problems during the
test
RequirementsR-19526026-500: R-19526026-500 Reference Architecture C-1 shall be supported with the following minimum
physical con
R-19526026-510: R-19526026-510 ] Reference Architecture C-1 shall support the following performance figure using S
R-19526026-530: R-19526026-530 The Policy Solution shall support the following general RADIUS and Sd message flows.
R-19526026-540: R-19526026-540 The general traffic model for each MPE for the call flow for RADIUS COA and Sd per [
Keywords:Performance
ANCHORAGE
10.4.2 CAM-94631:RADIUS WITH SD AND SPR PROFILE LOOKUP AT 100% LOAD
Author:
ppiacentini
Summary:
This test is to insure that the Netra X5-2 MPE will be able to handle a load of 100% of its rated capacity and stay
within the latency and performance guidelines as specified in the Reference Architecture B document. The bulk
of the traffic will be RADIUS sessions with quota management over Sd. A small amount of the traffic will be Gx
sessions
Steps:
1. Set up a Netra X5-2 MRA cluster, with at least one Netra X5-2 MPE cluster behind it
2. The MRA and MPE are configured for RADIUS, with a common shared secret
3. The MPE is configured with an SPR Data Source, and E164 will be used for the Subscription-Id
4. 10% of the total traffic will consist of Gx sessions
5. Policy will be in place to create an Sd session for each RADIUS session, and quota will be granted and
tracked on the Sd session
6. Set up seagull test drivers such that they create and terminate sessions to generate an aggregate rate on the
MPE that equals 100% of the rated load for this release. 80% of the RADIUS sessions will be Account-Start and
Account-Stop, with a 3 minute hold time. 10% of the sessions will have a mid-session update triggered by a
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 69 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
usage threshold on the Sd session, and another 10% of the sessions will have a mid-sessions update due to a
profile change on the SPR. Both of these mid-session updates will trigger a COA request to the BNG to install
services.
7. Traffic will run at steady state for two hours
8. Protocol counts, CPU and memory usage will be monitored during the test
Expected Results:


The measured latency should not exceed the established guidelines described in the Reference
Architecture B document.
The MRA and MPE clusters should not have any failovers or exhibit any system or forwarding
problems during the test
Requirements
R-19526026-500: R-19526026-500 Reference Architecture C-1 shall be supported with
the following minimum physical con
R-19526026-510: R-19526026-510 ] Reference Architecture C-1 shall support the
following performance figure using S
R-19526026-530: R-19526026-530 The Policy Solution shall support the following
general RADIUS and Sd message flows.
R-19526026-540: R-19526026-540 The general traffic model for each MPE for the call
flow for RADIUS COA and Sd per [
Keywords:
Performance
ANCHORAGE
10.4.3
TEST CASE CAM-94632: RADIUS WITH SD AND SPR PROFILE LOOKUP AT 120% LOAD
Author:
ppiacentini
Summary:
This test is to insure that the Netra X5-2 MPE will be able to handle a load of 120% of its rated capacity and stay
within the latency and performance guidelines as specified in the Reference Architecture B document. The bulk
of the traffic will be RADIUS sessions with quota management over Sd. A small amount of the traffic will be Gx
sessions.
Steps:
1. Set up a Netra X5-2 MRA cluster, with at least one Netra X5-2 MPE cluster behind it
2. The MRA and MPE are configured for RADIUS, with a common shared secret
3. The MPE is configured with an SPR Data Source, and E164 will be used for the Subscription-Id
4. 10% of the total traffic will consist of Gx sessions
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 70 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
5. Policy will be in place to create an Sd session for each RADIUS session, and quota will be granted and
tracked on the Sd session
6. Set up seagull test drivers such that they create and terminate sessions to generate an aggregate rate on the
MPE that equals 120% of the rated load for this release. 80% of the RADIUS sessions will be Account-Start and
Account-Stop, with a 3 minute hold time. 10% of the sessions will have a mid-session update triggered by a
usage threshold on the Sd session, and another 10% of the sessions will have a mid-sessions update due to a
profile change on the SPR. Both of these mid-session updates will trigger a COA request to the BNG to install
services.
7. Traffic will run at steady state for two hours
8. Protocol counts, CPU and memory usage will be monitored during the test
Expected Results:


The measured latency should not exceed the established guidelines described in the Reference
Architecture B document.
The MRA and MPE clusters should not have any failovers or exhibit any system or forwarding
problems during the test
Requirements
R-19526026-500: R-19526026-500 Reference Architecture C-1 shall be supported with
the following minimum physical con
R-19526026-510: R-19526026-510 ] Reference Architecture C-1 shall support the
following performance figure using S
R-19526026-530: R-19526026-530 The Policy Solution shall support the following
general RADIUS and Sd message flows.
R-19526026-540: R-19526026-540 The general traffic model for each MPE for the call
flow for RADIUS COA and Sd per [
Keywords:
Performance
ANCHORAGE
10.4.4
TEST CASE CAM-94633: RADIUS WITH SD AND SPR PROFILE LOOKUP AT 150% LOAD
Author:
ppiacentini
Summary:
This test is to insure that the Netra X5-2 MPE will be able to handle a load of 150% of its rated capacity and stay
within the latency and performance guidelines as specified in the Reference Architecture B document. The bulk
of the traffic will be RADIUS sessions with quota management over Sd. A small amount of the traffic will be Gx
sessions.
Steps:
1. Set up a Netra X5-2 MRA cluster, with at least one Netra X5-2 MPE cluster behind it
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 71 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
2. The MRA and MPE are configured for RADIUS, with a common shared secret
3. The MPE is configured with an SPR Data Source, and E164 will be used for the Subscription-Id
4. 10% of the total traffic will consist of Gx sessions
5. Policy will be in place to create an Sd session for each RADIUS session, and quota will be granted and
tracked on the Sd session
6. Set up seagull test drivers such that they create and terminate sessions to generate an aggregate rate on the
MPE that equals 150% of the rated load for this release. 80% of the RADIUS sessions will be Account-Start and
Account-Stop, with a 3 minute hold time. 10% of the sessions will have a mid-session update triggered by a
usage threshold on the Sd session, and another 10% of the sessions will have a mid-sessions update due to a
profile change on the SPR. Both of these mid-session updates will trigger a COA request to the BNG to install
services.
7. Traffic will run at steady state for two hours
8. Protocol counts, CPU and memory usage will be monitored during the test
Expected Results:


The measured latency should not exceed the established guidelines described in the Reference
Architecture B document.
The MRA and MPE clusters should not have any failovers or exhibit any system or forwarding
problems during the test
Requirements
R-19526026-500: R-19526026-500 Reference Architecture C-1 shall be supported with
the following minimum physical con
R-19526026-510: R-19526026-510 ] Reference Architecture C-1 shall support the
following performance figure using S
R-19526026-530: R-19526026-530 The Policy Solution shall support the following
general RADIUS and Sd message flows.
R-19526026-540: R-19526026-540 The general traffic model for each MPE for the call
flow for RADIUS COA and Sd per [
Keywords:
Performance
ANCHORAGE
10.4.5
TEST CASE CAM-94634: RADIUS WITH SD AND SPR PROFILE LOOKUP AT 200% LOAD
Author:
ppiacentini
Summary:
This test is to insure that the Netra X5-2 MPE will be able to handle a load of 200% of its rated capacity and stay
within the latency and performance guidelines as specified in the Reference Architecture B document. The bulk
of the traffic will be RADIUS sessions with quota management over Sd. A small amount of the traffic will be Gx
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 72 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
sessions.
Steps:
1. Set up a Netra X5-2 MRA cluster, with at least one Netra X5-2 MPE cluster behind it
2. The MRA and MPE are configured for RADIUS, with a common shared secret
3. The MPE is configured with an SPR Data Source, and E164 will be used for the Subscription-Id
4. 10% of the total traffic will consist of Gx sessions
5. Policy will be in place to create an Sd session for each RADIUS session, and quota will be granted and
tracked on the Sd session
6. Set up seagull test drivers such that they create and terminate sessions to generate an aggregate rate on the
MPE that equals 200% of the rated load for this release. 80% of the RADIUS sessions will be Account-Start and
Account-Stop, with a 3 minute hold time. 10% of the sessions will have a mid-session update triggered by a
usage threshold on the Sd session, and another 10% of the sessions will have a mid-sessions update due to a
profile change on the SPR. Both of these mid-session updates will trigger a COA request to the BNG to install
services.
7. Traffic will run at steady state for two hours
8. Protocol counts, CPU and memory usage will be monitored during the test
Expected Results:


The measured latency should not exceed the established guidelines described in the Reference
Architecture B document.
The MRA and MPE clusters should not have any failovers or exhibit any system or forwarding
problems during the test
Requirements
R-19526026-500: R-19526026-500 Reference Architecture C-1 shall be supported with
the following minimum physical con
R-19526026-510: R-19526026-510 ] Reference Architecture C-1 shall support the
following performance figure using S
R-19526026-530: R-19526026-530 The Policy Solution shall support the following
general RADIUS and Sd message flows.
R-19526026-540: R-19526026-540 The general traffic model for each MPE for the call
flow for RADIUS COA and Sd per [
Keywords:
Performance
ANCHORAGE
10.4.6
TEST CASE CAM-94635: RADIUS WITH SD AND SPR PROFILE LOOKUP LONGEVITY
TEST
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 73 of 94
PROPRIETARY INFORMATION
Author:
INTERNAL USE ONLY
ppiacentini
Summary:
This test is to insure that the Netra X5-2 MPE will be able to handle a load of 100% of its rated capacity for a
period of at least 48 hours.
Steps:
1. Set up a Netra X5-2 MRA cluster, with at least one Netra X5-2 MPE cluster behind it
2. The MRA and MPE are configured for RADIUS, with a common shared secret
3. The MPE is configured with an SPR Data Source, and E164 will be used for the Subscription-Id
4. Policy will be in place to create an Sd session for each RADIUS session, and quota will be granted and
tracked on the Sd session
5, Session cleanup will be enabled for the duration of this test
6. Set up seagull test drivers such that they create and terminate sessions to generate an aggregate rate on the
MPE that equals 80% of the rated load for this release. 80% of the sessions will be Account-Start and AccountStop, with a 3 minute hold time. 10% of the sessions will have a mid-session update triggered by a usage
threshold on the Sd session, and another 10% of the sessions will have a mid-sessions update due to a profile
change on the SPR. Both of these mid-session updates will trigger a COA request to the BNG to install services.
7. Traffic will run at steady state for 72 hours
8. Protocol counts, CPU and memory usage will be monitored during the test
Expected Results:

The Netra X5-2 will support RADIUS traffic for a period of at least 72 hours without any crashes,
failovers, abnormal memory or CPU usage, or any other unexpected failures during the 72 hour period.
Requirements
R-19526026-530: R-19526026-530 The Policy Solution shall support the following
general RADIUS and Sd message flows.
R-19526026-540: R-19526026-540 The general traffic model for each MPE for the call
flow for RADIUS COA and Sd per [
R-19526026-550: R-19526026-550 The Policy R12.1 system shall support 72 hour
duration tests running Hardware Confi
Keywords:
Performance
ANCHORAGE
10.5 TEST SUITE : REFERENCE ARCHITECTURE B IMPAIRMENT TESTS
This test suite contains Fault Insertion test cases for Reference Architecture B
10.5.1
TEST CASE CAM-95400: BL460 C-CLASS ACTIVE MPE FAILOVER TO STANDBY MPE
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 74 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
AT 100% LOAD
Author:
ppiacentini
Summary:
This test is to verify that the BL460 C-Class MPE when faced with a 100% load can be failed over to the
Standby MPE, which will proceed to process the load without issues
Test setup: (1) CMP cluster, (1) SPR, (1) MRA cluster, (10) MPE clusters
1. The MPE is configured in Gx quota mode
2. The MPE has Validation enabled, and has an SPR Data Source
3. The MPE is configured to index by E164 address
4. The test driver will use the same set of E164 addresses that are configured in the SPR
Steps:
1. Set up an MPE clusters so that the subscriber lookup will be to the SPR, and state and quote read/write will
written to the SPR
2. Start traffic such that each MPE will be processing 100% of it's rated load at steady state
3. Fail over the active MPEs by doing a "service qp_procmgr restart"
4. Verify that the Standby MPE takes over as Active and can still process the traffic at 100% rate without any
issues
Expected Results:

The Standby MPE should be able to become active and process 100% of the load in the case of a failure
of the active MPE
Requirements
None
Keywords:
Performance
Fault Insertion
ANCHORAGE
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 75 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
10.5.2 TEST CASE CAM-99006: BL460-G9 C-CLASS ACTIVE MPE FAILOVER TO STANDBY MPE
AT 100% LOAD
Author:
ppiacentini
Summary:
This test is to verify that the BL460-G9 C-Class MPE when faced with a 100% load can be failed over to the
Standby MPE, which will proceed to process the load without issues
Test setup: (1) CMP cluster, (1) SPR, (1) MRA cluster, (10) MPE clusters
1. The MPE is configured in Gx quota mode
2. The MPE has Validation enabled, and has an SPR Data Source
3. The MPE is configured to index by E164 address
4. The test driver will use the same set of E164 addresses that are configured in the SPR
Steps:
1. Set up an MPE clusters so that the subscriber lookup will be to the SPR, and state and quote read/write will
written to the SPR
2. Start traffic such that each MPE will be processing 100% of it's rated load at steady state
3. Fail over the active MPEs by doing a "service qp_procmgr restart"
4. Verify that the Standby MPE takes over as Active and can still process the traffic at 100% rate without any
issues
Expected Results:

The Standby MPE should be able to become active and process 100% of the load in the case of a failure
of the active MPE
Requirements
None
Keywords:
Performance
Fault Insertion
ANCHORAGE
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 76 of 94
PROPRIETARY INFORMATION
10.5.3
INTERNAL USE ONLY
TEST CASE CAM-95401: ORACLE X5-2 ACTIVE MPE FAILOVER TO STANDBY MPE AT
100% LOAD
Author:
ppiacentini
Summary:
This test is to verify that the Oracle X5-2 MPE when faced with a 100% load can be failed over to the Standby
MPE, which will proceed to process the load without issues
Test setup: (1) CMP cluster, (1) SPR, (1) MRA cluster, (2) MPE clusters
1. The MPE is configured in Gx quota mode
2. The MPE has Validation enabled, and has an SPR Data Source
3. The MPE is configured to index by E164 address
4. The test driver will use the same set of E164 addresses that are configured in the SPR
Steps:
1. Set up an MPE clusters so that the subscriber lookup will be to the SPR, and state and quote read/write will
written to the SPR
2. Start traffic such that each MPE will be processing 100% of it's rated load at steady state
3. Fail over the active MPEs by doing a "service qp_procmgr restart"
4. Verify that the Standby MPE takes over as Active and can still process the traffic at 100% rate without any
issues
Expected Results:

The Oracle X5-2 Standby MPE should be able to become active and process 100% of the load in the
case of a failure of the active MPE
Requirements
None
Keywords:
Performance
Fault Insertion
ANCHORAGE
10.5.4
TEST CASE CAM-95402: BL460 C-CLASS GEO-REDUNDANT SITE1 MPE FAILOVER TO
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 77 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
SITE2 MPE AT 100% LOAD
Author:
ppiacentini
Summary:
This test is to verify that the BL460 C-Class Geo-Redundant MPE when faced with a 100% load can be failed
over to the MPE, at the secondary site, which will proceed to process the load without issues
Test setup: (1) CMP cluster, (1) SPR, (1) MRA cluster, (10) MPE clusters, including at least 1 Geo-Redundant
cluster
1. The MPE is configured in Gx quota mode
2. The MPE has Validation enabled, and has an SPR Data Source
3. The MPE is configured to index by E164 address
4. The test driver will use the same set of E164 addresses that are configured in the SPR
Steps:
1. Set up an MPE clusters so that the subscriber lookup will be to the SPR, and state and quote read/write will
written to the SPR
2. Start traffic such that each MPE will be processing 100% of it's rated load at steady state
3. Fail over the active MPEs by doing a "service qp_procmgr restart"
4. Verify that the Standby MPE takes over as Active and can still process the traffic at 100% rate without any
issues
Expected Results:

The secondary site MPE should be able to become active and process 100% of the load in the case of a
failure of the MPE at the primary site
Requirements
None
Keywords:
Performance
Fault Insertion
ANCHORAGE
10.5.5
TEST CASE CAM-95403: ORACLE X5-2 GEO-REDUNDANT SITE1 MPE FAILOVER TO
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 78 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
SITE2 MPE AT 100% LOAD
Author:
ppiacentini
Summary:
This test is to verify that the Oracle X5-2 Geo-Redundant MPE when faced with a 100% load can be failed over
to the MPE, at the secondary site, which will proceed to process the load without issues
Test setup: (1) CMP cluster, (1) SPR, (1) MRA cluster, (2) MPE clusters, including at least 1 Netra GeoRedundant cluster
1. The MPE is configured in Gx quota mode
2. The MPE has Validation enabled, and has an SPR Data Source
3. The MPE is configured to index by E164 address
4. The test driver will use the same set of E164 addresses that are configured in the SPR
Steps:
1. Set up an MPE clusters so that the subscriber lookup will be to the SPR, and state and quote read/write will
written to the SPR
2. Start traffic such that each MPE will be processing 100% of it's rated load at steady state
3. Fail over the active MPEs by doing a "service qp_procmgr restart"
4. Verify that the Standby MPE takes over as Active and can still process the traffic at 100% rate without any
issues
Expected Results:

The Oracle X5-2 secondary site MPE should be able to become active and process 100% of the load in
the case of a failure of the MPE at the primary site
Requirements
None
Keywords:
Performance
Fault Insertion
ANCHORAGE
10.5.6
TEST CASE CAM-95404: BL460 C-CLASS ACTIVE MRA FAILOVER TO STANDBY MRA
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 79 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
AT 100% LOAD
Author:
ppiacentini
Summary:
This test is to verify that the BL460 C-Class MRA when faced with a 100% load can be failed over to the
Standby MRA, which will proceed to process the load without issues
Test setup: (1) CMP cluster, (1) SPR, (1) MRA cluster, (10) MPEs
Steps:
1. Start traffic to the MRA so that it's running at 100% of the rated transaction rate for this release
2. Let the traffic run at steady state for at least 15 minutes
3. Fail over the active MRA by doing a "service qp_procmgr restart"
4. Verify that the Standby MRA takes over as Active and can still process the traffic at 100% rate without any
issues
Expected Results:

The Standby MRA should be able to become active and process 100% of the load in the case of a
failure of the active MRA
Requirements
None
Keywords:
Performance
Fault Insertion
ANCHORAGE
10.5.7 TEST CASE CAM-99007: BL460-G9 C-CLASS ACTIVE MRA FAILOVER TO STANDBY MRA
AT 100% LOAD
Author:
ppiacentini
Summary:
This test is to verify that the BL460-G9 C-Class MRA when faced with a 100% load can be failed over to the
Standby MRA, which will proceed to process the load without issues
Test setup: (1) CMP cluster, (1) SPR, (1) MRA cluster, (10) MPEs
Steps:
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 80 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
1. Start traffic to the MRA so that it's running at 100% of the rated transaction rate for this release
2. Let the traffic run at steady state for at least 15 minutes
3. Fail over the active MRA by doing a "service qp_procmgr restart"
4. Verify that the Standby MRA takes over as Active and can still process the traffic at 100% rate without any
issues
Expected Results:

The Standby MRA should be able to become active and process 100% of the load in the case of a
failure of the active MRA
Requirements
None
Keywords:
Performance
Fault Insertion
ANCHORAGE
10.5.8
TEST CASE CAM-95405: ORACLE X5-2 MRA FAILOVER TO STANDBY MRA AT 100%
LOAD
Author:
ppiacentini
Summary:
This test is to verify that the Oracle X5-2 MRA when faced with a 100% load can be failed over to the Standby
MRA, which will proceed to process the load without issues
Test setup: (1) CMP cluster, (1) SPR, (1) Netra MRA cluster, (2) MPEs
Steps:
1. Start traffic to the Oracle X5-2 MRA so that it's running at 100% of the rated transaction rate for this release
2. Let the traffic run at steady state for at least 15 minutes
3. Fail over the active MRA by doing a "service qp_procmgr restart"
4. Verify that the Standby MRA takes over as Active and can still process the traffic at 100% rate without any
issues
Expected Results:

The Oracle X5-2 Standby MRA should be able to become active and process 100% of the load in the
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 81 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
case of a failure of the active MRA
Requirements
None
Keywords:
Performance
Fault Insertion
ANCHORAGE
10.5.9
TEST CASE CAM-95406: BL460 C-CLASS GEO-REDUNDANT PRIMARY SITE MRA
FAILOVER TO SECONDARY SITE MRA AT 100% LOAD
Author:
ppiacentini
Summary:
This test is to verify that the Geo-Redundant BL460 C-Class MRA at the primary site when faced with a 100%
load can be failed over to the secondary site MRA, which will proceed to process the load without issues
Test setup: (1) CMP cluster, (1) SPR, (1) Geo-Redundant BL460 MRA cluster, (10) MPEs
Steps:
1. Start traffic to the primary site BL460 MRA so that it's running at 100% of the rated transaction rate for this
release
2. Let the traffic run at steady state for at least 15 minutes
3. Stop the Standby MRA at the primary site by doing "service qp_procmgr stop", and then fail over the primary
site MRA by doing a "service qp_procmgr restart"
4. Verify that the secondary site MRA takes over as Active and can still process the traffic at 100% rate without
any issues
Expected Results:

The secondary site BL460 MRA should be able to become active and process 100% of the load in the
case of a failure of the primary site MRA cluster
Requirements
None
Keywords:
Performance
Fault Insertion
ANCHORAGE
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 82 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
10.5.10 TEST CASE CAM-95407: ORACLE X5-2 GEO-REDUNDANT PRIMARY SITE MRA
FAILOVER TO SECONDARY SITE MRA AT 100%
Author:
ppiacentini
Summary:
This test is to verify that the Geo-Redundant Netra X5-2 MRA at the primary site when faced with a 100% load
can be failed over to the secondary site MRA, which will proceed to process the load without issues
Test setup: (1) CMP cluster, (1) SPR, (1) Geo-Redundant Oracle MRA cluster, (2) MPEs
Steps:
1. Start traffic to the primary site Oracle MRA so that it's running at 100% of the rated transaction rate for this
release
2. Let the traffic run at steady state for at least 15 minutes
3. Stop the Standby MRA at the primary site by doing "service qp_procmgr stop", and then fail over the primary
site MRA by doing a "service qp_procmgr restart"
4. Verify that the secondary site MRA takes over as Active and can still process the traffic at 100% rate without
any issues
Expected Results:
The secondary site Oracle X5-2 MRA should be able to become active and process 100% of the load in the case
of a failure of the primary site MRA cluster
Requirements
None
Keywords:
Performance
Fault Insertion
ANCHORAGE
10.5.11 TEST CASE CAM-95408: SURGE OF SESSION CREATION REQUESTS
Author:
ppiacentini
Summary:
This test will verify that the MRA can withstand a burst of session creation attempts, such as the case where a
gateway restarts and a surge of session requests is seen at the MRA
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 83 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
Steps:
1. Start seagull traffic to the MRA, such that the total tps is hitting the rated maximum for the MRA. Set the hold
time of the sessions so that at steady state the MRA will be at or near its rated binding capacity.
2. Once the seagull is at a point where the above sessions are all terminating (should be at 50% of MRA tps
capacity), start a new set of seaguls such that the tps of the session creations i(CCR_Inits) s now >70% of the
rated tps capacity of the MRA. This should give a load on the MRA of 120%, where 50% is sessions terminating
and 70% is sessions being created
3. On the new set of seagull scripts, try pause/unpause the traffic several times while the sessions are being
created.
4. Try similar overloads with CCR-Updates, CCR-Terminates, and Rx messages
Expected Results:

The MRA should be able to handle surges of sessions being created that exceeds the number of sessions
being terminated, where the total tps rate is up to 20% higher than the rated load for the MRA
Requirements
None
Keywords:
Performance
Fault Insertion
ANCHORAGE
10.5.12 TEST CASE CAM-98453: 100% LOAD TO MRA WITH 80MS LATENCY ADDED BETWEEN
MRA AND MPE
Author:
ppiacentini
Summary:
This performance test will be using a configuration using the rest Of World configuration. In this configuration, an SPR
will be used for Profile lookup and quota management In this test, 100% of the rated capacity for the chassis will be sent
to the MRA, which will distribute the load evenly to the MPEs in it's pool. The chassis used will contain 7 Geo-Redundant
MPE clusters,
For this test an artificial latency of 80 ms will be introduced between the MRA and all of the MPEs.
Test setup: (1) CMP cluster, (1) SPR, (1) Geo-Redundant MRA cluster, (7) Geo-Redudnat MPE clusters
1. The MPE is configured in Gx quota mode
2. The MPE has Validation enabled, and has an HSS Data Source
3. The MPE is configured to index by E164 address
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 84 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
4. The test driver will use the same set of E164 addresses that are configured in the SPR
Steps:
1. Test drivers will be using seagull to generate the traffic to the MRA
2. The MRA will evenly distribute the traffic to the 7 MPEs in the chassis enclosure
3. Add 80ms latency between the MRA and the MPEs by using the following command example on the MRA, (changing
the MPE IP addresses and bond interfaces appropriately):
tc qdisc add dev bond0.240 root handle 1: prio
tc qdisc add dev bond0.240 parent 1:3 handle 30: netem delay 30ms
tc filter add dev bond0.240 protocol ip parent 1:0 prio 1 u32 match ip dst 10.15.25.159/32 flowid 1:3
4. Traffic will be sent to the MRA, so that it generates a 100% load on the MPEs in it's pool. Traffic will run for 2 hours.
5. During the test, the statistics of the MRA and the MPEs will be captured in a CSV file for memory usage, including
CPU load, memory usage, protocol counters
6. During the test, the KPI statistics in the CMP will be monitored for CPU load and memory usage, and other
information in real time
7 Remove the latency using the following commands on the MRA (again changing the bond interface and MPE IP):
tc filter del dev bond0.240 protocol ip parent 1:0 prio 1 u32 match ip dst 10.15.25.159 flowid 1:3
tc qdisc del dev bond0.240 parent 1:3 handle 30: netem delay 30ms
tc qdisc del dev bond0.240 root handle 1: prio
Expected Results:
1. 99% of the call latencies will be below (100 + 80) ms
2. 99.9% of the call latencies will be below (200 + 80) ms
2. There will be no unexpected crashes, exceptions, or other unusual conditions on the MRA or MPEs
Requirements
None
Keywords:
Performance
ANCHORAGE
10.5.13 TEST CASE CAM-98454: 100% LOAD TO MRA WITH 80MS LATENCY ADDED BETWEEN
MPES AND SPR
Author:
ppiacentini
Summary:
This performance test will be using a configuration using the rest Of World configuration. In this configuration, an SPR
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 85 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
will be used for Profile lookup and quota management In this test, 100% of the rated capacity for the chassis will be sent
to the MRA, which will distribute the load evenly to the MPEs in it's pool. The chassis used will contain 7 Geo-Redundant
MPE clusters,
For this test an artificial latency of 80 ms will be introduced between the MPEs and the SPR
Test setup: (1) CMP cluster, (1) SPR, (1) Geo-Redundant MRA cluster, (7) Geo-Redudnat MPE clusters
1. The MPE is configured in Gx quota mode
2. The MPE has Validation enabled, and has an HSS Data Source
3. The MPE is configured to index by E164 address
4. The test driver will use the same set of E164 addresses that are configured in the SPR
Steps:
1. Test drivers will be using seagull to generate the traffic to the MRA
2. The MRA will evenly distribute the traffic to the 7 MPEs in the chassis enclosure
3. Add 80ms latency between the MPEs and the SPR by using the following command example on the MPE, (changing
the MPE IP addresses and bond interfaces appropriately):
tc qdisc add dev bond0.240 root handle 1: prio
tc qdisc add dev bond0.240 parent 1:3 handle 30: netem delay 30ms
tc filter add dev bond0.240 protocol ip parent 1:0 prio 1 u32 match ip dst 10.15.25.159/32 flowid 1:3
4. Traffic will be sent to the MRA, so that it generates a 100% load on the MPEs in it's pool. Traffic will run for 2 hours.
5. During the test, the statistics of the MRA and the MPEs will be captured in a CSV file for memory usage, including
CPU load, memory usage, protocol counters
6. During the test, the KPI statistics in the CMP will be monitored for CPU load and memory usage, and other
information in real time
7 Remove the latency using the following commands on the MRA (again changing the bond interface and MPE IP):
tc filter del dev bond0.240 protocol ip parent 1:0 prio 1 u32 match ip dst 10.15.25.159 flowid 1:3
tc qdisc del dev bond0.240 parent 1:3 handle 30: netem delay 30ms
tc qdisc del dev bond0.240 root handle 1: prio
Expected Results:
1. 99% of the call latencies will be below (100 + 80) ms
2. 99.9% of the call latencies will be below (200 + 80) ms
2. There will be no unexpected crashes, exceptions, or other unusual conditions on the MRA or MPEs
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 86 of 94
PROPRIETARY INFORMATION
Requirements
None
Keywords:
Performance
ANCHORAGE
INTERNAL USE ONLY
10.5.14 TEST CASE CAM-99096: 100% LOAD TO 2-SITE MRAS WITH DROPPED DRMA
MESSAGES BETWEEN MRAS
Author:
ppiacentini
Summary:
This performance test will be using a configuration using the rest Of World configuration. In this configuration,
there will be 2 MRA sites. Each site will be receiving 100% traffic load, A small number DRMA messages
between the 2 MRA sites will be dropped.
For this test 1/10th of a percent of the DRMA messages between the MRAs will be dropped
Test setup: (1) CMP cluster, (2) MRA sites (Primary/Backup), sufficient MPEs or simulators behind each MRA
Steps:
1. Test drivers will be using seagull to generate the traffic to the 2 MRAs
2. Set up the MRAs so that .1% of the DRMA messages will be dropped
If using the "tc" command:
tc qdisc add dev bond1.240 root handle 1: prio
tc qdisc add dev bond1.240 parent 1:3 handle 30: netem loss 0.1%
of 1000) packets to be randomly dropped)
(This causes 1/10th of a percent (i.e. 1 out
tc filter add dev bond1.240 protocol ip parent 1:0 prio 1 u32 match ip dst 10.15.25.197 flowid 1:3
(to undo):
tc qdisc del dev bond1.240 root handle 1: prio
Alternatively, iptables may be used to do this:
iptables -A INPUT --dport 3868 -m statistic --mode random --probability 0.01 -j DROP
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 87 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
3. Using seagull, send traffic to both MRAs so that the load is 100% of the rated capacity of the MRA
4. Monitor the success of the calls to both MRAs
5. Verify that the DBR message timeout counters are increasing, but has no detremental effect on the MRAs
Expected Results:
The MRAs will not have any noticeable problems when a small number of DRB messeges are lost between them
Requirements
None
Keywords:
Performance
ANCHORAGE
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 88 of 94
PROPRIETARY INFORMATION
APPENDIX A.
Review Type:
Document:
Date:
Author:
Moderator:
INTERNAL USE ONLY
REVIEW SUMMARIES
On-Line
TP010213 v0.1
05/01/2015
Paul Piacentini
NA
Department
Reviewer
Reviewer Type
S
Matthew Buehler
CUSTOMER
NPX
Gregg Satkamp
CUSTOMER
CS
Vito Politano
SME
D
Joe Bearden
SME
TPM
Christopher Berluti
SME
Q
Mark Young
SME
PM
Ben Crane
SME
SA
Tarek Abou Assali
SME
S- Sustaining
Piriyakala Suresh
SME
Vote
Comments:
jbearden: Please mark Cust Doc as N/A for this review.
ppiacent: removed cost doc from the review
pisukapa: I guess, including the sample startup scripts for the testcase scenarios and policies for various
call flows would help in excution
ppiacent: Most of the script work and policy development will be done after this test plan is
completed. I can include a few of the policies that I have so far in the test plan for examples.
ppiacent: Added a sample of the policies that will be used in Call #1
pisukapa: Teh callflow for "LTE+ Subscriber Quota(2 PDN Connections) Separate Quota Reporting and
Revalidation Timeout" , is missing? or ..?
pisukapa: Also, for "VoLTE +S9 Outbound Roaming" AND
"VoLTE + S9 Inbound Roaming" are missing?
ppiacent: The "LTE+ Subscriber Quota(2 PDN Connections) Separate Quota Reporting and
Revalidation Timeout" is Call2. Allot GW sends Revalidation Timeout first, then quota update.
The S9 Inbound and Outbound are going to use the same VoLTE script as Call6, if possible. I
will add a mention of that.
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 89 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
pisukapa: Are there any (sample)policies used/deployed for the call flow tests? to satisfy the gnereal
goals of the tests ?
ppiacent: The work on script development and policy definitions will be done after this test plan
is written. I can include a few policies that were written so far for examples.
ppiacent: added a sample of policies to be used for Call #1
pisukapa: Apart from the listed Latency Goals listed As per the FE0077538, the Latecy Requirements are "3. The transaction response time of the Policy
system with two sites for 92% of all transactions shall not exceed 167 + (2 * LL) ms; and
4. The transaction response time of the Policy system with two sites for 99.9% of all transactions shall
not exceed 250 + (2 * LL) ms, where LL is the latency of the link between the two systems
5. The latency from the SPR is expected and assumed to be 25ms or less as part of the transaction.
6. The latency from the OCS simulator is expected and assumed to be 20ms or less as part of the
transaction
7. The latency from the PCEF and AF simulator is expected and assumed to be 20ms or less as part of
the transaction.
"
so wondering do u need the above golas as well?
ppiacent: added these to section 9.2
pisukapa: "Start all scripts used for the mixed call flows" - can include/publish the start scripts in the test
plan?
ppiacent: Most of the scripts that will be needed have not been written yet. Work on creating
those will be done after this test plan is completed.
ppiacent: Added a sample of the policies that will be used for Call #1
pisukapa: I guess, this comment belongs for all the TestCases:
Monitor the CPU, memory, call latency, other KPIs will be collected and logs of all servers for the
duration of the test.
"ther KPIs will be collected" is missing in step # 5
ppiacent: Changed the related step in all applicable test cases to"
"Monitor the CPU, memory, and logs of all servers for the duration of the test. CPU, Memory,
call latencies, average session size, and other KPIs will be recorded during the test. "
isukapa: The Requirements : section, the sentences are CUT and It seems, Incomplete as well.
For example:
R-19526026-200: R-19526026-200 Reference Architecture B-1 shall be supported with the following
minimum physical har
R-19526026-205: R-19526026-205 The Policy DL460 G8 blade configuration used for testing MPE and
MRA shall include ad
ppiacent: The test case repository tool (TestLink) truncates the requirement due to a field length
limitation. The full requirement description can be looked up either by logging into TestLink and
viewing the full description of the requirement, or by looking up the requirement in FE007538
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 90 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
pisukapa: TYPO: "desribed"
pisukapa: described
ppiacent: corrected spelling
pisukapa: The Requirements mappings are missing
ppiacent: There are no requirements in FE007538 related to impairment tests and recovery.
These are more like general expectations.
pravrajp: TYPO: "6.500"
pravrajp: Should this be 6500?
ppiacent: corrected
pravrajp: Also apply to whole document wherever space is required to be inserted
ppiacent: Checked and corrected formatting in steps for all test cases
pravrajp: Please insert space also apply to whole document wherever space is to be inserted
ppiacent: Checked and corrected formatting in steps for all test cases
pravrajp: This para is the expected result so please put this under Expected Result Row
ppiacent: Checked and corrected formatting in steps for all test cases
pravrajp: Please put this paragraph in the separate Expected Result row
ppiacent: Corrected paragraph header
pravrajp: Same as my previous two comments
ppiacent: Checked and corrected formatting in steps for all test cases
pravrajp: Same as pervious comment
ppiacent: Checked and corrected formatting in steps for all test cases
pravrajp: Same as my pervious comment
ppiacent: Checked and corrected formatting in steps for all test cases
pravrajp: Same as my pervious comment; also apply to whole document wherever required.
ppiacent: Checked and corrected formatting in steps for all test cases
pravrajp: TYPO: "wil"
ppiacent: Corrected spelling
pravrajp: TYPO: "wil"
ppiacent: Corrected spelling
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 91 of 94
PROPRIETARY INFORMATION
Review Type:
Document:
Date:
Author:
Moderator:
INTERNAL USE ONLY
In-Person
TP010213 v0.2
05/13/2015
Paul Piacentini
NA
Department
Reviewer
Reviewer Type
S
Matthew Buehler
CUSTOMER
NPX
Gregg Satkamp
CUSTOMER
CS
Vito Politano
SME
D
Joe Bearden
SME
TPM
Christopher Berluti
SME
Q
Mark Young
SME
PM
Ben Crane
SME
SA
Tarek Abou Assali
SME
S- Sustaining
Piriyakala Suresh
SME
Vote
3 (Prasad)
N/A
Comments:
Paul: Do we need to add the tests using “tc” command for latency back in
Yes, they will be added to test plan
Piriya: need a test with large number of policies
Will create one test case at 100% load using large number of policies
Jared/Mat: need to have 2 MRA sites, not just one
Topology B-1 will have 2 MRA sites
Sayan: add host based routing
Will add host based routing to 16 connection test
Sayan: use IMSI for LTE tests
Will add IMSI to LTE calls
Mat: are the mezzanine cards needed?
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 92 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
Chris B: yes a customer is using them (will add them to all blades)
Aruna: is a large SPR profile needed?
Will document average profile size if possible
Piriya: run session cleanup during longevity test
Longevity test will have session cleanup enabled with the default parameters
Sayan: need to run just the 100% S9 inbound/outbound tests
Removed all S9 tests except the 100% inbound and outbound
Piriya: Subscriber Activity log on for one user during tests
One subscriber will be added to subscriber trace for all tests
Piriya: document average session size during tests
The average session size will be documented
Paul: add baseline step tests for the MRA and MPE
Added baseline step tests for MRA and MPE
Sayan: add Gx sessions to RADIUS tests
Added some Gx traffic to the RADIUS tests
Review Type:
Document:
Date:
Author:
Moderator:
In-Person
TP010213 v0.3
06/03/2015
Paul Piacentini
NA
Department
Reviewer
Reviewer Type
S
Matthew Buehler
CUSTOMER
NPX
Gregg Satkamp
CUSTOMER
CS
Vito Politano
SME
D
Joe Bearden
SME
TPM
Christopher Berluti
SME
Q
Mark Young
SME
PM
Ben Crane
SME
SA
Tarek Abou Assali
SME
S- Sustaining
Piriyakala Suresh
SME
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Vote
N/A
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 93 of 94
PROPRIETARY INFORMATION
INTERNAL USE ONLY
Jared: add test case where some amount pf messages between MRAs are dropped
Added test case to drop .1% of messages between MRAs at 100% load
Bash: add some cases for BL460-G9 blades
Added 4 test cases for BL460-G9
Review Type:
Document:
Date:
Author:
Moderator:
In-Person
TP010213 v0.5
06/17/2015
Paul Piacentini
NA
Department
Reviewer
Reviewer Type
Vote
S
Matthew Buehler
CUSTOMER
3
NPX
Gregg Satkamp
CUSTOMER
3
CS
Vito Politano
SME
D
Joe Bearden
SME
TPM
Christopher Berluti
SME
Q
Mark Young
SME
PM
Ben Crane
SME
SA
Tarek Abou Assali
SME
S- Sustaining
Piriyakala Suresh
SME
N/A
Received votes of 3 from Guarav Paliwal and Prasad Isukapalli
DO NOT COPY. Hard copies of this
document are for REFERENCE ONLY.
Get the latest version from Documentum.
Title:
Reference Architecture B Performance test plan
Filename:
TP10213.docx
REV:
1.0
Page 94 of 94