Test automation
analysis in System
Testing for the
Intelligent
Packet
Core
Author: Mona Saxena
Supervisor: Professor Jörg Ott
Nokia Networks, Helsinki
Agenda
• Introduction
•
•
•
FlexiISN in Nokia Intelligent Packet Core
FlexiISN Interfaces, Protocols, Features and procedures
Software development process and testing phases
• Research problem
• Research method
• Results
• Conclusions
22.5.2007
Mona Saxena Test automation analysis for System Testing in the Intelligent Packet Core
2
Nokia Intelligent Packet Core
22.5.2007
Mona Saxena Test automation analysis for System Testing in the Intelligent Packet Core
3
FlexiISN Interfaces, Protocols and
Features
Prepaid
Charging
CG
OSC
Radius/
Diameter
Ga
NPS
Gi
SGSN
GTP
Gn
Corporate
Intranet
PoC
R
LD adi
AP us/
Postpaid
Charging
TA
FlexiISN
IP
N/W
Internet
Gi
SNMP
NetAct
Application
Service
Providers
Content
Service
Providers
WAP
22.5.2007
Mona Saxena Test automation analysis for System Testing in the Intelligent Packet Core
4
FlexiISN SW development process and
testing phases
Customer or Business
Requirements
Acceptance Testing
Phase
System Verification
and Integration Phase
Network or System
Requirement
System Level Program
Product Level Program
Product Requirement
Phase
System Testing Phase
Architecture Design
Phase
Functional
Specification Phase
Design and
Specification Phase
Functional Testing
Phase
Product Integration
Phase
Module Testing Phase
Source code implementation
and unit testing
22.5.2007
Mona Saxena Test automation analysis for System Testing in the Intelligent Packet Core
5
Research problem
Software has to be tested thoroughly and requires lot of
time and effort. Therefore a significant amount of time
and available resources could be saved if some aspects
of testing could be automated.
Which areas could be automated and how they could be
automated and their benefit before the product is
delivered to the end customer? How much time it could
take with the necessary available tools to automate the
system testing related configurations and procedures in
FlexiISN and related counterpart network elements of
the Intelligent Packet Core (IPC)?
22.5.2007
Mona Saxena Test automation analysis for System Testing in the Intelligent Packet Core
6
Research method
• Participating in FlexiISN System testing
• Literature study of Intelligent Packet core network
elements, protocols and procedures
• Literature study of software testing and software test
automation
• Discussions with other Service Access testing teams
and collecting and analyzing their automation details.
• Test Automation Analysis based on actual number of
test cases for different counterpart network elements
and functionality areas along side FlexiISN
22.5.2007
Mona Saxena Test automation analysis for System Testing in the Intelligent Packet Core
7
Test automation metrics
• Maintainability
• Efficiency and its relation to cost
• Reliability of an automated testing regime
• Flexibility
• Usability
• Robustness
• Portability
22.5.2007
Mona Saxena Test automation analysis for System Testing in the Intelligent Packet Core
8
Test Automation analysis
• Search for actual number of test cases along with
involved counterpart network elements or
functionality areas
•
•
Discovered what network element or functionality area has
the maximum test cases in all the product releases
Discovered what network element or functionality area has
the minimum test cases in all the product releases
• Search on benefits to specific areas of testing with
involved counterpart network elements or functionality
areas
•
•
•
22.5.2007
Pre-Configuration
Collection of results post execution
Analysis of results post execution
Mona Saxena Test automation analysis for System Testing in the Intelligent Packet Core
9
Test automation analysis results
• Based on actual number of test cases along with
involved counterpart network elements or
functionality areas the highest priority for automation
is towards test cases involving OSC and CG.
• Based on benefits to specific areas of testing with
involved counterpart network elements or functionality
areas
•
•
•
22.5.2007
For Pre-configuration maximum benefit was for test cases
using OSC, NPS, HA feature and LIG
For collection of results maximum benefit was for test cases
using CG, HA feature and LIG
For analysis of results the maximum benefit was for test
cases using CG followed by LIG
Mona Saxena Test automation analysis for System Testing in the Intelligent Packet Core
10
Test automation implementation for System
Testing
• Order of implementation of automation: Based on test
automation analysis
• Method of automation implementation
•
•
Use previously available scripts from other testing teams
Use scripts from System Verification in particular as the tasks
taken by them is quite similar.
• Tool setup for Automation Implementation
•
LASS Test Automation setup
• Hardware Environment need for automation
•
•
22.5.2007
Laptops for individual testers with USB ports for mobile phone
/Handset connectivity
Red Hat 8.0 Server for script storage and version control
Mona Saxena Test automation analysis for System Testing in the Intelligent Packet Core
11
Test automation implementation for System
Testing
• Effort Estimation for Test Automation Implementation
• Competence and Training needs for Automation
implementation
• Maintenance of Test Automation for the System
Testing team
22.5.2007
Mona Saxena Test automation analysis for System Testing in the Intelligent Packet Core
12
Conclusions and future work
• Test automation is an effective way for testing in a fast
and efficient manner.
• Everything that needs to be tested cannot be and
should not be automated. Automation in testing
depends upon software feature implementation and its
benefit for a particular test case scenario.
• The most important future work is to have the actual
test automation system in place and then analyze the
overall benefits of automation for the System Testing
team.
22.5.2007
Mona Saxena Test automation analysis for System Testing in the Intelligent Packet Core
13
Thank You!
Questions?
22.5.2007
Mona Saxena Test automation analysis for System Testing in the Intelligent Packet Core
14