WHITE PAPER
Application Service Testing
Enabling scalable delivery of layer 4-7
services
www.ixiacom.com
915-1735-01 Rev. C January 2013
2
Table of Contents
Introduction.................................................................................................. 4
Industry Drivers........................................................................................... 4
Multiplay Services........................................................................................ 4
Application Layer Forwarding...................................................................... 5
Security Threats........................................................................................... 6
Peer to Peer................................................................................................. 7
Putting it all together.................................................................................... 7
Testing Challenges........................................................................................ 8
Measuring application performance............................................................. 8
How can Ixia help?....................................................................................... 9
Real-world traffic modeling.........................................................................10
Subscriber behavior................................................................................... 11
DoS attacks................................................................................................. 11
Ease of Use.................................................................................................12
IxLoad Features...........................................................................................13
Conclusion...................................................................................................15
3
Introduction
Demand for multiplay services, each with its own bandwidth and delivery requirements,
is driving the emergence of application-aware networking devices. This is due to the fact
that legacy approaches to quality of service (QoS) enforcement are no longer sufficient
to properly differentiate service types. Application-aware switches, routers, and load
balancers need to perform deep packet inspection (DPI) to properly classify traffic in order
to implement appropriate QoS policies. Testing the accuracy and performance of such
content inspection devices poses unique challenges. Some of the testing challenges and
Ixia’s answer to them are discussed in this white paper.
Industry Drivers
Multiplay Services
Testing the accuracy
and performance
of such content
inspection devices
poses unique
challenges.
Service providers are increasingly looking to deliver multiplay services to businesses
and digital homes over their IP networks. 2007 revenue for services delivered by service
providers is expected to top $1.54 trillion, with $264 billion in CAPEX purchases1. This
is primarily driven by competition, especially in cases where service providers are
moving into new markets or expanding their services. For example, many telcos are
rushing to compete with cable operators in the delivery of video, while cable operators
are rushing to provide VoIP options to compete with the telcos. All of this is ultimately
driven by consumers looking to simplify their digital life. Service providers recognize this
opportunity and are looking to increase their average return per user (ARPU) by offering
bundles that consolidate their voice, TV, wireless and Internet services into a single bill.
The breadth of services delivered to the digital home and office are shown in Figure 1.
Figure 1. Multiplay service requirements
1 Service Provider Capex, Opex, ARPU & Subscribers Market Size & Forecasts, November 2006, Infonetics
Research.
4
Office and home subscribers use the full mix of rich interactive services including
business applications, online gaming, video on demand, and instant messaging. This is
against a background of high-bandwidth applications that include peer-to-peer, FTP, and
broadcast video. Each of these applications has its own performance requirements. The
three most common measurements are:
•
Bandwidth – the average amount of data transferred
•
Latency – the delay between request and response
•
Jitter – the disruption from a constant delivery rate
The sum of these and other factors is referred to as the quality of experience (QoE) and is
a reflection of how satisfied end-users are with the services they receive.
Note how each type of service comes with its own requirements. Voice over IP (VoIP)
has a very low bandwidth requirement, but requires low latency and jitter. IPTV uses
high bandwidth, but is tolerant of moderate jitter. P2P, on the other hand has very high
bandwidth requirements and can sustain high latency and jitter.
Service providers must ensure that the QoE of their services “feels right.” VoIP calls must
sound as good as land-line service, IPTV must be absent of blockiness, blurring, or frozen
frames and high-speed Internet services must appear responsive. Subscribers, especially
home users, have a very low tolerance for these types of defects; they’ll quickly switch
back to prior services or move on to competitors.
In addition, business customers often demand specific performance in their service level
agreements (SLAs). Hence, all of their services must be prioritized ahead of home users.
Application Layer Forwarding
To accommodate multiplay service delivery, service providers need to offer tiered or
differentiated pricing models based on services and service guarantees. As detailed in
Figures 2 and 3, traditional means of packet switching and routing cannot provide the
necessary specificity to effectively support QoS for multiplay traffic.
To accommodate
multiplay service
delivery, service
providers need
to offer tiered or
differentiated pricing
models based on
services and service
guarantees.
A different approach, involving deep packet inspection, is necessary. DPI allows the
application of QoS schemes based on service, customer or both. This last technique,
where both the service being provided and the customer to which they are provided
determine QoS policy, is called hierarchical QoS.
This need to deliver differentiated services has given rise to a new family of applicationaware devices that thoroughly analyze the packets and make forwarding decisions based
on content and policies. The growing list of application-aware devices includes:
•
Routers and switches
•
Firewalls
•
Session border controllers (SBCs)
•
Content delivery systems
Inspection of the application data within a packet makes available the information
necessary to determine the true usage of the traffic: interactive content, video, web page
contents, file sharing, etc. It also makes it possible to detect viruses, spam, and proprietary
information within data packets. For example, Windows Messenger uses HTTP, with a
special setting in the User-Agent field of a message. In order to apply the appropriate QoS
policy for instant messaging, the HTTP message must parsed for this value.
5
Figure 2. Traditional packet inspection
Traditional stateful packet inspection looks at the IP and TCP/UDP headers (and
occasionally the Ethernet header) to decide where and how packets are forwarded.
Losses due to
security breaches
that result in theft,
downtime and
brand damage now
stretch into the
tens of millions of
dollars per year for
large enterprises,
according to
Infonetics Research.
The essential information found there includes the source and destination IP address, TCP/
UDP port number and type of service (TOS). The TCP/UDP port numbers have well-known
associations; for example 21 is associated with FTP, 80 with HTTP, 25 with SMTP and 110
with POP3. This 5-tuple of information from layers 3 and 4 is the classic means by which
firewalls, routers and other switching devices decide on whether to and where to forward
packets and with what priority.
This information is increasingly insufficient to satisfy the requirements for multiplay
services in a mixed customer environment. Additional elements of each packet must be
inspected.
Figure 3. Deep packet inspection
The application layer (Layer 7) of the packet holds information specific to a protocol. All
bits and bytes are now available for deep packet inspection, allowing network devices to
finely classify traffic type and source. For example not only can you identify the traffic as
email using SMTP, you can now identify the source application as Microsoft Outlook by
examining the application signature. The information can be used to provide:
•
Subscriber and service based QoS policing
•
Peer-to-peer bandwidth management
•
Denial of service (DoS) and virus attack prevention
•
Intrusion detection and prevention
•
Web and e-mail content filtering
Security Threats
Losses due to security breaches that result in theft, downtime and brand damage now
stretch into the tens of millions of dollars per year for large enterprises, according
to Infonetics Research2. Attacks and failures are seen at every level – from online
applications, to networks, to mobile and core infrastructures.
2The Costs of Network Security Attacks: North America 2007, February 2007, Infonetics Research.
6
Conventional security software and appliances, such as anti-virus protection and firewalls,
have increasingly reduced the number of attacks, but the total losses continue to grow.
The 2007 CSI Computer Crime and Security Survey3 reported that in 2006 the average
loss per survey respondent more than doubled when compared to the year before.
Security issues have pushed defenses into network devices and have spawned a number
of auxiliary security enforcement devices. These functions include:
•
Intrusion detection systems (IDSs)
•
Intrusion prevention systems (IPSs)
•
Unified threat management systems
•
Antivirus filters
•
Antispam filters
Increasingly, application-aware devices are performing security functions – largely
because the information they need is now available through deep packet inspection.
Peer to Peer
Peer-to-peer traffic is estimated to account for 60% of all Internet traffic, with an expected
135 million P2P users by 2010. The amount of traffic seems to have been unaffected
by the adoption of antitheft mechanisms such as digital rights management (DRM),
shifting somewhat to legitimate P2P services. Moreover, there are strong indications that
increased IP video content will drive the use of P2P even higher.
Clever software and devices will use P2P to distribute content, obviating the need for
providers to host large video content infrastructures and making services quick and
inexpensive to deploy and sell. Joost, for example, uses home computers to send and
receive TV and other content. Joost and Vudu use P2P to make thousands of pay-perview movies available to users worldwide.
Putting it all together
The preceding discussion serves to highlight that application layer forwarding is a very
complex and resource-intensive task. Every bit of traffic traversing a device needs to be
inspected and matched against signature libraries containing patterns that match standard
Internet protocols, such as P2P, as well as virus, spam, and hacker intrusion.
Every bit of traffic
traversing a
device needs to
be inspected and
matched against
signature libraries
containing patterns
that match standard
Internet protocols,
such as P2P, as well
as virus, spam, and
hacker intrusion.
Figure 4. Application layer forwarding
3CSI Survey — The 12th Annual Computer Crime and Security Survey — http://www.gocsi.com/.
7
Some of the signatures are split across multiple packets and sometimes even across
multiple TCP connections, making the process more complex and resource intensive.
Hackers often use such techniques to mask their activity. Once a session has been
classified, the QoS policies that apply to that session need to be applied to ensure that the
traffic is placed in the appropriate priority queue – a resource intensive task, especially for
multi-gigabit per second devices.
Some application-aware devices take advantage of their ability to probe deeply into packet
contents to implement other services. For example:
•
Application load balancing
•
Long-term traffic analysis to detect intrusion detection
•
Security access control
•
Billing
These tasks of course make application-aware packet forwarding an even more complex
task.
Since the deep
Testing Challenges
packet inspection
that these devices Such complex devices pose significant problems for network equipment manufacturers
(NEMs) and service providers. They must validate their devices’ in terms of accuracy,
perform recognizes performance and robustness of their QoS strategy implementation.
complete sessions
NEMs must validate their devices’ performance in forwarding application-layer traffic in
and keys off order to compete with other vendors and to ensure customer satisfaction. Forwarding
protocol interchange needs to be verified for a resource-intensive mix of services and applications. The delivery
of services, based on QoS, must be tested to assure proper prioritization of voice and
messages, they need video traffic over data traffic as well as prioritization of business over consumer traffic.
to be tested with QoS can only be tested when an overload of traffic is applied with line-rate traffic on all
the device’s interfaces. Finally, NEMs must ensure that attacks are identified and thwarted
stateful application so that overall device performance is not affected. Insecure devices can be responsible for
traffic that follows financial and brand damage.
protocol rules. Service providers face similar challenges, but with a larger, changing scope. To keep their
customers happy and to maximize their capital investments, they must ensure that their
core network, consisting of many traditional and application-aware devices, maintains
proper QoS for all voice, video and data traffic. Service providers must be particularly
sensitive to their customers’ perception of QoE. Service providers must always be ready
to come to market with new, advanced services in order to quickly capitalize on market
demand with minimal risk. Because service providers are continually upgrading their
networks, either with additional devices or with new devices, network testing must be
frequently repeated.
Measuring application performance
The requirements for testing application-aware devices are as complex, if not more
complex, than those associated with application forwarding itself. Since the deep packet
inspection that these devices perform recognizes complete sessions and keys off protocol
interchange messages, they need to be tested with stateful application traffic that follows
protocol rules.
8
Devices need to be exercised at their limits and beyond to ensure that they will function at
optimum levels and properly apply QoS policies. This type of testing involves the use of a
wide range of multiplay traffic:
•
Data, including HTTP, P2P, FTP, SMTP, POP3
•
Video, including IGMP, MLD, RTSP/RTP
•
Voice, including SIP, MGCP
Modern, large-scale devices and networks can handle hundreds of thousands of sessions
at a time and must be tested under this type of load. A wide variety of measurements must
be made to ensure performance and QoE. These include:
•
HTTP/web response time for loading web pages and content
•
VoIP call setup time and voice quality
•
Consistent and reliable video delivery and quality
•
Video channel change time
•
Peer-to-peer (P2P) throughput
Negative tests must also be applied to ensure that attack traffic is correctly classified and
that it does not affect normal traffic performance. Of particular importance is the testing
of devices and networks under the influence of distributed denial of services (DDoS).
Scalability testing is of particular importance for capacity planning. NEMs must publish
limits that service providers will use and service providers must anticipate future
needs. The number of users, calls, and sessions must be established along with traffic
throughput.
How can Ixia help?
Negative tests must
also be applied to
ensure that attack
traffic is correctly
classified and that
it does not affect
normal traffic
performance.
Ixia’s IxLoad™ solution is specifically designed to minimize test development time, while
maximizing test thoroughness, allowing its customers to get to market faster, with higher
quality, faster performing products. IxLoad thoroughly tests application-aware devices,
measuring performance of all types. A highly-refined GUI makes test development and
result analysis quick, and easy – reducing product and network certification time. Here are
the key benefits related to application-aware device testing:
•
Real-world application traffic mix guarantees that devices and networks have been
tested with the same suite of services that will be used in deployment. IxLoad
emulates the broadest range of application clients and servers in a manner that
carefully models real-world conditions.
•
Subscriber behavior ensures that fielded services will match performance
guarantees. The manner in which subscribers use multiple services is modeled at a
city-level scale.
•
Denial of service attack tests reduce the risk of security failures. High-volume denial
of service attacks are emulated at the same time as real-world application traffic.
•
Ease of use minimizes test development, revisions and retest – ensuring that test time
is minimized for initial product development, product updates and expansion. IxLoad’s
GUI contributes to a highly efficient tool for developing, perfecting, running, analyzing
and reporting application performance tests.
9
Real-world traffic modeling
The Ixia test platform consists of a family of table-top and rack-mounted chassis that
utilize load module interface cards, each of which has a number of individual test ports.
IxLoad utilizes Ixia load modules that support 1 Gbps and 10 Gbps Ethernet, with fiber
and copper interfaces. Each Ixia load module port contains its own CPU with substantial
processing power and memory.
As shown in Figure 5, IxLoad tests devices and networks by emulating clients and servers
surrounding the device or system under test (DUT or SUT). Ixia test ports are connected
on either side of the DUT/SUT and are used to initiate and/or terminate sessions and
send/receive stateful application traffic.
Ixia test ports are
connected on either
side of the DUT/
SUT and are used
to initiate and/or
terminate sessions
and send/receive
stateful application
traffic.
Figure 5. IxLoad test scenario
Real-world, stateful traffic is essential for characterization of devices and networks.
Tests must carefully mimic complete sessions in order to invoke the application-aware
components of the devices they test. Without stateful traffic, there is no guarantee that
devices behave correctly or efficiently. The processing power of Ixia’s ports is used to
emulate protocol clients and servers with complete stateful sessions.
Using large numbers of client and server emulations, IxLoad can emulate a realistic
mix of traffic, using real voice, video and data applications simultaneously. Tests allow
verification that QoS schemes are delivering expected QoS and QoE for each of the
services under the influence of arbitrary mixes of other services. Tests may involve as
few as two or hundreds of test ports to achieve any scale desired. Both the number of
emulated clients/servers and the traffic volume scale with the number of test ports.
10
Subscriber behavior
Real-world traffic involves more than just protocol coverage. Many services cannot be
completely tested without simulating a wide audience of distinct users. It’s important that
end-user emulation reflect patterns of usage. This includes:
•
Multiservice emulation to depict the various application services that are common in
today’s digital home
•
Subscribers’ service usage patterns, including mixed call duration, channel surfing
pattern or Internet use.
•
User-specific behavior including user login for social networking sites or financial
sites.
Ixia provides an elegant mechanism to model subscriber profiles and uses an advanced
timeline to model differing service usage patterns – shown in Figure 6.
Ixia provides an
elegant mechanism
to model subscriber
profiles and uses an
advanced timeline
to model differing
service usage
patterns.
Figure 6. Subscriber Usage Profile
With real-world traffic and subscriber behavior modeling, NEMs can tune and test their
product properly so that they can properly compete and characterize their products for
their customers. They can rest assured that there will be no surprises downstream when
their customers deploy their products. Service providers can likewise feel comfortable
that their networks will satisfy their customers’ QoE demands.
DoS attacks
Denial of service attacks are a critical test that must precede any deployment; failure to do
so can result in network failure, monetary loss and brand damage. DoS attacks are used to
gauge DUT/SUT sensitivity to large amounts of malicious traffic, as shown in Figure 7.
11
The IxLoad GUI
is the ultimate in
ease of use and
power, allowing test
engineers to quickly
develop and run
tests – minimizing
test-related time to
market delays. .
Figure 7. Use of DoS attacks
In this example, baseline tests are run to determine VoIP throughput, latency and jitter for
large numbers of sessions. DoS attacks are then run to ensure that the same performance
characteristics are observed for VoIP traffic.
Ease of Use
The IxLoad GUI is the ultimate in ease of use and power, allowing test engineers to quickly
develop and run tests – minimizing test-related time to market delays. The key component
of an IxLoad test, the traffic model, is supported by a visual drag-and-drop interface –
shown in Figure 8
Figure 8. IxLoad traffic model GUI
12
•
Traffic flow editor to quickly create and manage complex scenarios
•
Resource manager saves test elements for frequent reuse to increase productivity
•
One-click automation enables test scenarios to be automated for regression
IxLoad also provides a graphical display of real-time statistics along with tabular results
for offline analysis. As shown in Figure 9, the overall run results are visually displayed
side-by-side with the details of the small numbers of errors that occurred.
Figure 9. IxLoad side-by-side statistics display
IxLoad Features
•
Highly scalable, integrated test solution
•
Highest traffic rate
•
•
ƒƒ
1 Gbps and 10 Gbps line-rate traffic
ƒƒ
Up to 12 Gbps per chassis
Realistic traffic modeling
ƒƒ
Emulates multiplay clients and servers
ƒƒ
Visual subscriber traffic profile creation
Complete Quality of Experience metrics
ƒƒ
•
Jitter, latency, MOS, PESQ, MDI, TVQM, PEVQ
Widest protocol coverage
ƒƒ
Full range of voice, video, data, security and infrastructure protocols
•
ƒƒ
•
Both IPv4 and IPv6 supported across the board
Quarterly additions of protocols and protocol updates
All-in-one application testing
ƒƒ
Triple-play protocols
ƒƒ
Infrastructure components
13
•
ƒƒ
IPSec/SSL security
ƒƒ
Integrated with router testing in IxNetwork™
ƒƒ
Other Ixia conformance and performance test applications run on the same
platform
ƒƒ
Integration with the Ixia Test Conductor™ regression framework
Ease of use
ƒƒ
Highly visual GUI
ƒƒ
Quickly move from small-scale setup to large-scale testing
ƒƒ
Graphic preparation of user traffic and usage profiles
•
•
•
14
o
Easy to drill-down to errors
Support for modern voice and video technology:
ƒƒ
SIP, MGCP
ƒƒ
H.264, MPEG-4, MPEG-2, IGMP, MLD, RTSP/RTP
Tests critical infrastructure components:
ƒƒ
Authentication: AAA, LDAP and RADIUS services
ƒƒ
IP addressing: DHCP and DNS
ƒƒ
Security: SSL and IPSec
•
Generates malicious and DDoS traffic for security testing
•
Realistic network modeling with impairment and complete TCP parameter control
•
Full data for analysis
ƒƒ
Customizable real-time statistics
ƒƒ
Raw data in CSV files for offline analysis
Conclusion
Multiplay services and security threats mandate application layer intelligence. Application
awareness requires intensive packet processing for deep packet inspection and complex
QoS implementation.
Ixia’s IxLoad is the industry-leading product for layer 4-7 testing of application-aware
devices. It offers:
•
A highly scalable, integrated test solution.
•
Realistic traffic modeling – with emulation of multiplay clients and servers.
•
Highest traffic rate – the only solution with 10 Gbps line-rate traffic.
•
Comprehensive application testing – covering all device testing needs, with triple-play,
infrastructure, security, and router components.
•
Widest protocol coverage – with the full range of voice, video, data, security and
infrastructure protocols.
•
Ease of use – IxLoad’s sophisticated GUI is the ultimate in productivity, quickly
moving from small-scale setup to large-scale testing.
Ixia offers everything on a single, shared platform. Ixia test applications cover the full
gamut of tools for IP network performance testing. Ixia applications also offer the fastest
path to automation, generating automation scripts with the push of a button – that may be
coordinated by the Test Conductor regression tool to create and run complete regression
suites. Ixia platforms have forward and backward compatibility, guaranteeing the longterm benefits of your investments.
15
WHITE PAPER
915-1735-01 Rev. C, January 2014