White Paper
Cable Network Testing
26601 West Agoura Rd., Calabasas, CA 91302
(818) 871-1800 FAX (818) 871-1805
Email: Info@ixiacom.com Internet at http://www.ixiacom.com
Cable Network Testing
White Paper
Introduction
The testing process for cable modems has become much more complex today as cable networks have
grown into two-way communication systems. As a result, designers and manufacturers of cable modems
have to not only test the forwarding capabilities of the downstream traffic from head-end to subscribers, but
also the upstream forwarding capabilities as more subscribers get on the internet for on-line shopping and
browsing.
The access to the cable modems on the network is controlled by a cable modem termination system
(CMTS), which is located in the premises of the cable operator called the head-end. Traffic is routed from
cable modems to the CMTS, which in turn is routed to the Internet. The traffic routed from the CMTS to
the cable modems is called downstream traffic, and the traffic routed from the cable modems to the CMTS
is called upstream traffic. A cable network is capable of transferring information downstream anywhere in
the range of 10 Mbps to 43 Mbps, and in the upstream direction in the range of 500 kbps to 10 Mbps.
The CMTS and cable modems manufacturers must engage in two levels, or stages, of testing of their
equipment. The first stage involves the verification of the stability and performance of their products,
which incorporates using a packet generation tool. This tool must be able to generate realistic patterns of
traffic at variable rates and packet sizes, along with traffic pertaining to Layer 2 and above. It must be able
to transmit and receive this data at speeds up to the maximum line rates and analyze the data to produce
metrics such as throughput, packet loss, latency and jitter. A typical test setup incorporates a CMTS,
multiple cable modems with Ethernet or USB interfaces, and various test tools such as RF signal generator
and analyzers, and an Ethernet packet generator and analyzer. The packet generation tool must be able to
connect to the Ethernet or USB interface of the cable modem, generate data in the upstream direction, and
analyze the received traffic behavior on the Ethernet interface of the CMTS. Additionally, it must be
capable of generating traffic such as IP, TCP, UDP, ARP, ICMP, and DHCP on the Ethernet interface of
the CMTS and receive it on the Ethernet or USB interface of the cable modem. Other functions that it
should be able to verify are Quality of Service (QoS) and IP multicast.
In the second stage of verification process, the DOCSIS™ compliance of the cable modems with a CMTS
is tested. DOCSIS™ (Data Over Cable Service Interface Specification) is an open standard for cable
modems produced by CableLabs®, formed as a result of the association of major cable operators and
manufacturers. The DOCSIS™-compliance verification stage of the testing process involves the
implementation of DOCSIS™ test procedures, defined in the Acceptance Test Plans (ATPs) in the
Ethernet, Media Access Control (MAC), Operations Support System Interface (OSSI), Backplane Privacy
Interface (BPI) and Physical Layer (PHY) areas. The packet generation tool, in this stage of testing, is used
for verifying protocols implemented in the cable modems such as IP, ARP, ICMP, and DHCP. It is also
used for verifying the QoS, IP Multicast, BPI, and OSS related functionalities. Also, at this stage, it is
beneficial if the packet generation tool can support automation capabilities to expedite the DOCSIS™
staging and certification process. The metrics produced by the test procedures are as outlined in the
specification, such as packet loss, and PASS/FAIL status.
Performance Measurements
There are two major categories that fall under the performance testing arena - Stress testing and
Interoperability testing.
The CMTS is designed to exchange data with hundreds to thousands of cable modems, in upstream and
downstream directions at high transmission rates. On the other hand, the cable modems should be able to
send data upstream to the CMTS and receive data from the CMTS at the maximum rates they are designed
for. Therefore, mechanisms to stress test the CMTS and cable modems must be devised. The Multiple
Service Operators (MSOs) today demand extensive verification of interoperability of equipment supplied
by various CMTS and cable modem vendors.
The following performance metrics are desirable:
Ixia
2 of 7
07/12/01
Cable Network Testing
White Paper
1.
Throughput: The forwarding rates of the CMTS in the downstream direction and the rates of the cable
modem in the upstream direction should be at the maximum. The highest rate at which the traffic can
be forwarded in upstream and downstream directions without any packet loss is called the throughput
of the cable modems and CMTS.
2.
Latency: The processing delay for each data packet must at its minimum. This delay is referred to as
the latency of the packet. The difference between the time a single packet is received by the cable
modem and forwarded to the CMTS in upstream direction, and the time this packet is processed and
sent out the Ethernet interface of the CMTS, is the latency of the packet in the upstream direction.
Similarly, downstream latency is the delay experienced by a single packet received by Ethernet
interface of the CMTS and forwarded to the cable modem.
3.
Packet Loss: The traffic should not be lost for any reason. If the cable modems and CMTS take longer
to process the packets (large latency), then the CMTS or the cable modems are forced to buffer the
packets, thus causing the packets to be dropped. There may be other architectural issues, such as the
address lookup mechanisms in the CMTS, which can also cause packet loss.
4.
Jitter: The delay variance experienced in inter-frame times at the egress point of a cable modem or
CMTS is called jitter. It is caused by the cumulative effects of queuing delays at the various
transmission and switching points along the path of a given session.
In addition to these metrics, it is important to address the class of service offered to the user, such as perflow throughput and latency, as these parameters may affect voice and video applications. Priority-based
traffic must be generated by the packet generation tool, which must be analyzed and reported for QoS
measurements. Traffic generated may be classified based on Layer 2 or Layer 3 parameters such as MAC
addresses, IP addresses, IP Type of Service (TOS), TCP or UDP port numbers.
The traffic profile may also be customized to determine the behavioral model of the cable modems.
Typically, a consumer at home may be downloading large files, while at the same time, sending e-mail or
clicking on a link of a web page. Downloading usually involves sending large size packets in the
downstream direction; whereas e-mail and web page access involves small packets in the upstream
direction. To simulate this scenario, the packet generation tool must be set up to transmit different size
packets in both directions. Another scenario may be to mix the different packet sizes in a data flow. Each
data flow may be transmitted at different rates. The more unpredictable the data pattern becomes, the closer
the test process comes to a typical consumer’s internet access usage patterns. Therefore, the packet
generation tool must be able to generate packets of different sizes, payload patterns (for example,
incrementing and random patterns), and rates (for example, different inter-packet gaps).
Test Setup
Figure 1 shows a typical cable modem and CMTS test setup. Each port of the packet generation tool is
connected to the CPE (Ethernet or USB) interface of the cable modem. The RF interface of the cable
modem is connected to the CMTS. On the other side, the port of the packet generation tool may be
connected to the Ethernet interface of the CMTS directly or via a switch or hub. A DHCP server is also
available in this setup that provides automatic IP address assignments to the cable modems and ports of the
packet generation tool.
To simulate a real world scenario, traffic must be generated from the cable modems to the CMTS
(upstream) and from the CMTS to the cable modem (downstream) simultaneously. Since several cable
modems may be involved in a typical test bed, it is useful to identify each cable modem with an identifier
or name, and the packets generated may contain sequence identifiers or “magic numbers” to distinguish
traffic belonging to each cable modem.
Ixia
3 of 7
07/12/01
Cable Network Testing
White Paper
DHCP Server
Data
Generation
Tool
Switch or
Hub
CM
Downstream
CMTS
RFI
RFI
CM
Upstream
CM
CM
Figure 1: Cable Modem and CMTS Test Setup
The following sequence of steps should be considered when designing a typical bi-directional test for cable
modems.
Ixia
1.
Connect the Ethernet or USB interface of the cable modem to one set of ports of the packet
generation tool, and the Ethernet port of the CMTS to another port of the packet generation tool.
Assign IP addresses to the ports of the packet generation tool such that the corresponding ports of
the cable modem and CMTS lie in the same subnet. Figure 2 shows an example of IP address
assignments. If DHCP server exists as part of the test setup, then send DHCP discover messages
from the ports of the data generation tool connected to the cable modems to the DHCP server in
the network. Verify that IP addresses are assigned to the ports automatically. The IP address of the
port of the packet generation tool connected to the Ethernet interface of the CMTS is usually
assigned statically. If DHCP server does not exist in the network, then assign static IP addresses to
the ports of the data generation tool connected to the cable modems, and to the port of the packet
generation tool connected to the Ethernet interface of the CMTS.
2.
Send ARP requests from the packet generation tool from each port to the cable modems and
CMTS. Note that the Ethernet port of the CMTS will receive the request and return an ARP
response. On the cable modem side, the ARP requests will be passed through the cable modems
(since they are usually bridged devices), and received by the RF port of the CMTS for each cable
modem, which will respond with an ARP response message. As shown in Figure 2 when ARP
request is sent from Port 1 (IP address 10.11.1.52) of packet generation tool, the Ethernet port of
CMTS (IP address 10.11.1.1) will return an ARP response. When ARP requests are sent from
Ports 2 to 5 towards the cable modems, the RF interface of CMTS (IP address 196.12.1.1)
responds with an ARP response message.
3.
Create UDP packets and send them from Port 1 of packet generation tool towards the CMTS
destined for the cable modems (which is received by Ports 2 to 5 of the packet generation tool),
and from Ports 2 to 5 towards the cable modems destined for the CMTS (received by Port 1 of
packet generation tool).
4.
Collect statistics on all ports of the packet generation tool and measure the throughput, latency,
packet loss and other desired metrics by analyzing the number of packets received and timestamp
values.
5.
Report results in an easily interpretable format.
4 of 7
07/12/01
Cable Network Testing
White Paper
Packet generation tool
CMTS
10.11.1.1
Ethernet
Port 1
10.11.1.52
Port 2
196.12.1.100
Port 3
196.12.1.101
Port 4
196.12.1.102
CM3
Port 5
196.12.1.103
CM4
196.12.1.1
RF
CM1
CM2
Figure 2: IP address assignment
DOCSIS™ Testing
DOCSIS™ contains a set of tests that must be executed by CM and CMTS vendors to pass the certification
process. These tests are contained in the Acceptance Test Plan (ATP) documents. These documents very
clearly outline each step required for test execution. DOCSIS™ testing involves the verification of the
function of the protocols and mechanisms between the CMTS and cable modems.
CableLabs® performs rigorous testing on cable modems to certify them. Maximum efficiency is achieved
if the test procedures in the ATPs can be automated. The automation process should include mechanisms to
execute tests in the shortest amount of time, to categorize tests based on their function, and to execute
various test simultaneously. It is in these areas that a sophisticated traffic generation tool can offer
significant benefits.
Since DOCSIS™ deals more with the verification of the function of cable modems rather than
performance, care must be taken when generating data based on these standards. The following areas must
be considered for DOCSIS™-based tests:
•
PHY – Physical layer testing that involves the transmission of data over the physical medium between
the CMTS and cable modems.
•
MAC – The MAC layer protocol is responsible for controlling the initialization, bandwidth allocation
and maintenance of data transfer between the CMTS and cable modem.
•
Ethernet – The data handling capability of cable modems must be verified by analyzing throughput and
packet loss experienced.
•
BPI – Encryption and key management protocol for cable modems that ensures data privacy and theft
protection for MSOs.
•
OSS – Remote management of cable modems that involves verification of SNMP protocol supported
by all cable modems.
There are two versions of DOCSIS™ standards available – DOCSIS™ 1.0, based on which commercial
products are being shipped today; and DOCSIS™ 1.1, which is still is its draft mode. DOCSIS™ 1.1 adds
some new features such as voice, enhanced encryption (BPI+), IP Multicast, and Quality of Service using
packet classification.
Ixia
5 of 7
07/12/01
Cable Network Testing
White Paper
Some of the tools needed for testing are RF signal generators, Logic analyzers, upstream and downstream
sniffers to analyze data passing through the RF channels, and Ethernet data generator and analyzer. A
typical ATP test may involve the following steps:
1.
Range and register the cable modems with CMTS.
2.
Initialize the configuration of cable modems (through SNMP management or downloading a userspecified configuration file from a TFTP server).
3.
Send a fixed number of packets in upstream or downstream direction from a packet generation
tool.
4.
Capture packets using the RF upstream or downstream sniffer.
5.
Capture the packets on the ports of the packet generation tool.
6.
Report a PASS/FAIL status after obtaining packet loss and other desired statistics.
Such a process may apply to Ethernet of MAC test procedures. OSS involves testing the syntax and
semantics of SNMP MIBs implemented in the cable modems. A software tool that can create SNMP
messages may be used for OSS testing. PHY and BPI testing may involve generating certain messages
using the RF signal generator on behalf of the cable modems or CMTS, or it may involve generating
data packets from the Ethernet data generator. The data generator may also be used for IP Multicast and
Quality of Service testing.
Automation of Tests
Automated testing has virtually become a requirement in today’s environment. This is because it is timeconsuming to use graphical user interfaces to configure the thousands of ports of a traffic generator for
transmission and reception of data. Scripts in Tcl or Perl can be created to generate and analyze the
received data in real-time. One added advantage of using automated tests is that the user can stress test the
CMTS and cable modems over longer periods of time and have the script produce results in a readable and
printable format. Additionally, the tests can be executed unattended. This mechanism is suited not only for
system testing but also for QA and manufacturing testing.
Tcl has become the de facto standard in providing the automated testing environment today. Tests written
in Tcl can be modified by the test engineers on the fly and custom scripts can be written and added on to
the provided ones. Scripting gives great flexibility to the test engineers to customize their testing
environment.
The automation of the performance tests provide easy configuration of cable modems, especially when
hundreds of modems are involved in the testing process. Results must be reported in a format that can be
easily interpreted. Tcl scripts must be designed in a way that would provide flexibility to add new features
and easy maintenance in the long run. Since an ATP procedure may contain several sub-sections, it may be
desirable to execute all the sub-sections in sequence, or it may be desirable to categorize the sub-sections
into groups and execute the group of sections in parallel with each other. Such flexibility should be
provided by scripts. This leads to an environment where the tests can be set up to be executed unattended to
generate consistent and reliable results. The other advantage that an efficient framework of scripts can
provide is that they can be integrated into other tools or test executive systems that may provide better
report generation and configuration management capabilities. Due to the nature of DOCSIS™ ATP
procedures, which will only grow in size and number as the demand for cable modems increase, scripts
should take into account this potential growth and provide scalability.
Summary
Cable modem testing not only involves extensive testing in the RF arena but also in the data-handling
arena. After all, it is data that is exchanged among consumers. The CMTS is the gateway to the Internet
that transfers this data and it maintains the path between multiple cable modems and itself by providing
features such as remote management, security, quality of service and bandwidth allocation. The two stages
Ixia
6 of 7
07/12/01
Cable Network Testing
White Paper
of testing of a cable modem system involve performance testing and DOCSIS™ testing. Performance
testing allows CMTS manufacturers to verify load-handling capabilities by analyzing metrics such as
throughput, packet loss, latency and jitter. Interoperability of a particular CMTS with various types of cable
modems must also be verified to pass DOCSIS™ certification. DOCSIS™ involves verification of cable
modem related features such as PHY, MAC, Ethernet, OSS and BPI. Quality of Service and IP Multicast
metrics must also be obtained to verify their behavior under stress conditions. Finally, due to the nature of
cable modem testing, which usually involves hundreds of cable modems interoperating with a CMTS,
automation of the test procedures by the packet generation tool is highly efficient. Automation can easily be
implemented into the test systems by using the most common scripting language today, Tcl. Scripts can
provide automation features such as unattended regression testing to create consistent and reliable results.
About the author – Hardev Soor is a Technical Product Manager at Ixia, the leading manufacturer of
Ethernet Traffic Generator and Performance Analyzer systems used for Cable Modem and CMTS testing.
Hardev Soor can be reached at hsoor@ixiacom.com.
Ixia
7 of 7
07/12/01