T
H
E
TO L LY
G R O U P
No. 205139
December 2005
Tasman Networks, Inc.
Test
Summary
1002 and 1004 Routers
Competitive Performance Evaluation versus Cisco
1841, 2811 and 2821 Integrated Service Routers
Premise: Wide-area network routers
that provide T1/E1 access for branch
offices and others to enterprise and
service provider networks must be
able to deliver high throughput, even
with Quality of Service, IPSec VPN
Security Services, Network Address
Translation or other services active
and vying for processor cycles.
Test Highlights
Tasman 1004 Router operated at or near wire-speed throughput and
outperformed Cisco 2811 and 2821 routers, delivering 6X and 2X more
throughput respectively, while simultaneously supporting active Quality
of Service, Access Control List filters and Network Address Translation
over four T1 lines
Tasman 1002 and 1004 Routers demonstrated wire-speed performance
for most packet sizes tested while simultaneously supporting active
QoS, IPSec VPN and stateful firewall services over two or four T1 lines
T
Tasman 1004 consistently outperformed the Cisco 2811 for all the
packet sizes tested, especially at smaller packet sizes, when tested
across four T1s with QoS, IPSec VPN and stateful firewall services,
delivering 3X more throughput than its counterpart
asman Networks, Inc. commissioned The Tolly Group to evaluate its Tasman 1004 and Tasman
1002 wide-area network routers with
integrated network services such as
Quality of Service (QoS), IPSec VPN
with on-board hardware acceleration,
stateful firewall, Network Address
Translation (NAT) and Access Control
Lists (ACLs) for enterprises and service providers.
To test IPSec VPN throughput over a
single 3DES/SHA1 tunnel with QoS
and stateful firewall enabled, engineers tested the Tasman 1002 against
the Cisco 1841 and the Cisco 2811 for
Layer 3 throughput with multilink
PPP traffic riding over two T1s. The
© 2005 The Tolly Group
4xT1 Multilink PPP (MLPPP) Aggregate WAN
Layer 3 Throughput Zero-Loss Performance
with QoS/ACL/NAT Enabled
Throughput (Mbps)
Tolly Group engineers measured the
multilink Point-to-Point Protocol
(MLPPP) zero-loss throughput of the
Tasman 1004 router against Cisco
2811 and Cisco 2821 routers, with
QoS, NAT and ACL features enabled
in a scenario with multilink PPP traffic riding over four T1s.
Tasman 1002 achieved wire-speed throughput at all packet sizes,
while performance of Cisco 2811 and 1841 weaken when handling 64128- and 256-byte packets tested across two T1s with QoS, IPSec
VPN and stateful firewall services
6
5.0
5
6.2
6.2
4.8
3.8
4
3
6.2
6.1
5.9
5.8
4.8
3.6
4.7
3.6
3.0
2.5
2
1.8
1.4
0.9
1
0
64
128
256
512
1024
1400
Packet Sizes (Bytes)
Tasman 1004
Source: The Tolly Group, September 2005
Cisco 2821
Cisco 2811
Figure 1
Page 1
The Tolly Group
Tasman Networks
1002 and 1004 Routers
4xT1 Multilink PPP (MLPPP) Aggregate WAN Layer 3
Throughput Zero-Loss Performance with
QoS/VPN/Firewall Enabled
Throughput (Mbps)
6.2
6.0
5.8
6
5
4
4.1
3.9
3
2.2
1.9
2
1.1
1
0
64
128
256
512
Packet Sizes (Bytes)
Tasman 1004
Cisco 2811
Source: The Tolly Group, September 2005
Tolly Group also tested the Layer
3 IPSec VPN throughput of the
Tasman 1004 versus Cisco 2811
in a scenario with multilink PPP
traffic riding over four T1s. All
tests were performed in
September/October 2005. The
Cisco 2811 and Cisco 2821
routers were installed with an
optional IPSec VPN hardware
acceleration module, while the
Cisco 1841 router and Tasman
1002/1004 routers had on-board
hardware acceleration support.
Test results show that the Tasman
1004 and 1002 routers deliver
superior throughput for the
majority of packet sizes tested,
especially with regards to smaller
packet sizes (64 bytes to 256
bytes), delivering up to 6.4X
greater throughput than the Cisco
devices tested.
© 2005 The Tolly Group
Figure 2
Results
Zero-Loss Throughput
over Four T1s
Tolly Group engineers tested the
Tasman 1004 router against
Cisco 2811 and Cisco 2821
routers to measure the fullduplex, zero-loss aggregate WAN
throughput across a point-topoint router connection equivalent to a group of four T1 circuits
combined using MLPPP under
two scenarios:
with QoS/Network Address
Translation(NAT)/ACL enabled
on the devices.
with QoS/IPSec VPN/stateful
firewall enabled on the devices
With QoS, NAT and ACLs
enabled, the Tasman 1004 router
delivered wire-speed throughput
for packet sizes greater than 256
bytes, and delivered more than
95% of wire-speed throughput
for the more taxing 64- and 128byte packet sizes. On the other
hand, the Cisco 2811 router
delivered significantly poorer
throughput compared to Tasman
1004 router at all packet sizes
tested. The Tasman 1004 router
outperformed the Cisco 2811
router by achieving 1.7X to 6.4X
greater throughput at all the
packet sizes tested, most notably
at the taxing smaller packet sizes
of 64 to 512 bytes (See Figure 1).
Moreover, the Tasman 1004
delivered anywhere from 32% to
more than 130% greater throughput than the Cisco 2821.
With QoS, IPSec VPN and stateful firewall enabled, the Tasman
1004 router was tested against
the Cisco 2811 router. Results
show the Tasman 1004 router
achieving either wire-speed or
near wire-speed throughput,
Page 2
The Tolly Group
depending upon the packet sizes
tested. In comparison, the Cisco
2811 router could only achieve
near wire-speed throughput for
the large packet sizes of 1,024
and 1,400 bytes. For the remaining packet sizes, the Tasman
1004 router outperformed the
Cisco 2811 router by achieving
1.5X to 3.5X more throughput,
particularly at the 64- and 128byte packet sizes, where the
Tasman 1004 router demonstrated more than 3X the throughput
of the Cisco 2811 router (See
Figure 2).
Zero-Loss Throughput
over Two T1s
Tolly Group engineers tested the
Tasman 1002 router against the
Cisco 2811 and the Cisco 1841
routers to measure the fullduplex, zero-loss aggregate WAN
throughput across a point-topoint router connection equivalent to a group of two T1 circuits
combined using MLPPP with
QoS, IPSec VPN and stateful
firewall services enabled on the
devices.
With QoS/IPSec VPN/stateful
firewall enabled, the Tasman
1002 router achieved wire-speed
throughput in all six packet scenarios with full-duplex traffic traversing a pair of T1 links supporting MLPPP. In contrast, the
Cisco 1841 router offered near
wire-speed performance only for
the two largest packet sizes tested
- 1,024 and 1,400 bytes, while at
other packet sizes the Tasman
1002 router outperformed the
Cisco 1841 by achieving 1.3X
(with 512-byte packets) to as
much as 4.4X (with 128-byte
packets) greater throughput (See
Figure 3). Similarly, the Cisco
2811 router achieved wire-speed
© 2005 The Tolly Group
Tasman Networks
throughput only for 512, 1,024
and 1,400-byte packet sizes,
while the Tasman 1002 router
consistently outperformed the
Cisco 2811 router at packet sizes
of 64 to 256 bytes with 1.2X to
3.4X greater throughput (See
Figure 3).
Analysis
Testing shows that the Tasman
Networks 1004 and 1002 routers
possess an abundance of processing headroom to accommodate
network services while simultaneously offering wire-speed
throughput.
Tests show that the Tasman
1004/1002 router can deliver
wire-speed throughput at most
packet sizes tested, while simultaneously processing a combination of QoS, NAT, ACL filters,
IPSec VPN and firewall services.
By contrast, tests show that the
performance of the Cisco
1841/2811/2821 routers sag
under the processing load, especially when smaller, more taxing
packet sizes come into play.
In head-to-head testing, the
Tasman 1004/1002 routers consistently deliver better performance than the Cisco
1841/2811/2821 routers under
almost all tested scenarios - at
times delivering up to 6.4X the
throughput of the Cisco devices
tested (See Figure 1).
Test Configuration
and Methodology
For the performance tests, The
Tolly Group engineers measured
the maximum zero-loss ( 0.001%
acceptable frame loss) Layer 3
throughput using Spirent
SmartBits and SmartFlow. For
1002 and 1004 Routers
Tasman
Networks, Inc.
Tasman 1004,
Tasman 1002
Zero-Loss,
Layer 3 Throughput
Tasman Networks
1004 and 1002 Routers
Product Specifications*
1004/1002 Router hardware features
Half rack-width 1U chassis (rack mount
option available), Two 10/100 Ethernet
ports, 256MB RAM, console and AUX
ports
4 T1/E1 ports (1004) and 2 T1/E1 ports
(1002)
T1/E1s can be remotely enabled using
software license keys
On-board VPN encryption acceleration
chip
Wire-speed hardware platform with
services enabled
TiOS advanced software features
Routing (RIP, OSPF, BGP4, PIMSM/SSM, IGMP)
Layer 2 features (VLAN tag/forward, Qin-Q, GRE)
Services such as sophisticated Multi-class
QoS, QoS on Layer 2 traffic
Stateful firewall (60+ DDoS protection,
30+ ALG including SIP/H323)
IPSec VPN (site-to-site, remote access,
DES/3DES/AES, MD5/SHA1)
Industry standard CLI and intuitive GUI
for ease of management
For more information contact:
Tasman Networks, Inc.
5400 Hellyer Ave,
San Jose, CA 95138 USA
Phone: (408) 216-4700
Fax: (408) 216-4701
URL: http://www.tasmannetworks.com
*Vendor-supplied information not verified by
The Tolly Group
Page 3
The Tolly Group
Tasman Networks
1002 and 1004 Routers
2xT1 Multilink PPP (MLPPP) Aggregate WAN Layer 3
Throughput Zero-Loss Performance with
QoS/FW/VPN Enabled
3
3.1
3.1
3.1
3.1
3.0
Throughput (Mbps)
2.6
2.4
2.5
2
1.5
1.5
1
1.1
0.9
0.8
0.7
0.5
0
64
128
256
Packet Size (Bytes)
Tasman 1002
Cisco 2811
512
Cisco 1841
Note: Cisco ISR 1841 & 2811: IOS 12.4.2, 1841 with on-board VPN chip, 2811 with AIM-VPN/EPII-PLUS VPN module
Source: The Tolly Group, September 2005
the 4XT1 Multilink PPP IPSec
VPN throughput tests, engineers
tested a Tasman 1004 outfitted
with two 10/100 Base-T Fast
Ethernet ports and up to four T1
ports. The Tasman 1004 was running OS Ver 8.2.1/ BootROM
Ver. T1k031605. Engineers tested
the Tasman 1004 against a Cisco
2811 Integrated Service Router
configured with an AIMVPN/EPII-PLUS VPN module
for VPN acceleration, and two
dual T1 WIC modules and dual
auto-sensing 10/100Base-T Fast
Ethernet interfaces. The device
was running OS Ver 12.4.2T1/
BootROM Ver. 12.3(8r)T7.
The Tasman 1004 also was pitted
against the Cisco 2811 Integrated
Service Router described earlier,
along with a Cisco 2821
Integrated Services Router in
tests with QoS/ACL/NAT
© 2005 The Tolly Group
Figure 3
enabled on the devices. The
Cisco 2821 Integrated Services
Router was running OS Ver
12.4.2T1/ BootROM Ver.
12.3(8r)T7, with AIM-VPN/EPIIPLUS VPN module, two dual T1
WIC modules and was configured with two integrated
10/100/1000 Ethernet ports.
For the 2XT1 Multilink PPP
IPSec VPN throughput tests, the
engineers tested a Tasman 1002
Router running OS Ver 8.2.1/
BootROM Ver. T1k031605
against the Cisco 2811 Integrated
Services Router (configured as
mentioned earlier) and a Cisco
1841 Integrated Services Router
configured with two Integrated
10/100 Fast Ethernet ports, and
running OS Ver 12.4.2T1/
BootROM Ver. 12.3(8r)T8. The
Cisco 1841 Integrated Services
Router had an on-board chip for
IPSec VPN acceleration while
the Cisco 2811 was configured
with AIM-VPN/EPII-PLUS VPN
module. The Cisco routers
utilized a dual T1 WIC module.
Each router was tested for its
Layer 3 throughput while connected back-to-back with an
identical router using either four
or two T1 WAN connectors. The
WAN protocol used on the WAN
link was Multilink Point-to-Point
Protocol (MLPPP) over multiple
T1 links. The throughput was
measured using 64-, 128-, 256-,
512-, 1,024- and 1,400-byte
packet sizes under two scenarios:
with QoS/IPSec VPN/stateful
firewall enabled on the devices
for the 4xT1 tests
with QoS/Network Address
Translation(NAT)/ACL enabled
Page 4
The Tolly Group
Tasman Networks
1002 and 1004 Routers
Test Bed
Test Scenarios :
- QoS + ACL + NAT
- QoS + Firewall + VPN
WAN:
2xT1 or 4xT1
Multilink PPP
Device Under Test
Traffic generated
by SmartBits 600
- 64, 128, 256, 512,
1024, and 1400-byte
packets
Device Under Test
Spirent SmartBits 600
Source: The Tolly Group, September 2005
Figure 4
on the devices for the 2xT1 tests.
HTTP (port 80) traffic for DSCP 00.
For both scenarios, the QoS configuration consisted of four traffic classes (DSCP 56, DSCP 44,
DSCP 32 and DSCP 00) on each
DUT. Spirent SmartBits 600 test
traffic generator was used to generate 100 UDP packet flows with
QoS priority bits configured as
follow: 25 flows with Telnet (port
23) traffic for DSCP 56 (highest
priority), 25 flows with FTP (port
21) traffic for DSCP 44, 25 flows
with FTP-DATA (port 20) traffic
for DSCP 32 and 25 flows with
For the QoS/NAT/ACL tests, in
addition to the QoS configuration
as described above, the engineers
configured 100 static IP addresses translation (NAT) on each
DUT, along with 100 ACLs (first
99 blocks and last filter accepting
any IP traffic) based on source
and destination IP addressing
scheme.
© 2005 The Tolly Group
Traffic generated
by SmartBits 600
- 64, 128, 256, 512,
1024, and 1400-byte
packets
For the QoS/IPSec VPN/Stateful
firewall tests, in addition to the
QoS configuration as described
earlier, engineers configured a
single site-to-site IPSec VPN tunnel with 3DES encryption and
SHA1 authentication enabled and
10 stateful firewall rules (first 9
blocks and last rule accepting any
IP traffic).
Each DUT was connected backto-back with an identical router
using either two or four T1 WAN
links, as appropriate for the test
case. The LAN ports on the DUT
at each end were connected to the
appropriate number of 10/100/1000
Ethernet ports on the SmartBits.
Page 5
The Tolly Group
See Figure 4 for the test bed illustration. Spirent SmartFlow (ver.
4.60) was used to configure the
tests. In the SmartFlow setup, the
test duration was set as 60 seconds, acceptable frame loss percentage as less than or equal to
0.001%. Binary search algorithm
was used. Using these settings in
SmartFlow and the DUT configured as required for the desired
WAN scenario, each test was run
three times, and the results averaged to obtain the final throughput.
Tasman Networks
1002 and 1004 Routers
Equipment
Acquisition and
Support
The Cisco Integrated Service
Routers with latest hardware versions (Models: 1841, 2811 and
2821) were acquired through normal product distribution channels. The Tolly Group contacted
executives at Cisco Systems and
invited them to provide a higher
level of support than available
through normal channels. Cisco
never responded to the offer.
The Tolly Group gratefully acknowledges the providers of test equipment used in this project.
Vendor
Spirent Communications
Spirent Communications
Product
SmartBits 600
SmartFlow ver 4.60
Web address
http://www.spirentcom.com
http://www.spirentcom.com
Terms of Usage
USE THIS DOCUMENT ONLY IF YOU AGREE TO THE TERMS LISTED HEREIN.
This document is provided, free-of-charge, to help you understand
whether a given product, technology or service merits additional investigation for your particular needs. Any decision to purchase must be based
on your own assessment of suitability.
This evaluation was focused on illustrating specific features and/or performance of the product(s) and was conducted under controlled, laboratory conditions and certain tests may have been tailored to reflect performance under ideal conditions; performance may vary under real-world
conditions. Users should run tests based on their own real-world scenarios to validate performance for their own networks. Commercially reasonable efforts were made to ensure the accuracy of the data contained
herein but errors and/or oversights can occur.
The test/audit documented herein may also rely on various test tools the
accuracy of which is beyond our control. Furthermore, the document
relies on certain representations by the sponsor that are beyond our control to verify. Among these is that the software/hardware tested is production or production track and is, or will be, available in equivalent or
better form to commercial customers.
The Tolly Group provides a fee-based service to assist users in understanding the applicability of a given test scenario to their specific needs.
Contact us for information.
When foreign translations exist, the English document is considered
authoritative. To assure accuracy, only use documents downloaded
directly from The Tolly Group's Web site.
Project Profile
Sponsor: Tasman Networks, Inc.
Document number: 205139
Product Class: WAN router
Products under test:
Tasman Networks 1004 OS Ver 8.2.1/ BootROM
Ver. T1k031605
Tasman Networks 1002 OS Ver 8.2.1/ BootROM
Ver. T1k031605
Cisco 1841 Integrated Services Router OS Ver
12.4.2T1/ BootROM Ver. 12.3(8r)T8
Cisco 2811 Integrated Services Router OS Ver
12.4.2T1/ BootROM Ver. 12.3(8r)T7
Cisco 2821 Integrated Services Router OS Ver
12.4.2T1/ BootROM Ver. 12.3(8r)T7
Testing window: September 2005
Software status:
Generally available
For more information on this document, or other services
offered by The Tolly Group, visit our World Wide Web site at
http://www.tolly.com, send E-mail to sales@tolly.com, call
(561) 391-5610.
Information technology is an area of rapid growth and constant change. The Tolly Group conducts engineering-caliber testing in an effort to provide
the internetworking industry with valuable information on current products and technology. While great care is taken to assure utmost accuracy, mistakes can occur. In no event shall The Tolly Group be liable for damages of any kind including direct, indirect, special, incidental, and consequential
damages which may result from the use of information contained in this document. All trademarks are the property of their respective owners.
The Tolly Group doc. 205139 rev. dmk 05 Dec 05
© 2005 The Tolly Group
Page 6