Avici TSR – An overview
“True scalable routing”
Ides Vanneuville
Systems Engineer - EMEA
Confidential
ArchitectureAvici
forCompany
the 21st
Century Network
Agenda





Introduction
Next (best) generation networking
Software features
Hardware features
Summary
Best generation networking
Who is Avici ?

Founded: 1996 Billerica, MA

Mission: To market scalable core routers that delivers service providers
reduced capital and operational expenditures and simplified operations
without service disruptions

Public Company (Nasdaq AVCI), $ 240 million raised

400 + employees worldwide, 11 in Europe

Gained market share (in last 12 months)
16 M$

2% in core router market (+1G links)
14 M$
12 M$

6.5% in core router market (10G links)
10 M$
8 M$
6 M$
4 M$
2 M$
M$
Q1 00
Best generation networking
Q2 00
Q3 00
Q4 00
Q1 01
Next generation network
Avici
Avici
Optical Core
Avici
Avici
Provider IP Core (P)
Provider Edge (PE)
Customer Premise Equipment (CE)
Best generation networking
Software features
Confidential
ArchitectureAvici
forCompany
the 21st
Century Network
About IPriori

Unicast Routing

Fully featured BGP-4 Implementation
– Confederates, Route Reflectors, damping, policy, Route
Redistribution, Aggregation, dynamic configuration, peer security

IGP Support

IS-IS
– Level 1 and 2 support, Route Redistribution, Database overflow
support, passive interface support

OSPF
– Stub Area support, Virtual links, Router Summarization, Passive
Interface support, Route Redistribution

Multicast Routing


PIM Sparse Intermediate Point
PIM Dense/Sparse, Auto-RP, MSDP, MBGP for mcast NLRI
Best generation networking
Easy Migration - Industry CLI
server-id 2 upper
hostname bay_TWO_UPPER_server
system-password 7 S9bQQdb9Sd
interface Loopback 0
ip address 6.6.6.1 255.255.255.0
no shutdown
interface Ethernet 0
ip address 10.5.222.1 255.255.0.0
no shutdown
module 1/15 4xoc3c
no shutdown
interface pos 1/15/1
ip address 1.1.1.1 255.255.255.0
crc 16
no keepalive
sonet scramble-atm
no shutdown
module 1/17 1xoc48c
no shutdown
interface pos 1/17/1
clock source internal
ip address 10.6.222.7.1 255.255.0.0
no keepalive
sonet scramble-atm
no shutdown
router ospf 100
network 10.10.10.0 0.0.0.255 area 0.0.0.0
router bgp 4293
bgp router-id 1.1.1.1
bgp dampening route-map 1
redistribute connected route-map connected-bgp
redistribute static route-map static-bgp
neighbor ebgp peer-group
neighbor ebgp send-community
neighbor 3.1.1.2 remote-as 200
neighbor 3.1.1.2 peer-group ebgp
neighbor 3.1.1.2 timers 6
neighbor ibgp peer-group
neighbor ibgp remote-as 4293
neighbor ibgp send-community
neighbor ibgp route-map IBGPMAP out
neighbor 1.1.1.2 remote-as 4293
neighbor 1.1.1.2 next-hop-self
neighbor 1.1.1.2 route-reflector-client
neighbor 1.1.1.2 timers 180
neighbor 2.1.1.2 remote-as 4293
neighbor 2.1.1.2 next-hop-self
neighbor 2.1.1.2 route-reflector-client
neighbor 2.1.1.2 send-community
neighbor 2.1.1.2 timers 180
neighbor 2.1.1.2 route-map IBGPMAP out
neighbor 10.10.10.2 remote-as 100
neighbor 10.10.10.2 timers 180
Best generation networking
ip route 5.5.5.0 255.255.255.0 null0
ip community-list 1 deny 45940756
ip community-list 1 deny 46071828
ip community-list 1 deny 46006292
ip as-path access-list 1 permit ^$
ip as-path access-list 100 deny _1_
ip as-path access-list 101 permit _65010_
ip as-path access-list 286 permit _286_
ip as-path access-list 286 permit _3561_
ip as-path access-list 286 permit _701_
route-map EBGPMAP permit 10
set local-preference 25
set metric 150
route-map IBGPMAP deny 10
match as-path 100 286
route-map IBGPMAP permit 20
set community 45940837
set local-preference 80
set metric 10
route-map connected-bgp permit 10
set community 45940847
set origin igp
route-map static-bgp permit 10
set community 45940848
set origin igp
bay 1/1
logging console platform all system bgp
end
Management and Security

SNMP




System event log for all events





Dual Servers can be deployed for additional redundancy
Traffic separation


Protocol activity (tracing)
Local and remote SYSLOG
Log all CLI commands
Standby Route Servers


Standard and vendor-specific MIBS
RMON Events and Alarms Group
64bit counters
All ForMe traffic is classified into 24 application-specific queues serviced by
WRR to avoid starvation of control traffic and ensure against DoS attack
Configurable for ForMe Traffic Filters
Security


TACACS+
Secure Shell
Best generation networking
Network and Traffic Engineering









Leverage emerging D-WDM Technology
Get ahead of the serial bandwidth curve
Stabilize topology
Enable Rapid Provisioning
Simplify the IP routing mesh
Provide Protection options
Provide TE network for the transport of Diff-serv traffic
Enable Circuit Emulation/Private Line Service
Enable 45ms restoration
Best generation networking
Composite Links

OC-48c
OC-48c 10G
OC-48c vOC192
OC-48c



!
interface pos 2/29/1
no ip address
no keepalive
sonet scramble-atm
no shutdown
!
interface composite-link TSR1_CL
ip address 100.100.100.1 255.255.255.0
peer default ip address 100.100.100.2
member-link POS 1/22/1
member-link POS 1/37/1
member-link POS 1/39/1
member-link POS 2/29/1
no shutdown





Combines up to 64 physical
connections to a single logical
connection
Removes serial bandwidth
limitations
Granular bandwidth scaling
4-1 speed mismatch
Preserves packet sequence
Add/remove trunks
dynamically
Recovery in 45ms
Compartmentalizes link failure
Routing table stability
Best generation networking
Composite Links & intelligent optics

Composite Link
3λ
4
TSR monitors load on composite
links

If total premium traffic exceeds a
limit, shifts ports from other links
Optical Switches
Composite Link
4λ
3
• TSR signals optical switch to
reassign wavelength to high traffic
link
• Dynamic reallocation of total
network
Label Switch Router
Label Switch Router
Demonstrated interoperability
Best generation networking
SuperCOM 2001 demo
Best generation networking
MPLS Traffic Engineering

Ingress, Midpoint, and Egress









Penultimate Hop Popping is NOT mandatory
Ingress behavior allows unparalleled tunnel
scaling
Constraint Based Routing with Avici
enhancements
Enhanced RSVP signaling for Label Distribution
Enhanced OSPF and IS-IS to support TE
LDP
End to end TE instrumentation for network mngt
Tight Integration with Composite Links
Hardware label switching on OC-3, OC-12, OC48, OC-192 and Gigabit Ethernet
Best generation networking
Traffic Engineering
Router K's Next
Hop For Routes in
ISP 1
EGRESS
LSR
I
BGP
Backbone
Provider
Shortest Path to
ISPs 1, 2, 3, and 4
Subscriber
ISP 1
K
Router E's Next Hop
For Routes in ISP 2
B
INGRESS
LSR
Incoming Transit
Traffic
E
C
BGP
J
Subscriber
ISP 2
A
G
D
F
Subscriber
ISP 4
Router G's Next Hop
For Routes in ISP 3
EGRESS BGP
BGP LSR
M
Best generation networking
H
Router G's Next Hop
For Routes in ISP 4
Subscriber
ISP 3
Optimized Path Layout

Intelligent MPLS Reflood Timing


Resilience Optimization


Improve responsiveness of Ingress path layout
Recover from MPLS failures with minimal impact to core
topology
Adaptivity Optimization


Take full advantage of positive changes in the core topology
Rebuild LSPs, ordered by required bandwidth
5
0% loaded, 600sec
D
200 sec
TE
Reflood
Timing
4
50% loaded, 180sec
C
100 sec
3
15 sec
90% loaded, 20sec
B
2
95% loaded, 10sec
A
5 sec
1
T=0
Percentage of link loading
Best generation networking
99% loaded, 3 sec
Integration with Composite Trunks
Compose Trunk
Subnet 1
Subnet 1
Subnet 2
Subnet 2
Higher Pre-emption value affects tunnel reroute
Re-route time affected by reservable bandwidth attribute
Best generation networking
MPLS Fast-Reroute Protection
A
B
I
C
H
D
G
E
F
Router switches traffic to pre-configured backup

Allows identification of which traffic to protect

Traffic switched to backup and traffic normally on link contend for resources
using QoS – no traffic is discarded if link is not over-utilized

Works for POS and gigabit Ethernet
Best generation networking
MPLS Fast-Reroute Drafts



draft-atlas-rsvp-local-protect-interop-01.txt (Avici)
draft-swallow-rsvp-bypass-label-01.txt (Cisco)
draft-gan-fast-reroute-00.txt (Juniper)
Draft
Manageability
Control
Scalability
Interop
Local Protect Info (Available
and In-Use) & Label Info at
Ingress. Make-Before-Break
on Backup Assures Protection
Always Available.
Control Backup Paths Merging on Backups.
& LSPs via
Share Bandwidth with
AdministrativeBackups. Bypass Tunnels
Colors, BW, Priorities, Possible for Many Tunnels
& Hop-Limit
Bypass
Local Protect Info (Available
and In-Use) & Label Info at
Ingress. Make-Before-Break
on Bypass Tunnels Only
None
Detour
None
Control Backup Paths Merging on Backups. No
& LSPs via
Bandwidth Sharing. No
AdministrativeBypass Tunnels
Colors, BW, Priorities,
& Hop-Limit
Interoperability
Creates Backups with
any type of Ingress.
As Ingress, supports
signalling required in
all drafts.
Bypass Tunnels Used for Only interoperates
Many Tunnels. Share
with itself (or Interop,
Bandwidth on Simple
depending on
Backups. Backup Merging implementation)
Unspecified
Best generation networking
Only interoperates
with itself (or Interop,
depending on
implementation)
VPN Support: RFC 2547

VPN-P Support



Support for Multi-Protocol Extensions to BGP-4 to support
route-distinguisher
Compatible with Cisco, Juniper, Unisphere, Riverstone,
Alcatel, etc.
Traffic Engineering/VPN tunnel integration

Provide a TE core for Virtual Private Networks
Peering
Node
Peering
Node
Avici
TSR
Optical
Switch
Optical
Switch
Avici
TSR
Customer
Customer
PE
Node
PE
Node
Avici
TSR
Customer
Peering
Node
Optical
Switch
Optical
Switch
Avici
TSR
Customer
Peering
Node
Customer
Best generation networking
QoS Functional Summary







Combination of WFQ, WRR, WRED and GPRA
(CBR)
Programmable ASIC Pipeline
Multiple hierarchy of Algorithms
Many iterations per algorithm implemented
Minimum packet data handling
Functionality increased for Multiservice module
Counters maintained per interface,VC,class traffic
management and accounting information
Best generation networking
QoS Treatment per node
REG
PR
BE
Group Policer
Class Policer
Classifier
Class RED
Class Shaper
Group Shaper
Direct Connect Fabric
CTL
CIR, limit, non-conform actions (drop, flag)
CIR, limit, non-conform actions (drop, flag)
DS Map
minQ, drop probability, non-conform, maxQ
CIR, limit, weight
CIR
Best generation networking
MPLS TE/Composite Trunks/QoS
WRED discards
tunnel two or
hop-by-hop traffic
during tunnel reroute
Compose Trunk
Subnet 1
Subnet 1
Subnet 2
Subnet 2
QoS operates on Diff-serv inferred MPLS exp marking
Best generation networking
Traffic Engineering with QoS

Enables no-loss, low delay variance, low latency
services







Virtual Leased Line
Voice over IP
Storage Area Networks
Add new properties to Service Level Agreements
Improve reliability at reduced cost
Deliver protection appropriate for each service, up-to
SONET APS levels
Improve utilization in network backbone
Best generation networking
Hardware features
Confidential
ArchitectureAvici
forCompany
the 21st
Century Network
The Avici Systems TSR
• Carrier class features
• Carrier class design
• Unprecedented scaling
• Future proof
Best generation networking
Patent-pending Velociti™ Fabric



Toroidal direct connect fabric
Scales to 560 active modules
Each element adds switching & forwarding capacity




Generation 1 - 60 Gbps
Generation 2 - 240 Gbps
Each module connects to
6 other modules
Path diversity through fabric


Resilient
Non-blocking
Best generation networking
Growing the TSR System

Expanding the interconnect

Two physical backplanes in each bay




Power Distribution


Connected top-to-bottom
Connected side-to-side
Connected over-the-top
-48VDC 60A per shelf redundant feed
Environment Controllers

Two per bay interconnected between
bays
Best generation networking
Avici Product Scalability
1 TSR
400 Gbps
6 TSR Bay
2.4 Tbps
4 TSR Bay
1600 Gbps
2 TSR
800 Gbps
1 SSR
200 Gbps
SSR/TSR
600 Gbps
2 SSR
400 Gbps
Best generation networking
TSR™ Interface Modules

Packet over SONET/SDH








Ethernet



Best generation networking
Concurrent IP and MPLS
1 port OC-192c/STM-64
1 port OC48c/STM-16
2 port OC48c/STM-16
4 port OC12c/STM-4
4 port OC3c/STM-1
16 port OC3c/STM-1
2 port GbE
8 port GbE Q2/02
10GbE 2H/02
Module I/O Architecture
Forwarding
Engine
Multicast
Expander &
Packet
Scheduler
Control Processor
FE Interface
Packet Memory
64 MB
E/O
Conversion
SONET
Termination
PPP/MPLS
Processing
Packet
QoS
Best generation networking
BACKPLANE
PPP/MPLS
Processing
Fabric
SONET
Termination
Switch
O/E
Conversion
Software & System Architecture
OSPF
BGP4
IS-IS SNMP
OSPF
BGP4
IS-IS SNMP
MSDP MBGP
PIM
FTP
MSDP MBGP
PIM
FTP
RSVP
CLI
RSVP
CLI
Server Memory 580Mb
Server Memory 320Mb
PowerPC
Processor
PowerPC
Processor
SSD
PCMCIA
HDD
PCMCIA
Fast
Fast
Ethernet
Fast
Ethernet
Fast
Ethernet
Ethernet
Control Processor
FE
Interface
Control
Processor
FE Interface
Console
Console
Switch
Switch
Switch
Switch Fabric
Fabric
Fabric
Fabric
Multicast
O/E
SONET
PPP/MPLS
Forwarding
Expander
&
Multicast
Conversion
Termination
Processing
Engine
O/E
SONET
PPP/MPLS
Forwarding
Packet
Expander
&
Multicast
Conversion
Termination
Processing
Engine
O/E
SONET
PPP/MPLS
Forwarding Scheduler
Packet
Expander
&
Multicast
Conversion
Termination
Processing
Engine
O/E
SONET
PPP/MPLS
Forwarding Scheduler
Packet
Expander &
Conversion
Termination
Processing
Engine
Scheduler
Packet
Control Processor
Scheduler
FE
Interface
Control
Processor
FE
Interface
Control
Processor
FE
Interface
Control
Processor
E/O
Server
InterconnectPacket
SONET
PPP/MPLS
Packet Memory
Conversion
QoS
Termination
Processing
E/O
Packet
SONET
PPP/MPLS
Conversion
QoS
Termination
Processing
64 Mb
E/O
Packet
SONET
PPP/MPLS
Conversion
QoS
Termination
Processing
E/O
Packet
SONET
PPP/MPLS
Conversion
QoS
Termination
Processing
Module
Control Software
Routing and
Configuration
tasks
Best generation networking
Forwarding
and Fabric
Routing tasks
Bay
Bay& &Environment
Environment
Controller
Controller
Field proven operation





First systems deployed in NTON (DARPA
Supercomputer research network)
Operational in the Internet in May 1999 (BGP)
Currently deployed in one of the worlds largest carrier
IP networks
Support OC-192 links
Operating as BGP Route Reflector in one of the
worlds largest RR configurations
Best generation networking
AT&T network
Best generation networking
Summary
Confidential
ArchitectureAvici
forCompany
the 21st
Century Network
Summary - Carrier imperatives

Cost


Velocity


Lower the cost of building and operating the network
Decrease the time to add or change services
Profit

Deliver value added services and quality of service
enhancements
Best generation networking
Lower Operational Costs
Backbone Interfaces
Backbone Interfaces
VS.
High Speed
(Optical)
Low Speed (electrical)
IP Transit
IP VPN / Low Speed Transit
Low Speed (electrical)
High Speed
(Optical)
Customer Facing Interfaces
Current best practice
Best generation networking
Customer Facing Interfaces
Avici alternative
Provisioning - Months to Minutes
10λ
14λ
6λ



Backbone capacity increased without truck-rolls
Market share not impacted by provisioning
Immediate provisioning for all bandwidth products up-to
physical network capacity
Immediate fulfillment of customer demands
Best generation networking
Velocity – Composite Links



Avoid hitting the transmission capacity wall
Backbone links with 64 times the capacity of our
competitor on the same transmission network
No impact to customer or network stability
Backbone Capacity added in days not months
Best generation networking
Profit



Use IP core for converging all services
Scale services past competitors offerings
Spend your time creating new services…


Make €€€€ at the edge of the network
Save €€€€ at the core of the network
Best generation networking
Conclusion

Next generation public network switch





Scalability
Non-stop system and software architecture
Quality of Service
Network and Traffic Engineering
Benefits



Reduce capital costs
Stable service velocity
Increased equipment life cycle
Faster, stable network expansion
& dramatic reduction in cost per bit delivered
Best generation networking
Thank You
Confidential
ArchitectureAvici
forCompany
the 21st
Century Network