Guangzhi Li, Jennifer Yates, Dongmei Wang, Panagiotis Sebos,
Joel Gottlieb, Albert Greenberg, Charles Kalmanek, and Robert Doverspike
AT&T Labs research, Florham Park, NJ 07932, USA
Abstract: We describe our implementation choices and experiences gained from developing a GMPLS-based control plane testbed prototype for end-to-end services. The prototype includes control plane components of routing and signaling protocols and provides rapid provisioning and restoration of connections across multiple optical network domains. Our experience in implementing the multi-domain signaling showed that it was possible to simply adapt the signaling schemes within the OIF UNI specifications. Our testbed measurements achieved average 27ms provisioning time and 16.3 ms restoration time for connections crossing three domains.
Index terms — GMPLS, Testbed, Routing, Signaling,
Restoration
A.
I NTRODUCTION
Traditionally, transport networks are managed by centralized management systems, which receive connection requests, perform path selection, and set up connections through transport network elements.
Recently, the intelligence for transport network functions such as topology and resource discovery, automated provisioning, and failure recovery is being moved into the network elements through the emergence of distributed transport control planes. Proprietary and pre-standard implementations have already been deployed [1], while control plane standards that extend signaling protocols from packet networks are being developed. The Internet Engineering Task Force (IETF) has extended Internet Protocol (IP)-based protocols used in the Multi-Protocol Label Switching (MPLS) control plane to define Generalized Multi-Protocol Label
Switching (GMPLS) [2]. The International
Telecommunication Union (ITU) is reviewing and extending the IETF protocols, along with other alternatives, to form an ITU transport control plane standard, within the Automatic Switching Optical
Network (ASON) architecture.
GMPLS is envisioned as a common control plane managing multiple technologies, including IP / MPLS and a range of transport technologies. However, standards developments have lagged vendor implementations and early carrier deployments, resulting in carriers deploying sub-networks running vendorspecific control planes that generally do not inter-work.
In addition, company mergers and the associated network management integration complexities, scalability concerns, as well as other economic reasons have led to carrier networks that often consist of several control islands, or control domains (CDs) , as shown in
1
Figure 1. In these networks, a CD is a sub-network in which all nodes run a common control plane. To address this multi-domain reality, the ITU transport control plane standard distinguishes between intra-domain and interdomain rather than simply between neighboring network elements of a single domain, as currently defined within the IETF GMPLS specifications. Inter-domain issues can be further classified as within a common trust domain (e.g., within a carrier) or across trust domains
(e.g., between different carriers). Recently, the Optical
Internetworking Forum (OIF) and ITU have been addressing inter-domain issues by developing a
Network-to-Network Interface (NNI) that is designed to provide end-to-end services across different control domains managed by a single carrier. Note that the NNI here always refers to inter-control domain interfaces, and not intra-domain interfaces.
Figure 1. Network consisting of nodes managed with multiple control domains.
As all these standards activities developed, we found that early prototypes and demonstrations were critical to exploring implementation challenges and for ensuring that the standards satisfy the needs of carriers for establishing and managing end-to-end services. This paper describes a prototype transport network test-bed and control plane developed inside AT&T Research. The goals for the test-bed were to gain early experience with
GMPLS protocols, including support for fast restoration, and to investigate distributed control domain interworking.
A number of test-beds have been developed within the
Research and telecommunications industry, typically focusing on the single domain connection establishment and integration with IP and optical networks [3,4,5,6]. A feature topic on GMPLS testbeds will appear on IEEE communication magazine at August 2005 [16]. We have explored these issues, along with how we can extend the developing standards and distributed protocols to

support rapid restoration from connection failures and connection management across multiple control domains. Here a connection is defined as an optical cross-connect supported fixed bandwidth path between source and destination, which is also referred as lightpath in literatures.
B.
S INGLE DOMAIN CONTROL PLANE
The optical network control plane in a single control domain is responsible for tracking the network topology and the state of network resources, and for establishing, removing and maintaining connections. These functions can be achieved using two basic protocols – a routing protocol for topology and resource discovery, and a signaling protocol for connection provisioning, restoration and deletion. Additionally, properties of the neighbor nodes and links may also be shared and negotiated via protocols. This is referred to as neighbor discovery and can be thought of as providing “plug and play” functionality for network elements. Neighbor discovery allows network elements to automatically detect neighbor nodes and the links interconnecting them, reducing the manual configuration required in a bid to simplify network deployments.
We implemented our control plane prototype on
Linux-based PCs that controlled a cross-connect (XC) via an XC control interface. Figure 2 depicts the software modules developed for our control plane prototype. Three functional modules have been developed – routing for topology and resource discovery, signaling for provisioning, restoration and deletion, and controller to XC communication. The first two are running in the user space and interoperate using standard inter-process communication (IPC) mechanisms, while the controller to XC communications is implemented as a kernel module to improve performance.
Control channel
Input ports user level kernel level
Controller (PC)
Routing for topology and resource discovery
IPC
XC
Signaling for provisioning restoration and deletion
Controller -XC protocol
XC control interface
Output ports
CPU, XC control software
1st.
Topology and resource discovery
The transport network topology and resource discovery are achieved using an Interior Gateway
Protocol (IGP) – we used Open Shortest Path First
(OSPF) in our prototype. Nodes flood information about each of their adjacent links throughout the network. Each link may consist of multiple channels - the basic unit for transport resource allocation used in our test-bed. Local node and link resource inventory is initialized within each controller either via manual entry or automated neighbor discovery (see Section B.5 for more details).
This information is maintained locally, keeping track of available resources as connections are established, restored and removed. The local inventory is passed to the OSPF daemon, which floods this information in opaque Link State Advertisements (LSAs) [7] throughout the network. Through this process, each node in the network obtains a complete representation of the network topology and resource utilization. The topology information is utilized for route selection in provisioning and restoration. Based on the GMPLS specification, the link characteristics advertised include link source, destination, type (e.g., SONET, Ethernet), capacity
(number of total, available and reserved channels), a link administration weight, and a list of shared risk link groups (SRLGs) 1 [8].
Ideally, extending existing protocols results in faster development of standards specifications and implementations than developing new protocols. Our prototyping experience was consistent with this ideal - we extended Zebra’s publicly available OSPF implementation [9] to advertise the transport network topology and resource information. The OSPF protocol itself did not require any extensions; as mentioned above, we carry the additional transport network specific information in opaque LSAs supported by standard
OSPF. Opaque LSAs were not available in the original
Zebra OSPF and we had to implement them. We also implemented the communication interface with the signaling module to obtain and share transport information with the signaling process, and mechanisms for constructing the GMPLS opaque LSAs that flood the transport-related information.
An important aspect of our implementation was to ensure that the Zebra software and our optical specific extensions could evolve independently. This is important so that we can easily adopt improved OSPF implementations, leveraging future extensions to the
OSPF protocol and the Zebra implementation. We messages between protocols
IP control packets
Figure 2. Node architecture showing software modules and interfaces.
2
1 A SRLG identifies a group of links that are potentially subject to a common failure (i.e., they share a common risk).

achieved this by minimizing our footprint in Zebra and by using separate modules that were linked with Zebra at compile time.
Overall, we found that OSPF was effective at providing reliable advertisement of the transport network information, and had the added advantage of providing
IP routing in our test-bed control plane using the same process.
2nd.
Provisioning connections
The OSPF module advertises information regarding the state of each link. The signaling module uses this information to select paths for connection provisioning and restoration. Connection provisioning signaling is based on the GMPLS extensions to the ResSource reserVation Protocol with Traffic Engineering extensions (RSVP-TE) [10]. The signaling process is responsible for managing the local resources – i.e. allocating them to connections, maintaining state information, and aggregating resource information for advertisement via OSPF. The local cross-connect state information is maintained within the signaling module to speed up access. Commercial implementations have the added complexity of having to ensure that this state information is stored in non-volatile memory, so that it can be recovered in the event of catastrophic failures.
In our implementation, connection paths are selected in the signaling module, rather than in the routing module. This required that topology information be passed between the signaling and routing modules – the signaling module passes summarized local information to the routing module, whilst the routing module passes information advertised by other nodes to the signaling module for path selection. Although this may require some heavier inter-process communication than if paths were selected in the routing process, it has the attractive property that connection provisioning and restoration can proceed even in the event of a routing module failure, albeit with potentially slightly outdated information.
A fundamental requirement for transport network control planes is that a control plane failure must not affect the data plane. This can become an issue for softstate protocols, such as RSVP-TE. Soft-state protocols do not require reliable signaling – instead, connections are continually refreshed with a lack of refreshes interpreted as corresponding to a connection that is no longer needed and is thus removed. However, control plane failures can occur that do not also fail the data plane – if the RSVP-TE refresh messages are lost due to these failures, then a connection could potentially be erroneously removed – this cannot be allowed in carrier transport networks. To meet the needs of transport networks, the signaling could use a hard-state protocol, with connections not being removed upon loss of refreshes. We achieved this with minimal changes in our prototype. However, GMPLS specifications have taken a different approach, adding extensions to refresh connections downstream of failures upon control channel failures to reduce the likelihood of erroneously removing connections. An even more attractive alternative would be to keep the RSVP-TE refresh, but not delete the connection upon a refresh timeout.
Instead, an alarm would be generated to a management system to alert network administrators of a possible incompletely removed connection. This approach appears to provide the best of both worlds – reliable connections and mechanisms for identifying capacity allocated to unused connections. Although not currently implemented within our test-bed, implementing this capability is expected to require only minimal changes to the control plane software, along with management system support. No changes to the GMPLS specifications are required.
3rd.
Restoration
Topology/resource discovery and signaling for connection provisioning are well defined within GMPLS
– however, restoration has not yet been thoroughly addressed. Although there are numerous proposals for fast restoration in mesh transport networks, we implemented the shared mesh restoration procedure that we proposed in [11,12]. Shared mesh restoration precomputes and reserves the necessary bandwidth for each restorable connection, ensuring that there is adequate available bandwidth to recover from any planned failure event (e.g., we guaranteed recovery from any single
SRLG failure). In the event of a failure, alarms are generated and notification messages are sent to the nodes responsible for triggering the restoration route establishment. The restoration signaling is implemented in the same module as that used for provisioning.
Careful design of the signaling module to support rapid, efficient processing of signaling messages meant that much of the signaling could be used for both provisioning and restoration [13].
Generally, restoration signaling mechanisms include one request message that traverses from source to destination, and then an end-to-end reply from destination to source. For correct operation, it is important to carefully consider the precise timing of control plane messages and the corresponding crossconnections in the data plane. One approach would be to perform cross-connection as each node receives the first request message. This would lead to the fastest restoration time. However, if two XCs on either side of a link attempt to simultaneously allocate the same channel to two different bi-directional connections, performing
3

the cross-connection on the request message can result in two customers being temporarily connected who should not have been – sensitive data could potentially be sent to the wrong customer. This scenario is known as misconnection [17], and is unacceptable for even a short period of time. In our prototype, we avoid misconnection by not cross-connecting the first XC along a connection’s restoration path until it has received an end-to-end reply message. Receipt of this message indicates that all contention has been resolved along the path. All other XCs along the restoration path can immediately perform the cross-connections upon receiving the request messages. This approach could slow down restoration a little, but ensures that misconnection is avoided.
We were able to experimentally demonstrate rapid signaling for the restoration of a limited numbers of connections [15]. However, there are still numerous challenges that must be addressed, including algorithms that efficiently pre-calculate restoration routes without requiring the advertisement of excessive amounts of information, signaling message aggregation schemes to reduce signaling message storms and thus speed restoration, and definition of multi-service priorities to support multiple service types within a single restoration procedure.
4th.
Controller to XC communication
The intelligent transport network node may be implemented using a standalone controller interfacing with the XC through a defined interface, as shown in
Figure 2, or may be an integrated system, in which the controller is part of the XC system. However, the use of an explicit and open interface, such as the IETF’s
General Switch Management Protocol (GSMP), allows interoperability between controller and XC manufacturers, allowing a separation of the control plane from the transport elements. We used our control plane with a range of different hardware platforms – including all-optical and Gigabit Ethernet (GbE) XCs. The flexibility of an explicitly defined interface allowed us to reapply our control plane to these technologies. To achieve this, we used an API in our signaling module that interfaces to XC specific modules that are responsible for communicating with the XC hardware
(i.e., the controller to XC module depicted in the kernel in Figure 2). The API provides simple connect and disconnect commands (with acknowledgments), and alarm reporting.
For some technologies, we were constrained by the control interfaces that the vendors provided. However, we developed a lightweight interface in collaboration with a startup company that was used to communicate between our PC controllers and their Gigabit Ethernet
(GbE) cross-connects. Our protocol provides similar functions to GSMP, but was optimized for the XC interface (as opposed to GSMP which is applicable to many technologies). The protocol was designed to be simple, fast and transparent to the link layer used to communicate with the XC. We implemented our protocol within the kernel of our PC controller, and on the GbE XC processor.
5th.
Configuration management
GMPLS requires a significant amount of configuration information to be provisioned in a node controller, including network element and link addresses, channel mappings between neighboring elements, and SRLGs. In our test-bed, we often needed to change the logical configuration of experimental setups, with different topologies, hardware and applications being utilized and demonstrated. This required that individual network elements be continually re-configured, which initially led to frequent mis-configurations and continual need to debug configuration errors and fiber mis-configurations.
The IETF and ITU are defining neighbor discovery procedures that exchange local port information to automatically identify the port mappings. However, given that our test-bed is significantly smaller than a large-scale operational network and that we often used a single physical cross-connect to emulate multiple logical network nodes (thereby requiring configuration to say which physical interface is associated with each logical node), we determined that such procedures would not provide significant additional functionality to justify implementation.
Instead, we implemented a configuration management system that would allow us to manage multiple configurations, download a network configuration to the relevant network elements, check configuration consistency, verify fiber connectivity and isolate faults for a given configuration. This tool dramatically improved our ability to change configurations, and to debug problems. The web-based interface provided great flexibility, and made it easy to add new functions. Such tools would be critical to other test-beds, testing facilities and network deployments. However, automated mechanisms for discovery of neighboring channel mappings and addresses will still be critical to the management of large-scale carrier deployments. This will reduce the amount of configuration required, but of course does not eliminate all of it (e.g., local node identifiers etc will still need to be configured).
Development of automated mechanisms for identifying
SRLGs will also be required [15] - although they can be identified today from manually entered databases, these databases, like any information source managed via manual processes, are often erroneous.
4

C.
M ULTI -D OMAIN C ONTROL P LANE
A single control plane standard, such as GMPLS, would ideally allow service providers to select equipment from different vendors and have them interoperate. However, GMPLS is only one of a range of competing control plane implementations and standards.
Carriers are deploying networks that operate with multiple control domains (CDs), potentially employing different control plane technologies. The NNI work addressing multi-domain inter-working is developing a standard that will enable automatic construction of endto-end connections with end-to-end service assurances built on the capabilities of individual domains. As an example, Figure 1 illustrates a network with two CDs, interconnected by two pairs of XCs (A
3
-B
3
, A
4
-B
1
).
The essential components of a distributed, standardized NNI solution are signaling and routing protocols to control the exchange of information between domains. We designed and prototyped our own
NNI routing and signaling protocols by extending
GMPLS protocols to investigate the feasibility of end-toend services across multiple control domains.
6th.
Topology/resource discovery with multiple domains
Control domains need some form of topology and/or reachability information regarding other control domains to route connections to remote CDs. But how each control domain topology is summarized when advertised to other CDs can be determined by each carrier, and does not require standardization. In one extreme, the complete local CD topology could be advertised. However, this will not scale to large networks. In the other extreme, a
CD could be represented as a single node. Then, this does not always provide adequate information for efficient routing across CDs.
The summarized transport plane topology can thus be represented as a set of nodes and links. Nodes may correspond to abstract (logical) or physical nodes. The nodes may in turn be categorized as either internal control domain nodes or border nodes. Links include external links and internal links. External links connect border nodes in two different control domains, and are typically physical links. Internal links may be either physical or abstract . Clients associated with each domain are also advertised between CDs.
Routing information is required to support provisioning across control domains. When there are multiple routes out of a given CD in a multi-domain network, connection provisioning across domains requires that the ingress element receiving a connection request be able to select an egress element for a particular destination client. This route selection may differ for different clients and may change in response to changing traffic loads both on the external links, and within CDs. In our prototype, we used an inter-domain routing table to provide the routing information required for routing connections across CDs. Each network element maintains a full intra-domain topology database and an inter-domain routing table. A network element computes the egress element in the inter-domain routing table using the intra-domain topology and the summarized inter-domain topology. This operation necessitates a routing protocol for inter-domain routing information exchange. Reusing our existing GMPLSbased OSPF (referred as intra-domain OSPF instance) was a natural choice. This OSPF instance is referred to as the inter-domain OSPF.
In our inter-domain routing information exchange protocol, one or more node(s) were selected from each
CD as NNI nodes to form an overlay control plane. This is illustrated in Figure 3, where nodes A1, C1, C5 and
B1 are the nodes participating in the overlay NNI control plane running the inter-domain OSPF. Note that the
OSPF syntax does not require the nodes participating in the inter-domain OSPF instance be the border nodes
(e.g., C5 is not a border node) – the LSAs describing the data plane topology simply need to carry the addresses of the border nodes. This is important for many reasons, including allowing centralized management systems to participate in NNI routing for CDs that are centrally controlled. The overlay control plane was used for exchanging inter-domain routing information. We chose to represent our summarized data plane topology using an abstract link between a border node and any other border node which was physically connected to a different control domain (e.g., all abstract links interconnecting nodes C1, C2, C3 and C4 in CD C of
Figure 3). client client
A1 C1 C3
Abstract link
B1
Control plane
Data plane client
5
Figure 3. A multi-domain network example showing interdomain control plane connectivity and the summarized data plane topology
Our implementation of inter-domain routing involves two steps of topology summarization: (1) the intradomain OSPF instance in each node participating in the
NNI routing (i.e., nodes A1, C1, C5, and B1 in Figure 3) summarizes topology information describing its local domain. The inter-domain OSPF instance automatically

synchronizes the summarized topology information by exchanging summary opaque LSAs among the nodes participating in the NNI routing. (2) Each control plane
NNI node combines and further summarizes the summary information it has received regarding other domains. Since each NNI node also participates in the intra-domain routing of its local domain, it passes the further summarized information regarding other domains to the intra-domain routing protocol, where it is flooded within the local domain using summary opaque LSAs.
We implemented both steps of the summarization process by calculating shortest path trees. Detailed routing information exchange and summarized routing information exchange parameters are discussed in [14].
Our inter-domain OSPF implementation used summary opaque LSAs to advertise summarized topology information. We thus extended the OSPF implementation that we used within a single domain, adding in functions to summarize topology information for advertisement to other domains. We also implemented an optical inter-domain routing table within the OSPF process – similar to the routing table calculated for traditional IP hop-by-hop packet forwarding. The optical inter-domain routing table provides entries for each destination, specifying the next inter-domain hop for connections routed across a domain. This enables us to use hop-by-hop routing across domains, as opposed to the strict explicit routing used within domains. The routing information summarization required significant software extensions, as did constructing the inter-domain routing tables.
However, none of these changes impacted the intradomain routing protocol. Thus we achieved inter-domain routing without affecting existing intra-domain routing protocols. In general, our inter-domain routing is able to support any intra-domain routing protocol, which is necessary due to proprietary intra-domain routing protocols and routing protocols based on other standards being developed.
7th.
Provisioning of connections across multiple domains
Customer connections were requested in our test-bed via a User-to-Network Interface (UNI) and established over multiple control domains. The inter-domain routing tables described in Section 3.1 enable hop-by-hop routing across CDs. Using hop-by-hop routing for connection signaling allowed us to reuse the optical UNI
[15] signaling protocol for inter-domain signaling.
Within each domain, we use our single domain GMPLS connection provisioning signaling described in Section
2.2.
As a connection request is received at an ingress point within a control domain, the inter-domain routing table is used to locate the domain egress node for the given destination. Intra-domain signaling combined with local domain topology information is then used to construct the connection within the domain to the egress node.
Because one egress node may connect to multiple ingress nodes within other domains and vice versa, once a connection request has reached an egress node, the inter-domain routing table is then used to select the ingress node of the next domain. Inter-domain signaling communicates the connection request to the next domain. The process continues until the connection request reaches the destination. To illustrate this with a specific example, consider routing a connection across
the network depicted in Figure 3 from the first client
from domain A to the client from domain C. The connection request is received in domain A, and the node receiving this connection request selects a route – either to A1 or A2 using the inter-domain routing tables created as a result of the NNI routing process. Once the egress node of domain A is selected (let’s assume it is
A1 here), then intra-domain routing and signaling protocols are used to establish the connection to A1.
Once the connection request is received at A1, another routing decision is made regarding how the connection is to be established to the next domain – domain C. Interdomain signaling is then used to signal the connection request between domains (e.g., from A1 to C1). Interdomain routing tables are then used to select the egress node from domain C, and so on, until the connection request is successfully signaled to the destination.
Using this approach, the inter-domain signaling protocol between adjacent network border nodes (e.g., ingress/egress nodes) only requires that the destination address for the connection be conveyed; no explicit routing information is needed. UNI signaling satisfies this requirement and can be directly applied for interdomain signaling. Importantly, using this approach, no change is required to the intra-domain signaling protocol. Our experience in prototyping NNI signaling using our existing UNI implementation demonstrated how easily NNI signaling could be adapted from the
UNI.
8th.
Restoration
Recovery from facility failures in our test-bed was achieved using domain-by-domain restoration, where each CD is responsible for restoring the connection segment inside it. External links utilize link protection.
Our shared mesh restoration procedure described in
Section 2.3 was used for restoration within each of the
CDs. Using this approach, restoration signaling after failure is constrained to the domain(s) directly affected by a failure, limiting the size of the resulting signaling storm. It also eliminates the need for exchange of SRLG information across the domain interfaces for restoration,
6

greatly simplifying the route calculation complexity for connection and backup path provisioning. Although multiple control domain links may share a single SRLG, and thus fail simultaneously, the failed connections will be restored without coordination among the CDs.
Domain-by-domain recovery mechanisms cannot be used to recover from border node failures. Instead, in the rare case where domain-by-domain recovery is not adequate, end-to-end connection restoration can be invoked across domains, such as dynamic restoration schemes that essentially re-provision failed connections.
Detail investigation for such an end-to-end reprovisioning restoration scheme is required.
D.
T EST BED AND MEASUREMENTS
Our multi-domain test-bed illustrated in Figure 4 had three CDs and a total of 16 nodes. Each node consisted of a GbE XC and a XC controller. In our test-bed, nodes
A1, A2, C1, C2, C3, C4, B1, and B2 were border nodes and the external links between these were assumed to be
1:1 or 1+1 link protected. There were three clients connected to the test-bed and these were identified with transport network addresses (TNAs). Two clients, TNA1 and TNA3, were connected to domain A, whilst another client, TNA2, was connected to domain B.
Figure 4. Multi-domain AT&T research transport network test-bed
Our XC controllers were implemented using a PC with a 1 GHz Intel Pentium III processor running Linux
(2.2.16 based kernel). Control plane messages were transported out-of-band on a 100 Mb/s Ethernet link.
The protocol described in Section 2.4 provided communication between the controller and the XC over a 10Mb/s Ethernet interface. The time from a crossconnect request being issued to the acknowledgment being received was approximately 4.5 ms.
We used our experimental test-bed to demonstrate rapid provisioning and restoration between clients across multiple domains. The provisioning time for a connection between TNA1 and TNA2 (Figure 4) across three domains, initiated using the OIF UNI [15], was measured to be 27 ms. This time is sufficiently fast for almost all applications. We also demonstrated efficient network routing, with low sensitivity to link loads. We configured each external link with one unit of bandwidth and then established two connections between TNA1 and TNA2. We verified that the two connections were routed over different external links as the link load increased of bandwidth. This experiment verified successful updates of our network inter-domain routing tables in response to changes in traffic loads.
Clearly, our experimental testbed is small relative to large operational networks. However, it is clear that the scalability of the NNI protocol depends on how the intra-domain topologies are represented within the interdomain routing protocol. The approach that we used here of advertising abstract links between border nodes will be effective in networks with relatively small numbers of border nodes interconnecting domains.
However, more efficient means of representing CD topologies will be required to scale to networks with large numbers of CDs with large numbers of border nodes in each CD. This is an important area for future
Research.
To demonstrate domain-by-domain restoration invoked by a single physical failure affecting two domains, we established a connection from TNA1 to
TNA2 over the three domains. The links between A1 and A4, and B5 and B3 passed through an additional XC that we failed to emulate a common failure affecting both domains A and B. Alarms were generated by the
XCs after detection of loss of light (LOL) and were passed to the PCs at nodes A1, A4, B5 and B3. The shared mesh restoration procedure described above was used to recover connections within each domain.
The restoration time was measured using the number of lost packets from a burst of constant bit-rate traffic sent on the established connection. A mean time of
16.3 ms was observed in restoring the connection for the single physical failure affecting the two domains. This time included failure detection, alarm and failure notification processing and forwarding, signaling to reestablish connectivity along the restoration route, and reconfiguration of the XCs within each domain. This time was comparable with the failure recovery times observed for failures affecting only a single domain. We observed that the two domains affected by the common failure were recovered in parallel, which dramatically reduces the connection restoration time compared with end-to-end failure restoration.
To further investigate the applicability of using IP protocols to execute fast restoration within a single CD, we evaluated the performance of the control plane restoration procedure independently of the physical cross-connects, with larger numbers of connections than we could achieve physically [13]. Our experimental measurements demonstrate that IP protocols can be used
7

to rapidly recover moderate numbers of connections.
However, the cross-connect architecture itself can significantly impact the restoration time. It is crucial that multiple cross-connects can be processed in parallel or batch fashion for “slow” cross-connects.
To further validate the correctness of our prototype, we also applied our control planes to commercial optical switches – Ciena CoreDirectors and interworked our
UNI implementation with Cisco optical UNI. Successful experiments demonstrated the feasibility of our control plane architecture and the correctness of our prototype.
E.
C ONCLUSIONS
This paper describes a research test-bed that demonstrates the feasibility of using GMPLS protocols to provide automated topology and resource discovery, fast provisioning and fast restoration for end-to-end services crossing multiple control domains. The fact that
GMPLS is based on standard IP-based protocols which are available in public-domain software allowed us to develop a relatively sophisticated set of network services in the context of a small research project. However, the unique challenges of transport networking required changes to some basic assumptions of the underlying IP signaling protocols. The use of soft-state in RSVP required changes to prevent connections from being deleted on control plane failures. The need for fast restoration required careful attention to contention in order to avoid misconnection. Fast restoration also required careful optimization of the interface between the controller and XC. In a lab environment, good configuration management tools are essential for rapidly and easily reconfiguring the test-bed topology.
Multi-domain inter-working is not necessary in
GMPLS, which assumes a uniform end-to-end control plane architecture. However, the current reality is that service providers need to support inter-working between different control planes. We demonstrated multi-domain inter-working by using OSPF to carry domain-specific, summarized topology information among domains and optical UNI to signal between domains. Our experience in implementing the NNI signaling showed that it was possible to simply adapt the signaling schemes within the UNI specifications. The test-bed also highlighted the ability to implement the significant software extensions required for inter-domain OSPF without impacting in any way the standard intra-domain OSPF routing functions.
References
1.
B. Cortez, “The Emerging Intelligent Optical Network:
Now a Reality,” OFC , 2002.
2.
E. Mannie, editor, “Generalized Multi-Protocol Label
Switching Architecture,” RFC 3945, Oct. 2004.
3.
A. Manzalini, K. Shimano, C. Cavazzoni, A.
D'Alessandro, " Architecture and functional requirements of control planes for automatic switched optical networks: experience of the IST project LION,” IEEE Comms Mag.
,
Vol. 40, No. 11, Nov. 2002.
4.
K. Liu, L. Changdong, J. Pastor, A. Roy, J. Wei,
“Performance and test-bed study of topology reconfiguration in IP over optical networks,” IEEE Trans.
On Comms , Vol. 50, No. 10, Oct. 2002.
5.
A. Richter, W. Fischler, A. Buchwieser, T. Buchberger,
B. Stilling, B. Lankl, " Field trial of an ASON in the metropolitan area,” IEEE/LEOS OFC , March 2002.
6.
C. Xin, Y. Ye, T. Wang, S. Dixit, C. Qiao, M. Yoo, “On an IP-centric optical control plane”, Communications
Magazine, IEEE , Vol. 39, No. 9 , Sept. 2001
7.
D. Katz et al.
, “Traffic Engineering Extensions to OSPF
Version 2,” RFC 3630, Sept. 2003 .
8.
P. Sebos et al.
, "Auto-discovery of Shared Risk Link
Groups," OFC , 2001.
9.
“A Routing Engine Software Based on the GNU General
Public License”, www.zebra.org.
10.
L. Berger, editor, “Generalized MPLS Signaling:
Resource ReserVation Protocol – Traffic Engineering
(RSVP-TE) Extensions,” RFC 3473, Jan. 2003.
11.
S. Chaudhuri et al ., "Control of lightpaths in an optical network," OIF2000.04
, 2000.
12.
G. Li et al., “Efficient Distributed Restoration Path
Selection for Shared Mesh Restoration,” IEEE/ACM
Transaction on Networking, pp 761-771, vol 11, no 5,
Oct, 2003.
13.
G. Li et al ,. "Experiments in Fast Restoration using
GMPLS in Optical / Electronic Mesh Networks," Post deadline Papers Digest, OFC 2001 .
14.
D. Wang et al.
, "OSPF for Routing Information Exchange
Across Metro/Core Optical Networks," Optical Networks
Magazine , Sept/Oct, Vol 3, 2002.
15.
B. Rajagopalan, editor, "User Network Interface (UNI)
1.0 Signaling Specification," OIF2000.125
, 2000.
16.
M. Gurusamy et al.
editors , “Optical Networking
Testbeds: Experiences, Challenges, and Future
Directions,” IEEE Comm. Mag. Feature Topic, CFP, www.comsoc.org/pubs/commag/cfpcommag805.htm
,
Dec. 2004.
17.
J. Yates and G. Li, "Challenges in Intelligent Transport
Network Restoration,” invited paper, OFC 2003.
8