Rough Outline for a Intra-Portal Protocol
Version 02
Stephen Haddock
August 23, 2012
1
Configuration
• Configuration vs Discovery
– Version 01 of this presentation observed that it would be
theoretically possible for the Intra-Portal Protocol to follow the
philosophy used to develop LACP to maximize discovery and have
minimal configuration.
– Version 01 recommended against this approach.
– Feedback was overwhelmingly in favor of configuration over
discovery.
– Still need to decide how much information needs to be configured
and what can be negotiated by protocol.
– Still need a state machine to establish communication between
Intra-Portal Protocol participants, verify correct configuration and
connectivity, control the “Emulated System” when communication
has been established, detect loss of communication between IPP
participants, etc.
2
What needs to be configured?
• Portal Identifier for each Portal
– Needs to be the same in all systems in a portal.
– Needs to be unique within each system of a portal.
– Could be the same as the System ID for the Emulated System
• Pros and cons of this to be discussed later.
– Used to detect mis-connections of Intra-Portal Links
– In systems with multiple Portals, used to assure Portal Ports and
Intra-Portal Ports get associated with the correct Portal.
• Intra-Portal Port(s) for each Portal
– Could be physical or virtual ports.
• Portal Port(s) for each Portal
– These are the ports on each system that will become ports on the
Emulated System for the Portal.
3
Configured/Negotiated Parameters -1
• System ID for Emulated System of each Portal
– Needs to be “globally” unique (i.e. unique across all interconnected networks).
• Could be configured, but leaves no way to resolve conflict in case of
misconfiguration
– Propose the each system in a Portal advertise a candidate Emulated
System ID.
• Numerically lowest candidate is used as the System ID.
• This also proposal also allows a means to negotiate other Emulated System
parameters (e.g. Emulated System Port IDs).
• LACP Key
– Needs to be unique within an Emulated System.
– Indicates aggregation capability
• Ports in a system with the same Key can be aggregated together.
4
Configured/Negotiated Parameters - 2
• Portal Port Identifiers
– Need to be unique within an Emulated System.
• Could be configured, but leaves no way to resolve a conflict if there is a
misconfiguration.
• Propose that each Portal System assign Port IDs that are unique within the
system and have the MSB = 0. When the Intra-Portal Protocol establishes the
Emulated System, the system with the numerically higher candidate System
ID sets the MSB of it’s Port IDs to 1. (This concept can be extended to Portals
with more than two systems.)
5
State Machine
• Suggestion received during presentation in San Diego was
to add a “Divorced” state.
– Instead of going from Estranged back to Single when all
communication with spouse is lost, go to Divorced state.
– Difference between Divorced and Single is that in Divorced you
still remember some history of being married.
• Presumably after some long timeout, the memory fades and you transition to
Single.
– Divorced state could be beneficial for handling potential split-brain
scenarios, and for implementing the Graceful Name Change
proposal.
6
Version 01 slides
7
LACP in a Nutshell
• Each Port on a System advertises:
1.
A System ID
•
2.
A Key
•
3.
The same System ID value is used for all Ports on a System.
The Key is an indication of aggregation capability. Ports that can be
aggregated advertise the same Key value.
A Port ID
•
The Port ID is included to handle some special cases. It is not important for
a high level understanding of basic LACP concepts.
• LACP Selection Logic will form an aggregation between
any ports that:
1.
2.
Advertise the same System ID and the same Key (called the
Actor_System and Actor_Key), and
Receive advertisements containing the same System ID and the
same Key (called the Partner_System and Partner_Key)
8
Two Systems without Distributed Aggregation
System A
Port
Port
Port
System B
Port
Port
Each Port on System A advertises:
1. Actor_System = A
2. Actor_Key = Ax
Where Ax may be the same value
on some or all of the ports, or
may be a different value on
different ports.
Port
Port
Port
Port
Port
Each Port on System B advertises:
1. Actor_System = B
2. Actor_Key = Bx
Where Bx may be the same value
on some or all of the ports, or
may be a different value on
different ports.
9
Two Systems with Distributed Aggregation
System A
Port
Port
System B
Port
Port
Port
Port
(possible) Network Link
Intra-Portal Link (could be virtual)
Network Link
Gateway
Link
(virtual)
Emulated System C
Port
Port
Each Network Port on System A advertises:
1. Actor_System = A
2. Actor_Key = Ax
Port
Port
Port
Port
Port
Port
Network Link
Gateway
Link
(virtual)
Each Network Port on System B advertises:
1. Actor_System = B
2. Actor_Key = Bx
Each (non Gateway) Port on System C advertises:
1. Actor_System = C
(C could be the same as A or B, but does not need to be)
1. Actor_Key = Cn
Where Cn is the same value on all of the ports.
10
Intra-Portal Protocol
Creating and Maintaining a Distributed Aggregation requires:
• State machines in System A and System B (the “real”
systems) to control the transitions between the state
without distributed aggregation and the state with
distributed aggregation.
• A protocol that
1.
2.
3.
Determines the System ID and Key values for the Emulated
System C.
Coordinates the Selection Logic for the Emulated System C.
Coordinates the distributed aggregation state machines in each of
the “real” systems.
11
Configuration versus Discovery
• LACP designed to allow minimal configuration and
maximal discovery:
– A default configuration is all ports advertise the same key value.
– LACP will then discover all groups of ports connecting the same
pair of systems and automatically aggregate them.
• The Intra-Portal Protocol could conceivably follow the
same philosophy:
1.
2.
3.
•
Advertise ability to do distributed aggregation on all ports.
Form an Intra-Portal Link to any connected system that is also
capable of distributed aggregation, and create an emulate system
between them.
Use LACP (possibly with enhancements) to discover links that
could form a distributed aggregation and “move” those ports to
the emulated system.
This is quite ambitious!
12
A less ambitious starting point
1. Configure the port(s) that are expected to form IntraPortal Links.
–
Use protocol advertisements on those ports to verify that the
other system also expects these to form Intra-Portal Links, and to
agree on Emulated System parameters (System ID, Key value,
Port IDs).
2. Configure which port(s) are expected to be “moved” to
the Emulated System.
–
–
Once the Intra-Portal Link is active and Emulated System
parameters agreed, use LACP to advertise the Emulated System
parameters these ports.
Intra-Portal Protocol coordinates the Selection Logic of the
Emulated System to form the distributed aggregation.
13
Rough Distributed Aggregation State Machine
Begin
SINGLE
•LACP advertises “real” system
parameters on all ports.
•Send Intra-Portal Protocol
advertisements (candidate
Emulated System parameters)
on potential Intra-Portal Port.
All communication
with spouse lost
ESTRANGED
•LACP advertises Emulated
System parameters on Emulated
System ports.
•Send Intra-Portal Protocol
advertisements on Intra-Portal
Port and alternate paths.
•Maintain Distributed
Aggregation
IPPort Operational &
IPProtocol
Advertisements
received
All communication
with spouse lost
All communication
with spouse lost
IPPort Operational &
IPProtocol
Advertisements
received
IPPort not Operational
but communication
through other paths
possible
BETROTHED
•LACP advertises “real” system
parameters on all ports.
•Send Intra-Portal Protocol
advertisements on Intra-Portal
Port.
Not in
sync
In sync (Emulated
System parameters
agreed)
MARRIED
•LACP advertises Emulated
System parameters on Emulated
System ports.
•Send Intra-Portal Protocol
advertisements on Intra-Portal
Port (and alternate paths?).
•Create Distributed
Aggregation.
14
Intra-Portal Protocol Advertisements
• Ethertype
• Maybe use Slow Protocols Ethertype with new sub-type
• Maybe use new Ethertype (without chatter limits)
• Modeled after LACP advertisements
– Contains Actor parameters
• Actor_System, Actor_Emulated_System, Actor_Distributed_Key, Actor_State
– Contains copies of parameters received from Spouse
• Spouse_System, Spouse_Emulated_System, Spouse_Distributed_Key, State
– In sync when the Spouse_* parameters in received advertisements
match the Actor_* parameters in transmitted advertisements.
• The agreed Emulated_System identifier is the numerically lower of that
proposed by the Actor or the Spouse.
• The agreed Distributed_Key is the value associated with the agreed
Emulated_System identifier.
• May contain other TLVs in some or all advertisements for
coordinating gateway selection, link selection, etc.
15
Obviously needs further refinement
16
Thank You
.
17