SAIN
Networking
SAIN = Synchronized Adaptive
INfrastructure
Ray W Sanders
Chairman, SAIN Networks, Inc.
overcoming
unintended consequences
in voice and data networks
What this talk is about
A simple paradigm that can
overcome the unintended
consequences of today’s
stochastic data network.
The paradigm results in a simple
underlayer that can ensure a
deterministic data network.
2
Sanders Prediction
Packets will be forever, but the global
Internet will morph into something
that looks a little like a late 1970’s
telephone network but with far
more capability and without the
fatal flaws of carrying only
connections that must last
for at least a few seconds
and support only voice
conversations
3
Comparing SAIN with
existing networks
Existing Networks
Route one-connection-at-a time
SAIN Networks
Route aggregations of connections
Hop-by-hop routing for each connection Route aggregations for a one-hop channel
Network uses multiple control planes
Network uses a single control plane
Networks are largely stochastic
Networks are deterministic
Wire speed latency can be 100 ns
Latency inversely proportional to data rate
Internet uses many overlay protocols
NICs make use of single purpose utility
Data sent in bursts
Bursts forwarded or smoothed out
Complicated Quality of Service required Guaranteed delivery—one metric: delay
Head-of-line blocking complications
Small packet wins by increasing data rate
Tough privacy and security problems
Disjoint objects conceal cellet relevance
Bursty data can require overprovisioning Aggregated streams require less BW
Many networking protocols must exist
2 simple algorithms manage BW and routes
4
A thought experiment
Assumptions:
1. 1,000 people want to see 1,000 two-hour movies starting at 8 p.m.
2. Each movie contains 9 gigabytes of data
3. The network can use up to 10 Gbps to deliver a collection of movies
4. Suppose we use 10 Gbps for each movie
It takes 7.2 seconds to send one movie
How long does it take to send all 1,000 movies one after another?
1,000 × 7.2 seconds= 2 hours
How long would the average customer need to wait to start seeing his movie?
1 hour
Now, suppose that we send each movie at 10 Mbps (1/1000th of 10 Gbps)
How long does each of 1,000 customers wait to start watching his movie?
0 hours
There is a compelling requirement to control bandwidth,
(and hence delivery time) to meet each customer’s need
This result can obtain if a network is deterministic
5
Goals for a SAIN network
Define and build elemental pieces
of a network architecture that:
1.can support all existing voice and data
network traffic
2.can support unknown future traffic types
3.can grow from data centers,
to metropolitan networks,
to a global interconnected network
4.is robust, efficient, and simple
5.is a circuit-based architecture that can endure
and scale for decades
6
Constraining networks to
really improve their efficiency
A core principle of the SAIN architecture
Partition a network into small disjoint
pairs of active objects such as
pairs of NICs and
pairs of switches
What does this do?
• Enhances a network’s privacy and security
• Prevents one object in a network from changing the state of
another without using a Control Vector to send messages from a
source object to a destination object
• Prevents any entity outside a network from changing the state of
an object inside the network
• Simplifies object addressing
Generic
Aggregation Switch
Interconnecting
Elements
Generic
Disaggregation Switch
Basic Aggregation / Disaggregation Switch Pairs
7
Constraining networks to
really improve their efficiency
Another core principle of a SAIN architecture
Nodes in a SAIN network are
synchronized to a common clock
What does this do?
• Enables very cheap high-performance switches that can
scale well beyond current limits
• Removes the need for complex Quality of Service facilities
inside a network
8
Constraining networks to
really improve their efficiency
A third core principle of the SAIN architecture
All user data protocols
are separated from
data transport
and its control
SAIN
Protocol
SAIN Protocol
Translator
Translato
Ingress | rEgress NIC
SAIN Protocol
Translator
Ingress | Egress NIC
What does this do?
• Defines an underlay network whose
only job is to transfer bits from a
20
20
data source to a data sink
4 SAIN 4
Host,
Host,
• Enables Network Interface
Terminal,
Terminal,
Controllers (NICs) to support Server, or
Underlay
Server, or
Network
Network
devices with any protocol.
Network
• Demands that an Egress NIC’s
protocols must match its
paired Ingress NIC’s protocols
• Lets a NIC match from only one other NIC to a large number of NICs in a network
9
Constraining networks to
really improve their efficiency
A fourth core principle of the SAIN architecture
Build a lot of a network’s
physical and logical
connectivity a priori
to its use
What does this do?
• Enables each port of a network to have a physical connection
to every other port of the network with a matching NIC; the
connections are set up when the network is built or modified
• Enables every possible route to be computed when the
network is built and need not be recomputed until new nodes
are added to the network
10
Constraining networks to
really improve their efficiency
A fifth core principle of the SAIN architecture
All connections are ‘virtual’ that consume
network bandwidth only when
there are data bits to transport
What does this do?
• Enables each connection to be set up prior to use
• Assures that no bandwidth is used until data is to be sent
• Assures that an amount of bandwidth allocated to a
connection is just enough to meet a customer’s needs
11
Constraining networks to
really improve their efficiency
A sixth core principle of the SAIN architecture
All connections from a source node to a
destination node are aggregated
into a single logical data flow
What does this do?
• Significantly reduces the number of objects to be routed
through a network
• Packets do not get routed independently ; they are
combined into aggregations sent from a source to a
destination node through preset routes
• No computing needed at each tandem node
• A route is a virtual connection between two nodes; if it
approaches congestion, another route can be quickly added
12
Constraining networks to
really improve their efficiency
A seventh core principle of the SAIN architecture
The amount of available bandwidth and delay must
be known for each possible route through a network
before a connection is made
What does this do?
• Prevents discarding packets because of network congestion
• Dynamically provides the most cost-effective route with
bandwidth to meet each a connection’s need
How can this be accomplished?
• Delay over a route is known when nodes are installed
• Each node connected to a transport connection (trunk)
sends the trunk’s bandwidth availability to each source node
in the network periodically (e.g. 1,000 times per second)
13
What are data networking’s
unintended consequences?
Some examples of unintended consequences
in today’s networks
1.Traffic congestion and discarded packets
2.Jitter (= delay variation); traffic shaping and policing
3.Overprovisioning and Quality of Service
4.Flow-based traffic and circuit emulation
5.Lack of privacy, security and survivability
14
Network Behavior Constraint 1
Eliminate Traffic Congestion
Packets and packet buffers are not going away in a SAIN network
For each end-to-end connection there is a packet buffer
at its ingress node and one at its egress node
Each connection that occurs at a source-node/destination-node pair
within a given period (an ‘epoch’ for a group of connections)
originates within a pair of switches
Generic
Aggregation Switch
Interconnecting
Elements
Generic
Disaggregation Switch
Basic Source Aggregation / Destination Disaggregation Switch Pairs
Interconnecting Elements include
source/destination node switches
in three aggregation tiers
above the lowest tier
The lowest tier aggregates customer data;
the higher tiers forward aggregations
Each higher tier aggregates the next lower tier’s data
15
Network Behavior Constraint 2
Eliminate jitter, traffic shaping and policing
Jitter (also known as delay variation) is the aperiodic arrival
of each packet. Aperiodic arrivals of packets in
data flows can cause service disruptions
Changing bandwidth of a connection can assure that
either the start time of a received packet or the
time required to receive an entire packet
provides uninterrupted service
SAIN network synchronization provides ‘traffic shaping’
and ‘policing’ without additional complexity
16
Network Behavior Constraint 3
Reduce overprovisioining
Aggregating connections into channels can benefit
from the Law of Large Numbers
The law can result in the bandwidth of a
large aggregation changing slowly
compared to faster bandwidth
changes of the lowest tier
Node synchronization can result in a network
not needing Quality of Service
as currently defined
A desirable metric is end-to-end delay of entire packets—
not wire speed starting time of sending a single packet
17
Network Behavior Constraint 4
Flow-based traffic without Circuit Emulation
Nodal clocks can provide physical circuits in a simple manner
compared to the current complexity
of circuit emulation
The physical circuits operate at all levels of aggregation
and can be virtual or real
The necessity of providing circuits for
flow-based traffic is a major reason
to implement the SAIN architecture
In addition to basic algorithms, a third
‘floating frame’ algorithm exists for
plesiochronous operation where
span lengths of trunks vary
(e.g., for moving nodes and
environment variations)
18
Network Behavior Constraint 5
Provide better privacy, security and survivability
Overcome current core network privacy and security weaknesses
A SAIN network can assure that all network objects used to forward
packet data through the network are disjoint.
Network data forwarding control can be massively distributed
with centralized monitoring and fault management
A network object cannot change the state of another object
except by using a certified Control Vector connected from
a source node to a destination node
A destination node can authenticate certification
of a connected Control Vector.
Certification can use round-trip delay
of destination and source nodes
Bandwidth management algorithm results in
ever-changing aggregation frames
‘Floating frames’ enhance security
19
More Network Behavior
Architecture scales beyond current limits
Instead of forwarding entire packets
a SAIN network forward only one
or a few bits of a packet at a time
This results in using very simple switches that forward
large aggregations without requiring
expensive large routers
Not only are costs reduced;
energy needs are reduced as well
20
More Network Behavior
A single metric defining application needs
There is no need for traffic shaping or policing;
there is no need for circuit emulation;
there are no out-of-order packets;
and the packet loss rate is zero
Synchronized network nodes and
implicit addressing
achieves this goal
Node synchronization can result in a
single metric that defines required
delays for application types
The single metric defines
end-to-end delay of entire packets—
not just the wire speed starting time
of sending a single packet
21
More Network Behavior
Results from simulations of a model network
17
07
4
010
5
07
27
2.1
042
278.0
226
.3
357.9
041
02
8
044
051
214.5
11
063
068
295.8
043
033
226.2
069
28
1.1
052
4
05
062
064
161.8
12
6
06
5 58.2
6
0
1
067
003
367
.9
05
5
20
3.
2
7
6
04
05
6
0.1
33
049
163.3
061
246.2
6
9.2
22
5
04
10
7
04
8.5
23
05
8
050
14
3
05
29
4.7
8
030
2
03
4
02
05
7
034
80
02
7
015
247
.0
004
217.
4
036
4
03
8
8
04
8.5
19
059
.6
273
33
0.7
5.9
26
3
02
.5
215
3
03
7
029
31
014
060
026
025
.1
204
021
9
03
5
15
5. 2
41
269.2
022
The green circles are
transit nodes (T-Nodes)
The red rectangles are
9
entry/exit nodes [E-Nodes]
Each [E-Node] (T-Node) contains
source switches connecting to
paired destination switches in all other
[E-Nodes] (T-Nodes) in a network
008
165.8
006
233.8
20
0
02
007
.8
248
002
071
0
04
188.8
035
2
1
001
013
7. 9
31
01
1
217.5
9
01
20
0.4
072
172
.1
016
18 012
005
017
423.5
.4
291
19
077
16
009
185.5
018
79
19
078
07
3
6
07
070
7
9.
27
13
A Metropolitan Area Network Example with 20 T-Nodes & 80 Simplex Trunks
500 E-Nodes each able to support >4,000 ports each with multiple IP addresses
22
Sanders Suggestion
We should not let ourselves make another
management mistake that the future of
networking will be based entirely on
using packet switches for routing
Our focus should morph into efforts that
enhance IP* addressing and DNS*
in a circuit-based world with
advanced NIC applications
* Internet Protocol addressing and
* Domain Name System
23
24
25
How to support goals 1 & 2
(Support existing and future traffic types)
• Transport of bits is independent of data type
SAIN
Underlay
Network
20
4
SAIN
Protocol
SAIN Protocol
Translator
Translato
Ingress | rEgress NIC
SAIN Protocol
Translator
Ingress | Egress NIC
Host,
Terminal,
Server, or
Network
20
4
Host,
Terminal,
Server, or
Network
• Packets appear only at ingress and egress ports with connected NICs
• Packet or circuit data appears at an ingress NIC and is
transferred to an egress NIC
• An ingress/egress pair of NICs can support any matched data type
• NIC pairs can support secure topologies and methods
• Packets are transferred bit-by-bit at a deterministic data rate
• An Egress NIC delivers the protocol entering its paired Ingress NIC
26
How SAIN works #1
What a packet flow can look like:
Packet Header
H
Input A
Packet Data
D
H
Input B H
D
B1
Output
H
D
A1
H D
A2
H
D
A3
D
B2
H
D
B1
H
D
A1
H D
A2
H
D
B2
H D
A4
H
This method of multiplexing uses ‘explicit addressing’
D
A3
H D
A4
What a SAIN flow can look like:
The size of each cellet is fixed for a given link in which a frame occurs
The duration of an Epoch can depend on the desired
end-to-end network delay of all embedded packets
This method of multiplexing uses ‘implicit addressing’ where
the position of each cellet defines its connection or channel identity
27
How SAIN works #2
A SAIN network contains
simple network switches with a
very different approach that uses very simple parts
Generic
Aggregation Switch
Interconnecting
Elements
Generic
Disaggregation Switch
Basic Aggregation / Disaggregation Switch Pairs
The ‘Interconnecting Elements’ are primarily made up of
Aggregation Switch / Disaggregation Switch pairs
that exist in three levels of aggregation
Each tier contains
Aggregation / Disaggregation Switch Pairs
The three aggregation levels pass data use three network tiers
plus an exchange tier to other networks and a virtual
distribution sub-tier shown in the next slide
28
How SAIN works #3
Connections exist in an Entry/Exit E-Node tier that includes a
virtual VE-Node subnetwork uses for traffic distribution
Each E-Node connects large aggregations of
connections within large channels to and from
a parent Transit T-Node tier
Each T-Node routes the aggregations of
E-Node traffic for delivery from a Source
T-Node to a Destination T-Node
In addition to its T-Node tier
routing functionality, a T-Node
can connect to an eXchange
X-Node that can have a
channel to other X-Node
domains including those
that make up a global
domain
eXchange
X-Nodes
Transfer
T-Nodes
Entry/Exit
E-Nodes
Virtual Entry/Exit Nodes
VE-Nodes
29
More Network Behavior
Routing in the model network
Each E-Node connects to a parent T-Node
Each T-Node has full period connections to every other T-Node
Each Source T-Node can set up a loop-less route
through T-Nodes to every other T-Node
Each route can be computed at network instantiation
The computation begins with a table of
single hops among the T-Nodes
A second hop for each entry can be added for each second hop
that does not include the first hop
Repeat this process recursively for a two-hop table to build a
three-hop table and continue for tables with more hops
The process results in finding all routes that do not contain loops
A 10-hop table has over 500,000 entries for all source to
destination routes in a 20 T-Node model network
The average number of routes for each of the
380 paired connections is about 1300
30
Today’s traffic is mostly
flow-based, not bursty
Today’s networks are based
on early 1970’s needs:
using minicomputers to
send messages and
transfer files
Queuing theory provided solutions for
an asynchronous stochastic world
Today’s needs are circuit-based to
satisfy a burgeoning market
for flow-based traffic
What is needed now is a network with
synchronized nodes that support
dynamic data rate connections
31
Another experiment
Assumptions:
1. A financial trading firm wants to minimize its network delay
2. The smallest Ethernet frame is 84 bytes including a 46-byte payload
3. A SAIN network frame can have 5 bytes plus the 46-byte payload
4. In either case, a 1 Gb/s channel is carrying the data
A SAIN 408-bit (51-byte) packet could be guaranteed delivery in one
microsecond or less
This compares to 672-bit (84-byte) Ethernet needing nearly one
microsecond if there is no other traffic using the channel. Its delay is
not guaranteed.
There is a compelling requirement to control bandwidth,
(and hence delivery time) to meet a customer’s need
32
A brief look at a basic principle
that really matters
100 msec
10 msec
1.0 msec
100 μsec
1
4
4
1
1
1b 2
bit bit by 2 by by 8 by 6 b 32 b 64 b 28
it
te
s
tes tes tes yte yte yte by
s
te
s
s
s
64 s
=4
10 μsec
15
6
+1
8b
yte
18
90
=1
50
0
+1
00
=8
8b
yte
98
s
2
+1
8b
yte
s
s
1.0 μsec
100 nsec
10 nsec
Delay vs. Data Rate
An 8 x 8 orders of magnitude
look at a key fundamental of
data networking
1.0 nsec
1.0 kb/s
10 kb/s
100 kb/s
1.0 Mb/s
33
10 Mb/s
100 Mb/s
1.0 Gb/s
10 Gb/s
100 Gb/s
Can we cover the earth
with a SAIN network?
The earth’s land mass area totals ~148,940,000 sq km.
The area of each square within a 2 millisecond
radius circle is ~320,000 sq km.
Area of the Square
≈ 320,000 km²
Area of the Circle
≈ 502,654 km²
~565.7 km
2
m
se
c
op
~4 tica
00 l f
km ibe
r
ra
di
us
The number of supermetro networks
needed to cover the land mass: 466
In the real world, sizes will likely be based on
number of users and/or number of ports
and market to determine a diameter
34