aBitCool / Ignite
A vast array of micro service providers with gigabit access
a 21vianet group project
APnic38 – Brisbane, AU
15-17 September 2014
Tony Hain
info@abitcool.com
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
Hain Global Consulting, Inc.
info@hain-global-consulting.com
Agenda
•
•
•
•
•
aBitCool project goals
Deployment plan
Successes & Challenges
Status
Wrap up
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
What is aBitCool?
• aBitCool is a project of 21Vianet Group, Inc:
“by the Geeks, for the Geeks”
What are the product and services that a “Super-Geek” would want?
(once proven commercially feasible, others will follow)
• Vision :
“Revolution is achieved through Consumer Empowerment”
• Mission :
“Transform Internet Deployment” / Restore Architecture Principles
• Strategy :
“Content Centric – Open (Permission Free) Exchange Network”
• Principle :
“Decentralized, Distributed, Disruptive”
3
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
aBitCool Project Goals
• Home as a Datacenter
– HomeAP access router as hub for content & services.
• Open eco-system at edge
– Members connect to an open Layer 2 exchange network, peering
with content and each other, rather than just acquiring IP transit.
– Equality (Members are active participants and actors)
– “Permission-free” innovation by members
• Redefining ISP
– Push State-of-the-art boundaries > l per member
– Drive advanced technology deployment rather than constrain
innovation to the limitations of the old.
– Encourage new-entrant ISPs, services, & content sources.
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
Consumer Network Bandwidth Evolution
AVAILABLE CONSUMER TECHNOLOGY
Consumer
Data Speeds
Enabling
Technology
1980s
1990s
2000s
2010s
2020s
300bps+
9.6kbps+
256kbps+
5Mbps+
10Gbps+
Dial-Up
Dial-Up /
GSM
ADSL
3G
Cable
4G
Fiber
5G
 1Gbps+ services will change the historical ~20x per decade multiple that limits application evolution
EMERGENT INNOVATIONS
???
Social
Media
???
N/A
4K 8K
5
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
Ignite : Bringing back “Inter-networking”
Each Member is
an “ISP” with
Public IPv6 / IPv4
ASN (& BGP)
Dedicated
Bandwidth per
Member (N x
Lambda, λ)
Core Network is
a open Internet
Exchange
B2B2C platform
Members can
choose / buy
directly
Peer-to-Peer L2
Fabric to reduce
physical Cost of
Cross-Connects
POP Router
IDC AS
HD Video
ICP1 AS
eBGP
Route Server
Member Sub1
λ1
Gaming
ICP2 AS
eBGP
λ2
Member Sub2
Member Sub3
eBGP
eBGP
ICP3 AS
Social Media
ICP Network
λ3
Transit Router
λ4
Distribution
Fabric
Core Fabric
China Telecom
China Unicom
Member Sub4
Switching Fabric Network
Member Network
WDM Network, each member on a unique Lambda for upgrade path
Sub3
DC-IX peers
DC-IX fabric
Transit Routing Network
Sub1
OADM2
OTN-1
OTN-5
OTN-2
OADM3
Sub4
OADM1
Sub2
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
N*10GE
WDM Device Distribution
Fabric
OTN-4
OTN-3
Optical Network
Network Architecture Description
The network is composed of logical blocks:
Member Access: Members run BGP inside aBitCool network.

Route Servers in Switching Fabric will establish EBGP Peers with ICP POP Routers and Residential subscribers
eBGP routers and reflect the BGP routes among 3rd parties.
 Switching Fabric: Distribution and Core Fabrics. Distribution will offer GE
fiber access service to members. Distribution connected with Core Fabric via
10GE or higher. Tbps switches will be clustered to provide capacity and
availability. Core Fabric will connect to ICP POP router at 10GE or higher.
Transit Routing: Is offered for members who have not established a direct
relationship with a transit provider.
ICP Network: The ICP network allows content owners to host abundant HD
video, Social Media and other inter-active service inside the aBitCool
network.
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
Members and IPv4 Address
 There are 2 type of member subscribers in the network.
Subs Type
ASN
CPE WAN interface
IPv4 inside
Subscribers network
Type A Member
Public ASN
Private IPv4
Public IPv4 network
Type B Member
No ASN
1 Public IPv4 acquired
from APNIC
Private IPv4
network
 Type A member may establish BGP with RouteReflector and may choose to receive
BGP routes for internet transit service.
 Type B member will have static NAT GW configured on CPE and all traffic will be
routed to the GW.
 Both the Type A and Type B member will be assigned a static IPv4 address for WAN
interface of CPE.
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
Initial Access Network Design
IP CORE
Core Network
 There are 2 ways to connect the members.
 Using Dark Fiber
 Using CWDM
S5700 SW
 Dark Fiber connection only can be used in the
circumstance of ONLY ONE Member in a community.
 CWDM and OADM can be deployed to save Dark
Fiber resource. One line card can support 8 members
CWDM
OADM
OADM
CWDN access Network
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
at maximum.
Initial BGP Design
Type A member1
Planned in Future
China Telecom
电信
NE40E-X8-M06-1
Type A member2
China Unicom
联通
IDC AS
Type A member3
AbitCool AS
 EBGP connection will be established between Type A members and NE40E-
ASR9010-M06-1
X8, IDC AS also establish EBGP with NE40E-X8.
 All Type A members, IDC edge router are in the same peering Subnet and
VLAN.
 All BGP routes will be propagated through NE40E-X8 and traffic will be
forwarded directly through the Switching Fabric.
 Peering Group for Type A members will be configured on NE40E-X8, so the
unified Route-Policy can be made on the outbound direction.
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
BGP Peer with Member
BGP Peer with ICP
BGP Peer with ISP
HomeAP - access router / distributed data center
• Provides consistent operational
interface to the network for
members that do not provide
their own router.
• 4 wan ports by default.
• 3 wifi radios : mesh as well as
client configurations
• Personal cloud
– P2P CDN
– Home as a data source
• Hosts distributed DC functions,
local caching, DNS, NTP
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
Content consumption
• UHD TV dongle that runs
Firefox-OS that enables mobile
apps to “fling” video to the big
TV screen
• Partially Compatible with
Chromecast 1.0/2.0
• “Open” Match Stick vs
“Closed” Chromecast
• Building block for the
Decentralized “Video Internet”
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
Real Example : A Current Home Network
Advent of Internet of Things will need 1,000 IPs @ home!
Dual IPv4 / IPv6 network
Home Network
Network Elements
ISP
REMOTE
Type
#
Gateways
2
Routers
10
Switches
4
IP Devices
54
Total
70
Subnets
9
SERVICES
SENSORS & APPLIANCES
GUEST
ROUTER INTERCONNECT
SOHO
MEDIA
DESKTOPS
Connected devices
include
• 20 Servers
• 4 HD TVs
• 1 Green House
• 1 Threadmill
(dynamic maps w/tilt
for run routes)
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
Successes & Challenges
• Successes
– Equipment works as expected
– ?? Members with Gbps ports
• Challenges
– Team consistency on business plan and vision
• Low initial subscriber density requires ‘investment in a plan’
• Costs of fiber & initial transit often result in ‘go slow’ approach
• Fiber install permissions not acquired early ‘due to cost’
– Unusual APnic relationship with many micro-service
providers & individual members
• Likely evolving toward LIR
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
Current Status
• Active Members : #####
• Members peering directly with ICP : #####
• Traffic volume : #######
Initial Measured Throughput
SDN allows even higher ACTUAL
throughput in user experience.
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
info@abitcool.com
info@hain-global-consulting.com
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
aBitCool / Ignite
A vast array of micro service providers with gigabit access
a 21vianet project
APnic38 - Brisbane
Sept 15, 2014
Tony Hain
CEO
Hain Global Consulting, Inc.
tony@hain-global-consulting.com
Copyright 2014 -– Hain
21vianet
Global
group
Consulting,
Inc, & Hain
Inc. Global Consulting, Inc.
Initial LINK Aggregation
Internet
NE40E-X8-M06-1
S5700-M06-1
S5700-SITE A-1
S5700-SITE B-1
S5700-SITE C-1
S5700-SITE D-1
 Initial physical topology is a Layer 1 full ring, provisioned as a logical star.
 More IP links can be used after the planned OTN Transmission Network is deployed.
 Link aggregation for the uplinks of S5700 will be deployed in order to increase uplink
bandwidth and link redundancy.
 LACP will be deployed to negotiated LINK GROUP between NE40E-X8 and S5700.
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
Type B member - CGN NAT Design
Type B member
CPE1 WAN：X1/22
NAT IPv4 Pool：Z/27
Internet
Type B member
CPE 2 WAN：X2/22
Type B member
CPE3 WAN：X3/22
Source IPv4 NAT translation：
X1,X2,X3Z
 There is no BGP connection from NE40E-X8 to China Telecom or China Unicom
 NAT service line card is deployed on NE40E-X8 to translate CPE WAN Public IPv4
addresses (TYPE B) into IPv4 address from NAT IPv4 Pool.
 Syslog server will be deployed to record the NAT LOGs.
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.