Network technology transition &
testbeds
Henning Schulzrinne
Dept. of Computer Science
Columbia University
June 19, 2007
Intel-UCSD (Santa Clara)
Overview
•
•
•
•
•
Networking technology evolution
Why do good ideas fail?
The role of testbeds
The limitations of testbeds
Moving research into the real world
June 19, 2007
Intel-UCSD (Santa Clara)
Internet transition: applications
• Moving analog applications to Internet
– digitization of communication largely completed
• Extending reach of application
– mobile devices
– vehicles
• Broadening access
– Minitel: SNCF had train schedule service
– web: anybody can have a blog
• Allowing customization and creation
– web pages to code modules
June 19, 2007
Intel-UCSD (Santa Clara)
Completing the migration of comm. applications
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Qui ckTime™ and a
TIFF (U ncompr essed) decompressor
are needed to see thi s pi cture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
June 19, 2007
QuickTi me™ and a
TIFF ( Uncompressed) decompressor
are needed to see thi s pi ctur e.
Intel-UCSD (Santa Clara)
Migration of applications
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTi me™ and a
TIFF ( Uncompressed) decompressor
are needed to see thi s pi ctur e.
QuickTi me™ and a
TIFF ( Uncompressed) decompressor
are needed to see thi s pi ctur e.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTi me™ and a
T IFF (Uncom pressed) decom pressor
are needed to see t his pict ure.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
June 19, 2007
text, still
images
audio
video
synchronous
IM
VoIP
video
conferencing
asynchronous
email
email,
voicemail
YouTube
Intel-UCSD (Santa Clara)
What’s wrong with the Internet - user perspective
•
•
•
•
Lack of trust
– small mistakes  identity gone
– can’t tell when one has “lost the wallet”
– waste time on spam, viruses, worms, spyware, …
Lack of reliability
– 99.5% instead of 99.999%
– even IETF meeting can’t get reliable 802.11 connectivity
Lack of symmetry
– asymmetric bandwidth: ADSL
– asymmetric addressing: NAT, firewalls  client(-server) only, packet relaying
via TURN or p2p
Users as “Internet mechanics”
– why does a user need to know whether to use IMAP or POP?
– navigate circle of blame
June 19, 2007
Intel-UCSD (Santa Clara)
Lifecycle of technologies
COTS
(e.g., GPS)
traditional technology propagation:
IM, digital photo
military
opex/capex
doesn’t
matter;
expert
support
Can it be done?
June 19, 2007
corporate
capex/opex
sensitive,
but
amortized;
expert
support
Can I afford it?
Intel-UCSD (Santa Clara)
consumer
capex
sensitive;
amateur
Can my mother use it?
Research  reality
protocols
architecture
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
cell-based networks
distributed data structures
network coding
(Internet, VoIP,
DTN, ...)
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
95% of effort
new scheduling
algorithm
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
June 19, 2007
Intel-UCSD (Santa Clara)
What has made the Internet successful?
•
36 years  approaching mid-life crisis  time for selfreflection
–  next generation suddenly no longer finds it hip
•
Transparency in the core
– new applications (web, VoIP, games)
•
Narrow interfaces
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
– socket interface, resolver
• HTTP and SMTP messaging as applications
– prevent change leakage
•
Low barrier to entry
– L2: minimalist assumptions
– technical: basic connectivity is within
– economical: below $20?
•
Commercial off-the-shelf systems
– scale: compare 802.11 router vs. cell base station
June 19, 2007
Intel-UCSD (Santa Clara)
Cause of death for the next big thing
QoS
multicast
not manageable across competing
domains


not configurable by normal users (or
apps writers)

no business model for ISPs


no initial gain

80% solution in existing system

mobile
IP

active
networks
IPsec
IPv6
















(NAT)
increase system vulnerability
June 19, 2007


Intel-UCSD (Santa Clara)


Why do good ideas fail?
• Research: O(.), CPU overhead
– “per-flow reservation (RSVP) doesn’t scale”  not
the problem
– at least now -- routinely handle O(50,000) routing
states
• Reality:
QoS
– deployment costs of any new L3 technology is
probably billions of $
– coordination costs
• The QoS problem is a lawyer problem, not an
engineering problem
• Cost of failure:
– conservative estimate (1 grad student year = 2
papers)
– 10,000 QoS papers @ $20,000/paper  $200
million
June 19, 2007
Intel-UCSD (Santa Clara)
quality-ofservice
IEEE
10,377
12,876
ACM
3,487
4,388
Good ideas
•
•
•
Myth: Good ideas will win
– “Build a better mousetrap and the world will beat a path to your door.” (Ralph
Waldo Emerson)
– modern version: IEEE 802.11 will dig through IEEE Infocom proceedings to
find your master paper
– even most Sigcomm papers have had no (engineering) impact
Myth: Just ahead of its time – it will take 10 years to have impact
– reality: most papers either have immediate impact or none, ever
Mediocre ideas with commitment win over brilliant ideas without
– particularly if part of a larger system
– cost of understanding ideas
– possible encumbrances (patents)
–  researchers need to accompany their “children” through teenage years
June 19, 2007
Intel-UCSD (Santa Clara)
More reasons for failure
• Old good enough
– reliability and familiarity outweighs modest gains in performance
– ATM in LANs
• No real need
– QoS: VoIP may finally motivate
• Not worth the money - price, not technology
– video conferencing: succeeds once no incremental cost
– 802.11 wireless, GPS
• Value not captured
– those getting the benefit not willing to pay for it
– e.g., enabling unknown new applications
• Market niche disappears
– cell switching vs. packet switching
June 19, 2007
Intel-UCSD (Santa Clara)
The Hockeystick Problem
utility
1.0
complexity
security risks
bandwidth
June 19, 2007
Intel-UCSD (Santa Clara)
adoption
Internet: What has gone wrong?
•
•
•
•
•
Familiar to anybody who has an old house…
Entropy
– as parts are added, complexity and interactions increase
Changing assumptions
– trust model: research colleagues  far more spammers and phishers
than friends
• AOL: 80% of email is spam
– internationalization: internationalized domain names, email character
sets
– criticality: email research papers  transfers $B and dial “9-1-1”
– economics: competing providers
• “Internet does not route money” (Clark)
Backfitting
– had to backfit security, I18N, autoconfiguration, …
 Tear down the old house, gut interior or more wall paper?
June 19, 2007
Intel-UCSD (Santa Clara)
Core goals for new networks
•
•
•
•
•
reliability
diagnosability
sustainability
adaptability
survivability
June 19, 2007
Intel-UCSD (Santa Clara)
(My) guidelines for a new Internet
•
Maintain success factors, such
as
– service transparency
– low barrier to entry
– narrow interfaces
•
New guidelines
– optimize human cycles, not
CPU cycles
– design for symmetry
– security built-in, not bolted-on
– everything can be mobile,
including networks
– sending me data is a privilege,
not a right
– reliability paramount
– isolation of flows
June 19, 2007
•
New possibilities:
– another look at circuit switching?
– knowledge and control
(“signaling”) planes?
– separate packet forwarding from
control
– better alignment of costs and
benefit
– better scaling for Internet-scale
routing
– more general services
– storage and computation as
network services
Intel-UCSD (Santa Clara)
Impact of networking research
• Very low publication-to-impact ratio
• Brilliant idea, magically transformed into reality
– by somebody else
• Research as point scoring
–
–
–
–
publication count
citation by other papers, also without impact
read mostly by other researchers
goal: graduate/get tenure
June 19, 2007
Intel-UCSD (Santa Clara)
Who’s the customer?
• Goals may not be identical
– Equipment vendors: preserve investment, confirm earlier choices
• ATM, SS7
– Carrier: preserve product differentiation, business model,
customer lock-in, monopoly rent, …
• walled gardens, WAP, AAA, DRM, IMS, …
– Consumer: fashion, functionality, cost
• search engines, WiFi, MP3, Skype, web hosting, …
• Easier for some organizations
– e.g., Google: direct customer is advertiser, but revenue driven by
page views  consumer
June 19, 2007
Intel-UCSD (Santa Clara)
Why testbeds?
• Hardware is cheap
– project-specific --> avoid reproducability problems
• Can’t build it at one institution
– cost
– scale (distances, nodes)
– maintenance (technicians, programmers)
• Can’t justify at one institution
– cost vs. usage intensity
June 19, 2007
Intel-UCSD (Santa Clara)
Four testbed flavors
users
• Experimental system: build, evaluate, write
– PlanetLab, EmuLab, ORBIT
– usually under-resourced
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
• Demo platform
– show sponsor that money was not wasted
– gain user visibility
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
• Deployment platform
– e.g., Coral on PlanetLab
– geographic reach: within one-hop access by millions
– user privacy concerns
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
• Migration path (pre-commercial)
– Internet2 (maybe)
– limited evidence (economics vs. technology)
– user privacy concerns, particularly for data plane
June 19, 2007
Intel-UCSD (Santa Clara)
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Testbed limitations
•
Small scale (10s of nodes)
– unlikely to test scalability of IPv8 or BGP++
•
Friendly users
– Internet2 can send malicious users to the dean’s office
•
Difficult to build community
– the less commodity, the harder (learning curve for grad students)
– unclear transition path (cf. Linux)
•
Fake economics
– usually, free to end users
– administered by wizards
•
Mostly landline
– spectrum and cost prevent large-scale mobile testbeds
– user interest is in ubiquitous mobility
•
System vs. components
– can only change one component at a time
June 19, 2007
Intel-UCSD (Santa Clara)
Conclusion
• Start from problem definition, not testbed (or $300M...)
–
–
–
–
user-focused, not researcher toys
just more QoS and multicast?
different from other research areas: plumbing vs. new capabilities
will it prevent spam and just work out of the box?
• Good research practices and evaluation
– or is everybody else supposed to do the hard work?
– rising expectations: can’t assume friendly Internet of 1980s
• Realistic expectations for testbeds
– different targets
• reproducibility vs. usability
– scale
• just large enough
• Consider federation of testbeds
– combine scale and reproducibility
June 19, 2007
Intel-UCSD (Santa Clara)