Advanced Load
Balancing/Web Systems
Edward Chow
Department of Computer Science
University of Colorado at Colorado Springs
6/21/2000
Edward Chow
Outline of the Talk
• Trends in Web Systems
• Web switches and the support for advanced web
system features.
• Load balancing Research
– load balancing algorithms research
– network bandwidth measurement research
– web server status research
6/21/2000
Edward Chow
Load Balancing/Web System 2
Readings/References
• Application level solution:
Apache/Jserv/Servlet
• Kernel level load balancing solution:
http://www.linuxvirtualserver.org/
– Joseph Mark’s presentation
– LVS-NAT(Network Address Translation) web page
– LVS-IP Tunnel web page
– LVS-DR (Direct Routing) web page
• Hardware solution: Foundry ServerIron Installation
and Configuration Guide, May 2000.
6/21/2000
Edward Chow
Load Balancing/Web System 3
Trends in Web Systems
• Increasing the availability, performance, and manageability of
web sites.
• High Performance through multiple servers connected by high
speed networks.
• High Availability (HA) 7x24 network services
• Reliable/Efficient Content Routing and Content Distribution
• Emerging Network Centric StorageNetworks
• Emerging Linux virtual server library for low cost HA web
systems.
6/21/2000
Edward Chow
Load Balancing/Web System 4
Networkshop’s Prediction
• Already, load-balancers are overcoming the inherent one-to-one
nature of the network and distributing queries across tuned
servers -- GIFs to a machine with a huge RAM cache, processing
to servers with fibre-channel-attached databases.
• I suspect we'll see content routing as a full-fledged concept in
Las Vegas next spring.
By Networkshop News 10/1999.
6/21/2000
Edward Chow
Load Balancing/Web System 5
Load Balancing Systems
• Cheap solution: Linux/LVS as load balancer for
distributing requests to backend real servers.
• Medium price solution: Microsoft Server Cluster;
Zeus Load Balancer
• High performance: Web Switches (special hardware)
from Arrowpoint(CISCO), Foundry ServerIron, Alteon
WebSystems, Intel XML distributor.
6/21/2000
Edward Chow
Load Balancing/Web System 6
Virtual Resource Management
•
•
•
Also called Server load balancing or Internet Traffic Management.
Goal: Increasing the availability, performance, and manageability of web
sites.
April 2000 Acuitive Report on 1999 VRM market share
6/21/2000
Edward Chow
Load Balancing/Web System 7
VRM Market Prediction
6/21/2000
Edward Chow
Load Balancing/Web System 8
F5 VRM Solution
Site II
losangeles.domain.com
Internet
Internet
Site I
newyork.domain.com
Router
3-DNS
BIG-IP
BIG-IP
Local DNS
GLOBAL-SITE
Webmaster
Site III
tokyo.domain.com
Server Array
User
london.domain.com
6/21/2000
Edward Chow
Load Balancing/Web System 9
BIG/ip - Delivers High Availability
•
E-commerce - ensures sites are not only up-and-running, but taking
orders
•
Fault-tolerance - eliminates single points of failure
•
Content Availability - verifies servers are responding with the correct
content
•
Directory & Authentication - load balance multiple directory and/or
authentication services (LDAP, Radius, and NDS)
•
Portals/Search Engines – Using EAV administrators perform key-word
searches
•
Legacy Systems - Load balance services to multiple interactive
services
•
Gateways – Load balance gateways (SAA, SNA, etc.)
•
E-mail (POP, IMAP, SendMail) - Balances traffic across a large
number of mail servers
6/21/2000
Edward Chow
Load Balancing/Web System 10
3DNS Intelligent Load Balancing
• Intelligent Load Balancing
– QoS Load Balancing
• Quality of Service load balancing is the ability to select
apply different load balancing methods for different users
or request types
– Modes of Load Balancing
• Round Robin
• Least Connections
• User-defined Quality-of-Service
• Completion Rate (Packet Loss)
• Global Availability
• Topology Distribution
• LDNS Round Robin
• E-Commerce
6/21/2000
Edward Chow
Ratio
Random
Round Trip Time
BIG/ip Packet Rate
HOPS
Access Control
Dynamic Ratio
Load Balancing/Web System 11
GLOBAL-SITE Replicate
Multiple Servers and Sites
•
•
•
•
•
•
File archiving engine and scheduler for automated site and server
replication
BIG-IP controls server availability during replication and
synchronization
– Gracefully shutdown for update
– update in group/scheduled manner
FTP provides transferring files from GLOBAL-SITE to target servers
(agent free, scalable)
RCE for source control
No client side software
Complete, turnkey system (appliance)
(adapt from F5 presentation)
6/21/2000
Edward Chow
Load Balancing/Web System 12
Content Distribution
• Secure, automate content/application distribution to single
(multiple server)/wide area Internet sites.
• Provide replication, synchronization, staged rollout and roll back.
• With revision control, transmit only updates.
• User-defined file distribution profiles/rules
6/21/2000
Edward Chow
Load Balancing/Web System 13
Intel NetStructure
•
•
Routing based on XML tag (e.g., given preferred treatment for buyers,
large volume)
http://www.intel.com/network/solutions/xml.htm
6/21/2000
Edward Chow
Load Balancing/Web System 14
1. Compared to SUN E450 server
6/21/2000
Edward Chow
Load Balancing/Web System 15
Phobos IPXpress
•
Balances web traffic among all 4 servers.
•
Easily connects to any Ethernet network.
•
Quick set up and remote configuration.
•
Choose from Six different load balancing algorithms
Round Robin
Least Connections
Weighted Least Connections
Fastest Response Time
Adaptive
Fixed
•
Hot standby failover port for web site uptime.
•
U.S. Retail $3495.00
6/21/2000
Edward Chow
Load Balancing/Web System 16
Phobos In-Switch
• Only load balancing switch in a PCI card form factor
• Plugs directly into any server PCI slot
• Supports up to 8,192 servers, ensuring availability
and maximum performance
• Six different algorithms are available for optimum
performance: Round Robin, Weighted Percentage,
Least Connections, Fastest Response Time, Adaptive
and Fixed.
• Provides failover to other servers for high-availability
of the web site
• U.S. Retail $1995.00
6/21/2000
Edward Chow
Load Balancing/Web System 17
Foundry Networks
ServerIron Internet Traffic Management Switches
• One Million Concurrent Connections
• SwitchBack™ Also known as direct server return
• Throughput: 64 Gbps with BigServerIron
• Session Processing: Lead with 80,000 connections/sec.
• Symmetric LB: picking up the full load where the failed
switch left off without losing stateful information.
• Switching Capacity: BigServerIron deliver 256 Gbps of
total switching capacity.
6/21/2000
Edward Chow
Load Balancing/Web System 18
BigServerIron
• BigServerIron supports up to 168 10/100Base-TX
ports or 64 Gigabit Ethernet ports.
• Internet IronWare supports unlimited virtual server
addresses, up to 64,000 Virtual IP (VIP) addresses
and 1,024 real servers.
• Web Hosting: enable network managers to define
multiple VIPs and track service usage by VIP.
• Health Checks: provide Layer 3,4,7 Health Checks
Include HTTP, DNS, SMTP, POP3, iMAP4, LDAPv3,
NNTP, FTP, Telnet and RADIUS
6/21/2000
Edward Chow
Load Balancing/Web System 19
BigServerIron LB Algorithms
•
•
Round Robin
Least Connections
•
Weighted Percentage (assign perform weight to server)
•
Slow Start - To protect the server from a surging flow of traffic at startup.
It can really happened!!
“Ya, LVS has performed for us like a champ.. under higher volumes, I have
had some problems with wlc.... for some reason LVS freaks and starts
binding all traffic to one box... or at least the majority of it.. it is
really wierd... but as soon as you switch to using wrr then everything
worked fine... I have been using LVS for about 4 months to manage our
E-Commerce cluster and I haven't had any problems other than the wlc vs wrr
problem…” -- Jeremy Johnson <jjohnson@real.com> 6/1/2000
6/21/2000
Edward Chow
Load Balancing/Web System 20
BigServerIron LB Features
•
Set max connection limit for each server
•
Cookie Switching - This feature directs HTTP requests to a server
group based on cookie value. For client persistent and servlet
•
URL Switching - directs requests based on the text of a URL string
using defined policies. Can place different web content on different
servers
•
URL Hashing - map hash value of Cookie header or the URL string to
one of the real servers bound to the virtual server. This HTTP request
and all future HTTP requests that contain this information then always
go to the same real server.
•
URL Parsing - Selects real server by applying pattern matching
expression to the entire URL. ServerIron supports up to 256 URL rules
•
SSL Session ID Switching - ensures that all the traffic for a SSL
transaction with a given SSL session ID always goes to the same
server.
6/21/2000
Edward Chow
Load Balancing/Web System 21
IronClad Security
•
•
NAT
TCP SYN attack protection: stops binding new sessions for a user
definable timeframe when the rate of incoming TCP SYN packets
exceed certain threshod.
•
Guard against Denial Of Service (DoS) Attacks -against massive
numbers of uncompleted handshakes, also known as TCP SYN
attacks, by monitoring and tracking unfinished connections
•
High Performance Access Control Lists (ACLs) and Extended
ACLs - By using ACLs, network administrators can restrict access to
specific applications from/to a given address or sub-net, or port
number.
•
Cisco-syntax ACLs - ServerIron supports Cisco-syntax ACLs, which
enables network administrators to cut/copy/paste ACLs from their
6/21/2000
Edward Chow
Load Balancing/Web System 22
existing Cisco products.
Session Persistence for
eCommerce Transactions
•
Port Tracking: Some web applications define a lead port (http) and
follower (SSL) ports. ServerIron ensures connections to the follower
ports arrive at the same server
•
Sticky Ports - ServerIron supports a wide variety of 'sticky'
connections: client’s request for next port or all ports go to same server
•
Support large range of user programmable options
•
Mega Proxy Sever Persistence - treat a range of source IP addresses
as a single source to solve the persistence problem caused by certain
mega proxy sites in the Internet.
•
Use Source IP address for session persistenece when cookie missing.
6/21/2000
Edward Chow
Load Balancing/Web System 23
High Availability Services
•
Remote Backup Servers - If no local servers or applications are
available, ServerIron sends client requests to remote servers.
•
HTTP Re-direct - ServerIron can also use HTTP redirect to send traffic
to remote servers if the requested application is not available on the
local server farm.
•
Active/Standby - When deployed in Active/Standby mode, the standby
load-balancing device will assume control and preserve the state of
existing sessions in the event the primary load-balancing device fails
•
Active/Active - When deployed in Active/Active mode, both loadbalancing devices work simultaneously and provide a backup for each
other while supporting stateful fail-over.
•
Quality of Service - Network administrators can prioritize traffic based
on ports, MAC, VLAN, and 802.1p attributes, grant priority to HTTP
traffic over FTP
•
Redundant hot-swappable power supplies
6/21/2000
Edward Chow
Load Balancing/Web System 24
Linux Virtual Server (LVS)
•
Virtual server is a highly scalable and highly available server built
on a cluster of real servers. The architecture of the cluster is
transparent to end users, and the users see only a single virtual
server.
6/21/2000
Edward Chow
Load Balancing/Web System 25
LVS-NAT Configuration
• All return traffic go
through load balancer
6/21/2000
Edward Chow
Load Balancing/Web System 26
LVS-Tunnel Configuration
•
•
Real Servers need to be reconfigured to handle IP-IP packets
Real Servers can be geographically separated and return traffic go
through different routes
6/21/2000
Edward Chow
Load Balancing/Web System 27
LVS-Direct Routing Configuration
•
Similar to the one implemented in IBM's NetDispatcher
•
Real servers need to configure a non-arp alias interface
with virtual IP address and that interface must share same
physical segment with load balancer.
•
Load balancer only rewrites server mac address; IP packet
not changed
6/21/2000
Edward Chow
Load Balancing/Web System 28
HA-LVS Configuration
6/21/2000
Edward Chow
Load Balancing/Web System 29
Persistence Handling in LVS
• Sticky connections Examples:
– FTP control (port21), data (port20)
For passive FTP, the server tells the clients the port that it listens to,
the client initiates the data connection connecting to that port.
For the LVS/TUN and the LVS/DR, LinuxDirector is only on the clientto-server half connection, so it is imposssible for LinuxDirector to get
the port from the packet that goes to the client directly.
– SSL Session: port 443 for secure Web servers and port 465 for
secure mail server, key for connection must be chosen/exchanged.
• Persistent port solution:
– First accesses the service, LinuxDirector create a template between
the given client and the selected server, then create an entry for the
connection in the hash table.
– The template expires in a configurable time, and the template won't
expire until all its connections expire.
– The connections for any port from the client will send to the server
before the template expires.
– The timeout of persistent templates can be configured by users, and
the default is 300 seconds
6/21/2000
Edward Chow
Load Balancing/Web System 30
Performance of LVS-based
Systems
“We ran a very simple LVS-DR arrangement with one PII-400 (2.2.14
kernel)directing about 20,000 HTTP requests/second to a bank of
about 20 Web servers answering with tiny identical dummy responses
for a few minutes. Worked just fine.”
Jerry Glomph Black, Director, Internet & Technical Operations,
RealNetworks
“I had basically (1024) four class-Cs of virtual servers which were
loadbalanced through a LinuxDirector (two, actually -- I used redundant
directors) onto four real servers which each had the four different classCs aliased on them.”
"Ted Pavlic" <tpavlic@netwalk.com>
6/21/2000
Edward Chow
Load Balancing/Web System 31
What is Content Intelligence?
By Erv Johnson, Arrowpoint
Layer 3
(IP)



Switching on
MAC address,
VLANs
IP Routing
802.1 P/Q policy
6/21/2000
Layer 4
(TCP)



Session load
balancing based
on IP address and
TCP port
Network Address
Translation (NAT)
Policies based on
TCP port
Edward Chow
Layer 5-7
(content)




Content Routing
based on:
 Host Tag
 Entire URL
 Dynamic
Cookie location
 File extension
# of rules
# of services
# of services per
content rule
Load Balancing/Web System 32
ArrowPoint’s Content Smart Web Switch
Architecture from CCL viewgraph
4顆MIPS RISC CPU& 512 MB Mem
Control Plane (content Policy Services)
Flowwall
Security
Content
Based QoS
Switch
Switch
Fabric
Switch
Fabric
Flow
Fabric
Managers
Forwarding Plane
Content
Location
Services
Site &
Server
Selection
LAN I/O
Mapped
Row Cache
Shared
Memory
8 Mb Mem
6/21/2000
Switch
Fabric
Up to
16 ports
LAN I/O
Mapped
Row Cache
註明:系統處理能力
1B hits
Edward
Chow per day
Load Balancing/Web System 33
Load Balancing Study
• The current web switches do not take server load or
network bandwidth directly into consideration. How
can we improve them?
• The node with the least connection may have the
heaviest load.
• The current wide area load balancing does not
consider the available/bottleneck bandwidth.
• Lack of simulation and network planning tools for
suggesting network configuration.
6/21/2000
Edward Chow
Load Balancing/Web System 34
Server Load Status Collection
• Three basic approaches:
– Observe response time of requests
– modify web servers to report current queue/processing
speed
– Use web server agent to collect system data
• The 2nd approach requires access to web server code/internal
• We have modified Apache code (v1.3.9) by accumulating size of
pending request (in terms bytes) in active child servers and
diving it with the estimated processing speed.
• Note that it is harder to estimate CGI script of Servlet
processing.
6/21/2000
Edward Chow
Load Balancing/Web System 35
Apache Server Status Report
Apache Server Status for gandalf.uccs.edu
Current Time: Wed Dec 10 00:32:51 1997
Restart Time: Wed Dec 10 00:32:27 1997
Server uptime: 24 seconds
Total accesses: 0 - Total Traffic: 0 kB
CPU Usage: u0 s0 cu0 cs0
0 requests/sec - 0 B/second
1 requests currently being processed, 4 idle servers
...
– Forked web server processes with no work (idle
servers)
– Requests per second
(history) Load Balancing/Web System 36
6/21/2000
Edward Chow
Collecting System Statistics
• Web server agent collects system data
– Run queue (#)
– CPU idle time (%)
– Pages scanned by page daemon (pages/s)
• Web server agent uses
– vmstat 1 2
• every 1 second collect 2 samples
6/21/2000
Edward Chow
Load Balancing/Web System 37
Vmstat Output and Meaning
procs
memory
page
cpu
r
b
w
swap
free
re
mf
pi
po
fr
de
sr
us
sy
id
0
0
0
151748
51636
10
156
151
0
0
0
0
5
11
84
0
0
0
141816
30592
0
1
0
0
0
0
8
0
0
100
– r - # of processes waiting to run (extent)
– sr - # of pages scanned by page daemon to put
back on the free list
– id - % of CPU idle time 100 - (us + sy) = id
(discrete)
6/21/2000
Edward Chow
Load Balancing/Web System 38
Network Bandwidth Measurement
• Bottleneck bandwidth BBw can be measured by
sending burst of packets (of size S) and measuring
the return time gap(Tg). BBw=S/Tg if no interference
• Available bandwidth ABw is harder to measure.
– Cprobe (U. Boston) sends burst of packets and
measures the time-gap between 1st and last msg.
– Estimate ABw based on packet round trip time or
comparison with history of round trip time.
6/21/2000
Edward Chow
Load Balancing/Web System 39
Smart Probe Simulation Results
6/21/2000
Edward Chow
Load Balancing/Web System 40
Weight Calculation
•
Rate each web server with weight based on statistics sent from the web
server agents
Characteristic
Throughput
Averages
Relative
Throughput (%)
Approximate
Thresholds
Idle Servers
CPU
Run queue
Requests per second
Average Ping
Scan Rate
52.19
51.92
51.98
52.09
45.73
11.27
19.68
19.58
19.60
19.64
17.24
4.23
0 servers
0%
30 processes
0 rps
400 ms
200 pages
weight of server= ((19.68*rid) + (19.58*rcpu) +
(19.60*rrq) + (19.64*rrps) + (17.24*rap) + (4.23*rsr))
6/21/2000
Edward Chow
Load Balancing/Web System 41
Weight Calculations (Example)
• CPU idle time had an average throughput of 51.92.
The sum of averages for the characteristics was
265.18. To find the relevant percentage 51.92/265.18
= 0.1958 = 19.58% was then multiplied by the actual
CPU percent idle divided by the approximate
threshold (found to be 100% during the benchmarks),
to get the weight: <cpu weight> = 19.58*(<actual
cpu>/100)
6/21/2000
Edward Chow
Load Balancing/Web System 42
Network Design/Planning Tool
• Need realistic network traffic (Self-similar) load to
exercise the simulator.
• Need tools for
– specifying network topology,
– detecting bottlenecks in the web systems
– suggesting new topology and configurations
6/21/2000
Edward Chow
Load Balancing/Web System 43
Why is the Internet hard to model?
• It’s BIG
– January 2000: > 72 Million Hosts1
• Growing Rapidly
– > 67% per year
• Constantly Changing
• Traffic patterns have high variability
• Causes of High variability
– Client Request Rates
– Server Responses
– Network Topology
6/21/2000
Edward Chow
Load Balancing/Web System 44
Characteristics of
Client Request Rate1
•
•
•
•
Client Sleep Time
Inactive Off Time
Active Off Time
Embedded References
1Barford
and Crovella, Generating Representative Web Workloads for
Network and Server Performance Evaluation, Boston University, BUCS-97-006, 1997
6/21/2000
Edward Chow
Load Balancing/Web System 45
Internet Traffic
Request Pattern
Hourly Request Rate
Percent Total
10
8
Web
6
Telnet
4
Simulated
2
0
0
4
8
12
16
20
Hour
6/21/2000
Edward Chow
Load Balancing/Web System 46
Inactive Off Time
• Time between requests (Think Time)
• Uses a Pareto Distribution
– Shape parameter: a = 1.5
– Lower bound: (k) = 1.0
– To create a random variable x:
• u ~ U(0,1)
• x = k / (1.0-u)^1.0/ a
6/21/2000
Edward Chow
Load Balancing/Web System 47
Inactive Off Time
Inactive Off Time
1.00
P[X<x]
0.80
0.60
0.40
0.20
0.00
0
5
10
15
20
Seconds
6/21/2000
Edward Chow
Load Balancing/Web System 48
Active Off Time
• Time between embedded references
• Uses a Weibull Distribution
– alpha: a = 1.46 (scale parameter)
– beta: b = 0.382 (shape parameter)
– To create a random variable x:
• u ~ U(0,1)
• x = a ( -ln( 1.0 – u ) ^ 1.0/b
6/21/2000
Edward Chow
Load Balancing/Web System 49
Active Off Time
Active Off Time
1
P[X<x]
0.8
0.6
0.4
0.2
0
0
5
10
15
20
25
30
Seconds
6/21/2000
Edward Chow
Load Balancing/Web System 50
Example HTML Document
with Embedded References
<html><head>
<title>CS522 F99 Home Page</title>
</head>
<body background="marble1.jpg">
<BGSOUND SRC="rocky.mid"><embed src="rocky.mid" autostart=true
hidden=true loop=false></embed>
<td ALIGN=CENTER><img SRC="rainbowan.gif" height=15
width=100%></td>
6/21/2000
Edward Chow
Load Balancing/Web System 51
Embedded References
Embedded References
1.00
P[x<X]
0.80
0.60
0.40
0.20
0.00
0
1
2
3
4
5
6
7
8
9
10
Number of References
6/21/2000
Edward Chow
Load Balancing/Web System 52
Server Characteristics
• File Size Distribution
– Body – Lognormal Distribution
– Tail – Pareto Distribution
• Cache Size
• Temporal Locality
• Number of Connections
• System Performance: CPU speed, disk access time, memory,
network interface
6/21/2000
Edward Chow
Load Balancing/Web System 53
File Size Distribution - Body
• Lognormal Distribution
– Build table with 930 values
• Range: 92 <= x <= 9020 bytes
– To create a random variable x:
• u ~ U(0,1)
• if ( u <= 93% ) then
look up value in table[ u * 1000 ]
else
use tail distribution
6/21/2000
Edward Chow
Load Balancing/Web System 54
File Size Distribution - Body
File Size Distribution - Body
1.00
P[x<x]
0.80
0.60
0.40
0.20
0.00
10
100
1000
10000
100000
Bytes
6/21/2000
Edward Chow
Load Balancing/Web System 55
File Size Distribution - Tail
• Pareto Distribution
– Shape parameter: a = 1.5
– Lower Bound: k = 9,020
– To create a random variable x:
• u ~ U(0,1)
• x = k / (1.0 – u) ^ 1.0/a
6/21/2000
Edward Chow
Load Balancing/Web System 56
File Size Distribution – Tail
File Size Distribution - Tail
1.00
P[X<x]
0.80
0.60
0.40
0.20
0.00
0
100000
200000
300000
400000
500000
File Size (bytes)
6/21/2000
Edward Chow
Load Balancing/Web System 57
Self-similarity
• Fractal-like characteristics: Fractals look the same at
all size scales
• Statistical Self-similarity: Empirical data has similar
variability over a wide range of time scales.
6/21/2000
Edward Chow
Load Balancing/Web System 58
Verification of Self-similarity
• Methods
– Observation
– Variance Time Plot
– R/S Plot
– Periodogram
– Whittle Estimator
6/21/2000
Edward Chow
Load Balancing/Web System 59
Self-similarity - Observation
6/21/2000
Edward Chow
Load Balancing/Web System 60
Variance Time Plot
•
Variance Time Plot
-0.5
log10(Normalized
Variance)
Hurst Parameter
H=1–b/2
 b: inverse of the slope
• ½<H<1
• H = 0.7
-1
-1.5
-2
-2.5
-3
-3.5
0
1
2
3
log10(Sample Size)
6/21/2000
Edward Chow
Load Balancing/Web System 61
4
Load Balancing vs. Load
Sharing
• Load Sharing
– System avoids having idle processors by placing new
processes on idle processors first
• Load Balancing
– System attempts to distribute the load equally across all
processors based on some global average.
• Static
– Processes are placed and executed on only one
processor.
• Dynamic
– Processes are initially placed on one processor but at
some point in time the process may be migrated to
another processor based upon some decision criteria.
6/21/2000
Edward Chow
Load Balancing/Web System 62
Load Balancing Algorithms
• Stateless
– Select a processor without consideration of the system state.
• Round Robin
• Random
• State-based
– Select a processor based upon some knowledge of the
system state.
• Greedy
• Subset
• Stochastic
6/21/2000
Edward Chow
Load Balancing/Web System 63
Simulation Entities
•
•
•
•
Request
Client
Load Balance Manager
Server
6/21/2000
Edward Chow
Load Balancing/Web System 64
Request Event Loop
6/21/2000
Edward Chow
Load Balancing/Web System 65
Experimental Design
• Cooperative Environment
– For each algorithm (round robin, random, greedy,
subset, stochastic)
• Eight Servers with 1, 4 Connections
• 8, 16, 32, 64, 128, 256,512 Clients
• 1, 2, 4 Load Balance Managers
6/21/2000
Edward Chow
Load Balancing/Web System 66
Servers with One Connection
1 LBM - 8 Servers - 1 Connection
2 LBMs - 8 Servers - 1 Connection
80
Round Robin
60
Random
40
Greedy
Subset
20
Stochastic
Response Time
(seconds)
Response Time
(seconds)
80
0
Round Robin
60
Random
40
Greedy
Subset
20
Stochastic
0
0
0.2
0.4
0.6
0.8
1
0
0.2
Utilization
0.4
0.6
0.8
1
Utilization
4 LBMs - 8 Servers - 1 Connection
Response Time
(seconds)
80
Round Robin
60
Random
40
Greedy
Subset
20
Stochastic
0
0
0.2
0.4
0.6
0.8
1
Utilization
6/21/2000
Edward Chow
Load Balancing/Web System 67
Global vs. Local Info.
Random
140
120
100
80
60
40
20
0
Response Time
Response Time
Round Robin
1 LBM
2 LBMs
4 LBMs
0
0.2
0.4
0.6
0.8
140
120
100
80
60
40
20
0
1 LBM
2 LBMs
4 LBMs
1
0
0.2
Utilization
0.4
70
60
50
40
30
20
10
0
1 LBM
2 LBMs
4 LBMs
0.2
0.4
0.6
0.8
1
Subset
Response Time
Response Time
Greedy
0
0.6
Utilization
0.8
1
70
60
50
40
30
20
10
0
1 LBM
2 LBMs
4 LBMs
0
0.2
Utilization
0.4
0.6
0.8
1
Utilization
Response Time
Stochastic
70
60
50
40
30
20
10
0
1 LBM
2 LBMs
4 LBMs
0
6/21/2000
0.2
0.4
0.6
Utilization
0.8
Edward Chow
1
Load Balancing/Web System 68
Servers with Four
Connections
1 LBM - 8 Servers - 4 Connections
20
15
10
5
0
Round Robin
Response Time
(seconds)
Response Time
(seconds)
2 LBMs - 8 Servers - 4 Connections
Random
Greedy
0
0.5
Subset
1
Stochastic
Utilization
20
15
10
5
0
Round Robin
Random
Greedy
0
0.5
Utilization
1
Subset
Stochastic
Responss Time
(seconds)
4 LBMs - 8 Servers - 4 Connections
20
15
10
5
0
Round Robin
Random
Greedy
0
0.5
Utilization
6/21/2000
Edward Chow
1
Subset
Stochastic
Load Balancing/Web System 69
Global vs. Local Info.
Round Robin
Random
20
15
Response Time
(seconds)
Response Time
(seconds)
20
1 LBM
10
2 LBMs
4 LBMs
5
0
15
1 LBM
10
2 LBMs
4 LBMs
5
0
0
0.2
0.4
0.6
0.8
1
0
0.2
Utilization
0.4
Greedy
0.8
1
Subset
12
10
8
1 LBM
6
2 LBMs
4
4 LBMs
Response Time
(seconds)
12
Response Time
(seconds)
0.6
Uitilization
2
0
10
8
1 LBM
6
2 LBMs
4
4 LBMs
2
0
0
0.2
0.4
0.6
0.8
1
0
0.2
Utilization
0.4
0.6
0.8
1
Utilization
Stochastic
Response Time
(seconds)
12
10
8
1 LBM
6
2 LBMs
4
4 LBMs
2
0
0
6/21/2000
0.2
0.4
0.6
0.8
Utilization
Edward
Chow
1
Load Balancing/Web System 70
Experimental Design
• Adversarial Environment
– For each algorithm (greedy, subset, stochastic)
• Eight Servers with 1, 4 Connections
• 8, 16, 32, 64, 128, 256,512 Clients
• 4 Load Balance Managers with 1, 2, 3 Random
Load Balance Managers as adversaries
6/21/2000
Edward Chow
Load Balancing/Web System 71
Servers with One Connection
4 LBMs - 1 Random - 8 Servers
80
60
Greedy
40
Subset
Stochastic
20
0
0
0.2
0.4
0.6
0.8
Response Time
Response Time
4 LBMs - 0 Random - 8 Servers
80
60
Greedy
40
Subset
Stochastic
20
0
1
0
0.2
Utilization
Greedy
40
Subset
Stochastic
20
0
0.6
0.8
Response Time
Response Time
60
0.4
1
1
80
60
Greedy
40
Subset
Stochastic
20
0
0
Utilization
6/21/2000
0.8
4 LBMs - 3 Random - 8 Servers
80
0.2
0.6
Utilization
4 LBMs - 2 Random - 8 Servers
0
0.4
0.2
0.4
0.6
0.8
1
Utilization
Edward Chow
Load Balancing/Web System 72
LBM w/ Adversaries
Greedy
150
No Adversaries
Response Time
Response Rate
Subset
1 Adversary
100
2 Adversaries
50
3 Adversaries
All Random
0
0
0.2
0.4
0.6
0.8
1
150
No Adversaries
1 Adversary
100
2 Adversaries
50
3 Adversaries
All Random
0
0
Utilization
0.2
0.4
0.6
0.8
1
Utilization
Response Time
Stochastic
150
No Adversaries
1 Adversary
100
2 Adversaries
50
3 Adversaries
All Random
0
0
6/21/2000
0.2
0.4
0.6
Utilization
0.8
Edward Chow
1
Load Balancing/Web System 73
Servers with Four Connection
12
10
8
6
4
2
0
4 LBMs - 1 Random - 8 Servers
Greedy
Subset
Stochastic
0
0.2
0.4
0.6
0.8
1
Response Time
Response Time
4 LBMs - 0 Random - 8 Servers
12
10
8
6
Greedy
Subset
4
2
0
Stochastic
0
0.2
Utilization
Greedy
Subset
4
2
0
Stochastic
0.4
0.6
0.8
1
Utilization
6/21/2000
0.8
1
4 LBMs - 3 Random - 8 Servers
Response Time
Response Time
12
10
8
6
0.2
0.6
Utilization
4 LBMs - 2 Random - 8 Servers
0
0.4
14
12
10
8
6
4
2
0
Greedy
Subset
Stochastic
0
0.2
0.4
0.6
0.8
1
Utilization
Edward Chow
Load Balancing/Web System 74
LBM w/ Adversaries
Subset
Response Time
Response Time
Greedy
20
No adversaries
15
1 Adversary
10
2 Adversaries
3 Adversaries
5
All Random
0
0
0.2
0.4
0.6
0.8
1
20
No Adversaries
15
1 Adversary
10
2 Adversaries
3 Adversaries
5
All Random
0
0
Utilization
0.2
0.4
0.6
0.8
1
Utilization
Response Time
Stochastic
20
No Adversaries
15
1 Adversary
10
2 Adversaries
3 Adversaries
5
All Random
0
0
6/21/2000
0.2
0.4
0.6
0.8
1
Utilization
Edward
Chow
Load Balancing/Web System 75
Analysis of Experimental
Results
• Single Connection
– Global
• Greedy: 2.2-27.6x improvement in Response
Time
• Subset: 1.8-3.4x
• Stochastic: 1.3-2.2x
– Local
• Greedy: 2.2-6.6x
• Subset: 1.4-2.5x
• Stochastic: 1.1-2.0x
6/21/2000
Edward Chow
Load Balancing/Web System 76
Analysis of Experimental
Results – cont.
• Four Connections
– Global
• Greedy: 1.7-4.3x improvement in Response
Time
• Subset: 1.8-3.4x
• Stochastic: 1.1-2.5x
– Local
• Greedy: 1.0-4.1x
• Subset: 1.0-3.0x
• Stochastic: 1.0-2.3x
6/21/2000
Edward Chow
Load Balancing/Web System 77
Analysis of Experimental
Results – cont.
• Single Connection w/ Adversaries
– Greedy: 1.1-4.3x
– Subset: 1.1-2.1x
– Stochastic: 1.0-1.9x
6/21/2000
Edward Chow
Load Balancing/Web System 78
Analysis of Experimental
Results – cont.
• Four Connection w/ Adversaries
– Greedy: 1.0-3.7x
– Subset: 1.0-2.7x
– Stochastic: 1.0-2.1x
6/21/2000
Edward Chow
Load Balancing/Web System 79
NetLobars: Load Balancing Research Simulator
6/21/2000
Edward Chow
Load Balancing/Web System 80
System Bottleneck Detection
6/21/2000
Edward Chow
Load Balancing/Web System 81
Conclusion
•
•
•
•
•
Survey of some major web switches
Discuss their features and related support functions
Introduction of its low-cost competitor (LVS)
Discuss future improvement directions
Present load balancing Simulation result with realistic
web traffic modeling.
• Present preliminary design of a network load
balancing research tool
6/21/2000
Edward Chow
Load Balancing/Web System 82