Game Theory and the Internet
Flow and Congestion Control
“Differential QoS and Pricing in Networks:
where flow-control meets game theory.”
Peter Key and Derek McAuley, 1999
2
Notation
rR
jJ
xr
A = (Ajr)
Ax
user / network route
network resource (eg link)
resources demanded by user r
(eg bandwidth per unit time)
1 if resource j is consumed by user r
0 otherwise
(assume fixed routing)
is the (offered) load to the resource
vector
3
Notation cont.
Ur(xr)
Cj(y)
utility function for user r, relating
bandwitdth to worth
cost of loading y on resource j
Assume Ur(x) is concave:
f((x1 + x2)/2)  (f(x1) + f(x2))/2
Assume Cj(y) is increasing, differentiable.
4
“Social Optimum”
Maximize
U(x)=r Ur(xr) - j Cj(r Ajrxr)
over x  0.
5
If Cj(y) are convex functions, the optimum for
nonzero xr satisfies
Ur’(xr)=j jAjr
where the j are associated with the shadow
prices:
j = pj (j Ajr xr)
with
pj(y) = dCj(y)/dy
(Shadow price: marginal increment in cost at the
resource for a marginal increment in load.)
6
User optimum
Assume users are charged ar per amount of data
sent.
Maximize:
Ur(xr) – arxr
over xr  0
7
User optimum cont.
Solution:
x*, where Ur’(x*) = ar,
if Ur’(0) > ar
xr = 
0,
otherwise
The user accepts any amount of bandwidth offered
up to x*, if the prices are “rational”, i.e. Ur’(0) > ar
User optimum coincides with social optimum if the
prices are “right”:
ar = jr j
8
TCP congestion control (in brief)
Congestion is controlled by changing the rate of
sending packets.
When a resource in the network becomes
congested, some packets will be lost.
Loss of a packet is interpreted by the destination
as an indication of congestion.
The source is notified to slow down transmission.
The transmission rate is gradually increased until a
new notification of congestion.
This allows the utilization of available bandwidth.
9
Framework
Users generate load x(t) and receive feedback
signals f(t), which are related to shadow prices.
For example certain packets receive “marks”.
A packet is marked if and only if it contributed to a
loss event, i.e. the removal of the packet arriving
at some resource would result in one less packet
loss at that resource.
So, mark all packets arriving in an overloaded
resource.
10
More specific…
Consider single unbuffered resource, capacity C.
The load Y is a random variable with mean y,
following Poisson or Gaussian distribution.
The expected loss is E[(Y - C)+]
The shadow price can be shown to be equal to the
probability of resource saturation Psat = P{Y  C}
11
So marking scheme for unbuffered resource:
Mark all the packets that contribute to the load
when the resource is overloaded, otherwise
mark none.
The probability of loss: Ploss = E[(Y – C)+]/E[Y]
Typically Ploss << Psat, so many more packets are
marked than lost.
12
Flow control
Optimality depends on the utility functions of the
users, but these functions are unknown to the
network.
So:
Let the network determine the prices and let the
users place demands on the network, which the
network will try to meet.
The users can behave as they wish, possibly
performing some user optimization.
13
Redefinition of flow control algorithm
Implement a means to feed back information to the
user: mark a packet
Every resource used for the transmission of the
packet has the ability to mark it.
But the probability of multiple markings is very
small, so one bit can be used to approximate a
“mark”.
This bit can be incorporated in most existing
protocols easily.
14
Algorithms for flow control
“Willingness to pay” scheme:
xt+1 = xt + k(w – ft)
w represents “willingness to pay”,
k affects the rate of convergence.
“TCP-like” scheme:
xt+1 = xt + k(w – ft xt/2)
15
Experimentation
Run a distributed emulation of a network, where
users are free to invent their own algorithms or
behave as they wish.
It is a distributed multi-user game, where the users
try to optimize their own objective.
16
Proposed framework
Users play against a network representing the
resource system:
• The game consists of a distributed set of users
and a Network, emulated by a PC
• A protocol interfaces between the users and the
Network
• Users can interact (connect) across the Internet
• Users can interact in real time or through
programs
• Game time can be slowed down or speeded up
17
• Users select a destination and generate load
(packets), which are sent to the Network.
• The Network decides which resources to
allocate to the user and sends back a set of
marks, depending on the state of the resources.
• The Network also sends to the user signals
about the start/end of the game and its relative
speed compared to real time.
18
Protocol for a distributed game
<packet> : <pkt>=<destination><token><size>
<token> is generated by the user and used to
detect loss. <destination> is used to identify a
route in the Network
<ACK> : <ACK>=<token><mark>
<token> can be void or corrupted. <mark> gives
the number of marks in the packet
<time> : <time>=<sec>
sends heartbeats back to the user giving time in
the game world
19
Typical sequence of events
User requests to join the game
Receives acknowledgement and relative speed
Receives notice of start of game
Repeat
Generate token
Calculate load (packet size)
Send <packet>=<destination><token><size>
Receive <ACK>=<token><mark>
Receive <time>=<sec>
Until End of game
User ends session
20
Typical objectives
We consider optimality from the perspective of the
user, not the network.
Reminder (and some slight changes):
User r generates load xt = x(t) at time t.
Number of marked packets is ft = f(t).
Cost factor a, so cost incurred is: aft.
21
Typical objectives cont.
• Control cost: Keep the cost constant and equal
to w (“willingness to pay”). L is some loss
function
minimize L(w – aft)
• Throughput: Maximize throughput minus the
cost
maximize Tt=1(xt-aft)/T
• Constraint throughput: Same as above given a
cost constraint 
maximize Tt=1(xt-aft)/T, such that Tt=1aft  
22
Typical objectives cont.
• Maximise net utility: Given a utility function U
maximize t(U(xt) - aft)
• File transfer: Transfer an amount of data F at
minimum cost
minimize taft, such that txt = F
• Fixed time file transfer: Same as above given a
time constraint T
minimize Tt=1aft, such that Tt=1xt = F
23
Conclusions
It is possible to provide differential quality of
service to users by means of a simple pricing
scheme, where the network sends back the
correct congestion prices to the user and the
users choose how to react.
Use the presented framework and play the defined
distributed game, to assess the effectiveness
and stability of various flow control schemes.
24
A modest proposal for preventing Internet
congestion
Andrew Odlyzko, 1997
25
The problem of congestion
There is a demand for improved Quality of Service
on the Internet, especially for reducing
congestion.
Proposed schemes are usually complicated and
have high cost in development and operation.
The basic problem is one of allocating a limited
resource.
A solution will have to involve some pricing
mechanism.
26
The Paris Metro (or Subway)
Until some years ago, the Paris Metro had two
different classes of cars.
1st and 2nd class cars were identical in the number
and quality of seats.
1st class cars were more expensive and therefore
less congested. Used by people who wanted a
seat, silence etc.
The system was self-regulated: if 1st class cars
became very popular, some people would
decide they were not worth the extra cost and
move to 2nd class, thereby reducing congestion
in 1st class.
27
PMP
PMP = Paris Metro Protocol
Basic idea: partition the network into several
logically separate networks.
Each network has a fraction of the capacity of the
entire network.
All networks route packets using existing protocols
on a “best effort” basis.
Every network charges a different price for the
same transfer.
28
The pricing mechanism is expected to affect user
demand and thus modify the traffic loads.
Networks with higher prices will be less congested,
offering better QoS.
PMP will not provide QoS guarantees, only
expected levels of service.
Not necessarily bad. QoS demands can be
complicated and widely varying, impossible to
satisfy all of them.
PMP will offer few classes of service satisfying the
majority of applications.
29
PMP advantages
Approaches that don’t use any network scheme
and just increase capacity fail. With no pricing
mechanism all packets travel at the highest
priority and traffic grows to exhaust all available
capacity.
Policies trying to control bandwidth and provide
QoS guarantees require the coordination of
network nodes along the transmission path,
causing delays and “reinventing telephone
technologies”.
30
Details
The number of subnetworks in PMP should be
small (<5). This minimizes losses from not
aggregating all the traffic and fits consumer
preferences for simple schemes.
Every subnetwork need not be assigned a fixed
fraction of the total capacity. But this
assignment, as well as the prices, should stay
constant for extended periods of time, to allow
usage patterns to stabilize.
31
Problems and Solutions
To provide a higher QoS than the current Internet,
the low-traffic networks will have to be lightly
loaded. Will the capacity utilization have to be so
low that it proves to be uneconomic?
We don’t know. There is not enough information
about the operation and performance of the
Internet. The user requirements are not precisely
defined.
32
Problems and Solutions
How to set the network prices?
Use the variations in traffic patterns with the time
of day. (Also customer surveys, complaints, etc.)
33
Problems and solutions cont.
The average performance of the networks can be
unstable.
For example, an extended congestion period on
the lowest-priced channel might lead some
users to decide to pay extra and send their
packets to the higher-priced networks.
Degradation of service on one channel will thus
propagate to other channels.
34
Problems and solutions cont.
So, how to keep the average performance of the
networks stable?
Segregate the different types of services on
different networks. E.g. introduce artificial packet
delays in the lowest-priced network, making it
unusable for certain applications. (“damaged
goods” approach)
This can be used to induce a more even load on
the separate networks.
35
PMP implementation
Use current protocols:
IPv4 packets have an unused 3 bit priority field.
Protocols without priority fields can operate at
the lowest cost (and priority) network.
Set the cost of the lowest-priority network to zero
(at least initially).
Use software to handle all traffic on a user’s
machine according to user set rules, until
applications are rewritten to take the multiple
network choices into account.
36
PMP implementation cont.
Also the service providers will have to:
Change the router software to maintain separate
queues for the different channels.
Install software to count the packets sent by each
user.
Special provisions needed for multicasting.
37
The obstacle of the users
and how to overcome it.
[Disclaimer: The following proposals represent
only the opinions of the author of the paper.]
The need for usage-sensitive pricing appears
irresistible. It has impeccable economic logic
and practical evidence behind it.
But it collides with the user’s preference for flatrate pricing.
38
Flat rates
Flat rate pricing can cause significant problems.
Example: America Online
When it switched to flat-rate pricing, usage
increased considerably.
Users had trouble getting a new connection and
so left their connections open even when not
using them.
39
Flat rates cont.
Perverse incentives exist on the Internet to
increase data transfers
[Note: The paper is from 1997]
To improve performance in Web surfing users
use programs that prefetch linked pages when
the connection is idle. With flat-rate pricing there
is an incentive for the users to use them
indiscriminately, even for minor benefits.
40
Flat rates cont.
The survival of the Internet owes much to altruism
and ignorance.
TCP slows down transmissions when congestion
occurs.
Rogue versions of TCP can be used to speed up
transmission of packets when delays or losses
occur. No extra cost is incurred to their users
under flat-rate pricing. Their extensive use can
cause the network to collapse.
41
So why are flat rates used?
• Users know how much they will have to
pay.
• Consumers overestimate their actual
usage of a service.
• Hassle factor: consumers don’t have to
worry if every use of the service is worth
its money.
• No need for traffic measurement and
charging infrastructure.
42
How to introduce pricing without
user complaints.
Users have accepted usage-sensitive rates
in a variety of services (e.g. long distance
calls)
It should be possible to persuade users to
accept usage-sensitive pricing.
PMP should make the transition easier that
with most other schemes, since the
lowest-priced channel could be offered
initially at zero cost per packet, thus
behaving just like today’s internet.
43
PMP offers a simple and easily understood
pricing plan, which fits consumer desires.
It does not offer service guarantees. No
problem: most purchases are made on the
basis of expectations (magazine
subscriptions, clothes, meals etc.)
Consumers are willing to accept
occasional deviations from the expected
quality of service, as long as the average
quality is predictable enough.
44
Auctions and consumers
Consumer and business behavior can be
economically irrational.
Free enterprise companies prefer the socialist
method of rationing by queue to that of rationing
by price.
Laws against ticket scalping are common and
widely supported. But to most economists
scalpers fulfill a socially useful role: they provide
tickets to those that are willing to pay the most
for them. Why don’t theaters and sports arenas
adjust ticket prices to clear the market and
appropriate some of the gain the scalpers
obtain?
45
Auctions and consumers cont.
It seems that the cause lies in the
consumers’ irrationall economic behavior.
The challenge is to design pricing schemes
that approach the efficiency of auction
mechanism, but respect the consumer
aversion to the auction.
It is important to convince consumers of the
fairness of pricing schemes.
46
Other pricing schemes
• Charge packets when they contribute to
congestion.
– Complicated implementation
• Use a set of service classes and priorities
– Substantial overhead
– Low priority classes can be denied any
bandwidth if traffic from higher priority classes
exceeds some threshold.
47
Gains from network segmentation
We will examine simple models of the gains
obtained by logically separate networks working
at different utilization levels.
48
Two types of demand for data tranfer.
Type A users are insensitive to delays (e.g. bulk
file transfers)
When the price per byte is x, type A users wish to
send
ax-1e-x
bytes (per hour).
They generate network revenues:
ae-x
49
The cost of operating a network that carries w
bytes is
cw3/4
for some constant c > 0.
This is a conservative assumption, since it
corresponds to less than a 16% reduction in
costs when the network doubles in size (23/4 =
1.681…). In practice the reductions are higher,
corresponding to even smaller exponents.
50
If there are only type A users, the cost of the
network will equal the revenues:
ae-x = c(ax-1e-x)3/4
x3e-x = a-1c4
The unique maximum of x3e-x occurs at x = 3 and
equals 1.344…
For c4 > 27ae-3 there is no price x that will recover
costs.
For c4 < 27ae-3 there are two solutions for x and
we prefer the smaller one xA, that corresponds to
higher traffic and revenue.
51
Type B users use a network only when its
utilization rate is at most half of that acceptable
to type A users.
This assumption is conservative because even
small reductions in network loads produce
substantial improvements in service.
When the price per byte is x, type B users wish to
send
bx-1e-x
bytes (per hour).
They generate network revenues:
be-x
52
Constructing a separate network for type B users
will cost
c(2bx-1e-x)3/4
The 2 is because of the lower utilization rate.
A price x that will equalize revenue and costs is
found by solving:
x3e-x = 8b-1c4
and exists when 27b  8c4e3. Let xB denote the
minimal solution (if more than one exists).
53
Single network for both
Case 1: Single network with single price for both
types of users.
At price x would have revenue
(a+b)e-x
and will cost
c(2(a+b)x-1e-x)3/4
because its average utilization would have to be
half of a network meant for type A users alone.
54
The price x that equalizes revenue and costs is
found by solving:
x3e-x = 8(a+b)-1c4
and exists when 27(a+b)  8c4e3. Let xAB denote
the minimal solution.
If b > 7a (demand from type B users is larger), type
A users benefit from lower prices: xAB < xA
If b << a, then xAB > xA and type A users pay more
(and also get better service but they don’t need
it)
Type B users benefit always from having type A
users on their network, because prices are
always reduced.
55
Separate logical networks
Case 2: Same physical network, logically
separated into two for each type of user.
Price for type A users is y, for type B is z.
Revenue:
ae-y + be-z
Cost of the network:
c(ay-1e-y + 2bz-1e-z)-3/4
56
y and z must satisfy:
ae-y + be-z = c(ay-1e-y + 2bz-1e-z)-3/4
Great freedom of choices.
If y   or z   we reduce to networks
exclusively used by type A and type B users
respectively.
57
Example 1
a = b = 3, c = 1
xA = 0.952
xB does not exist
xAB = 2.784
y = 0.9
z = 1.338
network
A only
B only
A + B on single
network
A + B on logically
separate networks
traffic revenue
1.215
1.157
0.133
0.37
1.942
2.006
Type A users gain about 5.5%.
Type B gain in that they are offered a service.
58
Example 2
a = 20, b = 10, c = 1
xA = 0.424
xB = 1.563
xAB = 0.856
y = 0.42
z = 0.606
network
A only
B only
A + B on single
network
A + B on logically
separate networks
traffic revenue
30.829 13.083
1.34
2.095
14.88
12.74
40.27
18.59
Type A users experience once more only a slight gain.
Type B users pay a lot less.
59
Example 3
a = 10, b = 20, c = 1
xA = 0.559
xB = 1.043
xAB = 0.856
y = 0.53
z = 0.693
network
traffic
revenue
A only
B only
A + B on single
network
10.22
6.75
14.88
5.71
7.04
12.74
A + B on logically
separate networks
25.51
15.88
The network provider is almost indifferent between having
two separate networks and one giving all traffic the
quality demanded by type B users.
60
In all the examples, gains to type A users are
small.
This might explain why there has not been much
pressure from users of the Internet (whose
application are designed to work well in the
presence of congestion) for higher quality of
service.
In the examples, a and b are comparable,
corresponding to comparable traffic from type A
and type B users.
This might not be true now, but will most probably
be the case, if the quality of service on the
Internet is improved.
61