Understanding Networked Applications:
A First Course
Chapter 17
by
David G. Messerschmitt
Summary
•
•
•
•
Performance and quality
Concurrency
Scalability
Operating systems
Understanding Networked Applications
2
A First Course
What is performance
• Characterizing non-functional application
attributes
• Quantitative metrics
– Latency, speed, etc
– Parameterized by load, users, etc
• Performance of application as a whole
usually not meaningful
– Subdivide into tasks
Understanding Networked Applications
3
A First Course
Tasks
• Collection of actions that are not meaningfully
separated
– Only completed task is of interest; partial results not
useful
– How does this relate to atomicity in transactions?
• For repetitive tasks, users or operators often care
about
– Task completion time (sec)
– Task throughput (tasks/sec)
Understanding Networked Applications
4
A First Course
Important questions
• For any application
• What are the tasks?
– Which is important: completion time,
throughput, or both, or something else?
– What are the quantitative objectives?
Understanding Networked Applications
5
A First Course
Example: Web surfing
• Task:
– User: Click on URL, retrieve page, display
– Server: HTTP request received, HTML page
formed as a message, message transmitted
• What performance metric(s) are important
to:
– User?
– Server?
Understanding Networked Applications
6
A First Course
Examples
• What are the tasks and their appropriate
performance metrics for the following
applications:
–
–
–
–
–
Schwab stock trading system?
Stock ticker?
ATM system?
Travel reservation system?
Remote conferencing?
Understanding Networked Applications
7
A First Course
Typical components of task
completion time
Client
Formulate
request
Server
Message latency
Processing time to
formulate response
Message latency
Interpret
response
Understanding Networked Applications
8
A First Course
Completion time and throughput
are unrelated
Concurrency
Large completion time
and high throughput
Time
Small completion time
and low throughput
Understanding Networked Applications
9
A First Course
Concurrency
• Two tasks are concurrent if they overlap in
time
• Four cases:
Time
Understanding Networked Applications
10
A First Course
Concurrency may be….
• …a way to satisfy intrinsic application
requrements
– e.g. a number of users may generate concurrent tasks
• …a way to improve fairness
• …a way to increase task throughput
– e.g. assign tasks to different hosts to get more done
• …a way to reduce task completion time
– e.g. divide a task into subtasks and assign those
subtasks to different hosts
Understanding Networked Applications
11
A First Course
Concurrency for fairness
Tasks not concurrent
Scheduling delay
Time
Tasks concurrent
Short task completes sooner
Understanding Networked Applications
12
A First Course
Concurrency may contribute to
fairness on a single host
Tasks not concurrent
Scheduling delay
Time
Tasks concurrent
Even though each task takes longer,
short task still completes sooner
Understanding Networked Applications
13
A First Course
Concurrency on a single host
User view
Reality
Why do this?
What is effect on
completion time of
each task?
Time slice
Context switch
Understanding Networked Applications
14
A First Course
Why networks use packets
Messages arrive at a communication link
simultaneously
Transmitting one message at a time
Transmitting messages concurrently
Divide each message into packets
Understanding Networked Applications
15
Message arrives
earlier
A First Course
Isolation of transactions
Two concurrent
transactions
Transaction_1
Shared resources
Give the same
result as:
Or:
Transaction_2
Transaction_1
Transaction_2
Transaction_2
Transaction_1
Understanding Networked Applications
16
A First Course
Messages among active objects
Task
Task
Queue
An active object
delegates a task to
another active object,
which retrieves from
queue when ready
Concurrency
Understanding Networked Applications
17
A First Course
Understanding Networked Applications:
A First Course
Scalability
by
David G. Messerschmitt
Definition
• An application is scalable if its performance
parameters can be improved as necessary
– Adding but not replacing equipment is
acceptable
– Re-configuration but not re-programming is
acceptable
– Normally equipment cost should increase no
more than linearly with performance
Understanding Networked Applications
19
A First Course
Basic technique
• Concurrent tasks are a key to scalability
• Some obstacles:
–
–
–
–
Concurrent tasks aren’t there
Dependency among tasks
Communication overhead
Variations in task loads result in congestion
Understanding Networked Applications
20
A First Course
Some problems to try to avoid
• Blocking
– Processing blocked waiting for external events
– Make sure processor can work on something
else
• Duplication
– In creating concurrent tasks, duplication of
effort is created
Understanding Networked Applications
21
A First Course
Problems to avoid (con’t)
• Load balancing
– Workload not equal among processors
– Some processors not fully utilized
– More difficult if concurrent tasks are dependent
• Congestion
– Workload fluctuates
– Makes load balancing more difficult because
utilization can’t be predicted
Understanding Networked Applications
22
A First Course
Source of congestion
Server
Irregular
arrival of
tasks
Mux
&
queue
Understanding Networked Applications
Irregular
task
completion
times
23
A First Course
Congestion model
Queue
1. Tasks arrive at
random times
3. Task
service time
is random
2. Tasks wait in
queue waiting
for processing
Understanding Networked Applications
24
A First Course
Statistical waiting time
Waiting time
30
25
A specific statistical
model for arrival
times and service
times is assumed
20
15
Tradeoff between task
completion time (waiting
time) and task throughput
(utilization)
Series1
10
5
0
0.1
0.3
0.5
0.7
0.9
Utilization
Understanding Networked Applications
25
A First Course
Example: Web server
How do users know the server is reaching
full utilization?
When a single Web server/processor is
exhausted, what do we do?
Understanding Networked Applications
26
A First Course
Understanding Networked Applications:
A First Course
Operating systems
Some OS goals
• Abstraction: hide hardware details
• Manage concurrency
– Multitasking (time-slicing) concurrency
– Process and thread
– Inter-process communicaton
• Manage resources
– Memory, storage, processing, network access
Understanding Networked Applications
28
A First Course
Memory/storage hierarchy
Processor cache
Main memory
Speed
Online (non-volatile) external storage
Archive storage
Off-line
Size
Understanding Networked Applications
29
A First Course
Processor
Processor
Shared
memory
Network
interface
Processor
Private
memory
Processor
Network
interface
Processor
Private
memory
Processor
Private
memory
High-speed communication bus
Understanding Networked Applications
30
A First Course
Process
Memory
Memory
Process
Kernel
level
OS kernel
Network
Storage
Understanding Networked Applications
User
level
Memory
31
Resources
A First Course
Process
User
mode
Kernel
mode
write_file(),
send_message(),
etc.
Kernel performs
service, then
returns
Time
Understanding Networked Applications
32
A First Course
Threads
Thread n
Thread n+1
Request to
HTTP server
to return
homepage
Another request
Page returned
Shared
repository
of WWW
links
Understanding Networked Applications
33
Another return
A First Course
Understanding Networked Applications:
A First Course
Supplements
By
David G. Messerschmitt
Execution models for objects
• Active object
– Independent center of activity
– Works without methods being called
• Passive object
– Works only in response to method invocations
Understanding Networked Applications
35
A First Course
Two observations
• For anything to get done, there has to be at least
one active object!
• Natural for each active object to work on one task
at a time
– If so there must be at least two active objects for
concurrency
– (although in principle an active object could time slice
among tasks, this is much more complicated and not a
good idea)
Understanding Networked Applications
36
A First Course
Natural for each active object to
work on one task
Active object
Passive objects
Executing
One object is executing
at a time
Task
Time
Understanding Networked Applications
37
A First Course
Concurrency with two active
objects
Active
object
Passive objects
Understanding Networked Applications
38
Active
object
A First Course
A possible problem
Active
object
Passive objects
Active
object
Concurrency
Potential resource
conflict
Understanding Networked Applications
39
A First Course
Without pipelining
Task
Stage1
Stage2
Stage3
Partially completed
piece of work
Understanding Networked Applications
40
A First Course
With pipelining
Task_start
Stage1
Stage2
Stage3
Task_complete
Repetitive
tasks
Understanding Networked Applications
41
A First Course