Duke Systems
Servers
and
A Little Bit of Networking
Jeff Chase
Duke University
Unix process view: data
A process has multiple channels
for data movement in and out of
the process (I/O).
The channels are typed.
I/O channels
(“file descriptors”)
stdin
stdout
tty
Process
stderr
Each channel is named
by a file descriptor.
pipe
Thread
socket
The parent process and
parent program set up and
control the channels for a
child (until exec).
Files
Program
Servers and the cloud
Where is your application?
Where is your data?
Where is your OS?
networked
server “cloud”
Cloud and Software-as-a-Service (SaaS)
Rapid evolution, no user upgrade, no user data management.
Agile/elastic deployment on clusters and virtual cloud utilityinfrastructure.
Networked services: big picture
client host
NIC
device
client
applications
kernel
network
software
Internet
“cloud”
Data is sent on the
network as messages
called packets.
server hosts
with server
applications
Sockets
socket
A socket is a buffered
channel for passing
data between
processes over a
network.
client
int sd = socket(<internet stream>);
gethostbyname(“www.cs.duke.edu”);
<make a sockaddr_in struct>
<install host IP address and port>
connect(sd, <sockaddr_in>);
write(sd, “abcdefg”, 7);
read(sd, ….);
• The socket() system call creates a socket object.
• Other calls establish connections between socket pairs (e.g, connect).
• A file descriptor for a connected socket is bidirectional.
• Write bytes at one end; read returns them at the other end.
• The read syscall blocks if the (stream) socket is “empty”.
• The write syscall blocks if the (stream) socket is “full”.
• Both read and write fail if there is no valid connection.
Sockets: client/server example
request
“GET /images/fish.gif HTTP/1.1”
reply
client (initiator)
server
sd = socket(…);
connect(sd, name);
write(sd, request…);
read(sd, reply…);
close(sd);
s = socket(…);
bind(s, name);
listen(s, 10);
sd = accept(s);
read(sd, request…);
write(sd, reply…);
close(sd);
Socket syscalls
connect(csd, <IP address and port>). For a client: connect the socket named
by descriptor csd to a server at the specified IP address and port. Block until
the connection is established.
bind(sd, <…port>). For a server: associate the socket named by descriptor sd
with a port number reachable at an IP address of the host machine. Does not
block, but may fail, e.g., if some other process is already bound to the port.
listen(sd, qsize). For a server: indicate that the socket named by descriptor sd
is a server socket. When a connect request arrives for its port, establish the
connection and place it on the accept queue (unless the accept queue is full).
Listen does not block: it merely sets some parameters on the socket.
accept(sd, …). For a server: accept a connection from the accept queue for
the server socket named by descriptor sd. Block if the accept queue is empty.
Returns the IP address and port of the client for this connection, and a new
socket descriptor csd for the connection.
Given a socket descriptor csd for an established connection (from a completed
connect or accept) a process may use write (or send) to send bytes to the
connection peer, and may use read (or recv) to receive bytes sent by the peer.
catserver
...
struct sockaddr_in socket_addr;
sock = socket(PF_INET, SOCK_STREAM, 0);
memset(&socket_addr, 0, sizeof socket_addr);
socket_addr.sin_family = PF_INET;
socket_addr.sin_port = htons(port);
socket_addr.sin_addr.s_addr = htonl(INADDR_ANY);
if (bind(sock, (struct sockaddr *) &socket_addr, sizeof socket_addr) < 0) {
perror("bind failed");
exit(1);
}
listen(sock, 10);
while (1) {
int acceptsock = accept(sock, NULL, NULL);
forkme(acceptsock, prog, argv); /* fork/exec cat */
close(acceptsock);
}
}
Web Server
Inside your Web server
Server application
(Apache,
Tomcat/Java, etc)
accept
queue
packet
queues
listen
queue
disk
queue
Server operations
create socket(s)
bind to port number(s)
listen to advertise port
wait for client to arrive on port
(select/poll/epoll of ports)
accept client connection
read or recv request
write or send response
close client socket
Socket descriptors in Unix
Disclaimer:
this drawing
user space
kernel
spaceis oversimplified
file
int fd
pointer
per-process
descriptor
table
pipe
Inbound
traffic
socket
tty
“open file table”
global
port
table
There’s no magic here: processes use read/write (and other syscalls) to
operate on sockets, just like any Unix I/O object (“file”). A socket can
even be mapped onto stdin or stdout.
Deeper in the kernel, sockets are handled differently from files, pipes, etc.
Sockets are the entry/exit point for the network protocol stack.
Ports
• Each IP transport endpoint on a host has a logical port
number (16-bit integer) that is unique on that host.
• This port abstraction is an Internet Protocol concept.
– Source/dest port is named in every IP packet.
– Kernel looks at port to demultiplex incoming traffic.
• What port number to connect to?
– We have to agree on well-known ports for common services
– Look at /etc/services
– Ports 1023 and below are ‘reserved’ and privileged: generally
you must be root/admin/superuser to bind to them.
• Clients need a return port, but it can be an ephemeral
port assigned dynamically by the kernel.
Ports and packet demultiplexing
The IP network carries data packets addressed to a destination
node (host named by IP address) and port. Kernel network stack
demultiplexes incoming network traffic: choose process/socket to
receive it based on destination port.
Incoming network packets
Network adapter hardware
aka, network interface
controller (“NIC”)
Apps with
open
sockets
Wakeup from interrupt handler
return to user mode
trap or fault
sleep
queue
sleep
wakeup
ready
queue
switch
interrupt
Example 1: NIC interrupt wakes thread to receive incoming packets.
Example 2: disk interrupt wakes thread when disk I/O completes.
Example 3: clock interrupt wakes thread after N ms have elapsed.
Note: it isn’t actually the interrupt itself that wakes the thread, but the interrupt
handler (software). The awakened thread must have registered for the wakeup
before sleeping (e.g., by placing its TCB on some sleep queue for the event).
The network stack, simplified
Internet client host
Internet server host
Client
User code
Server
TCP/IP
Kernel code
TCP/IP
Sockets interface
(system calls)
Hardware interface
(interrupts)
Network
adapter
Hardware
and firmware
Network
adapter
Global IP Internet
Note: the “protocol stack” should not be confused with a thread stack. It’s
a layering of software modules that implement network protocols:
standard formats and rules for communicating with peers over a network.
The Internet concept wasn’t
obviously compelling, at least
not to everyone.
It had to be marketed, even
within the tech community.
Insert “Power of TCP/IP”
slide, /usr/net/87.
(The poster in my office)
In 1986, the US National
Science Foundation (NSF)
opened the door to a
commercial Internet (then
NSFNET). IP support in
sockets (Berkeley Unix) was
widely used among academics.
The driving force for adopting
TCP/IP was a collection of Unixoriented startups and upstarts
arrayed against a few large
companies with their own
proprietary network standards.
Stream sockets with
Transmission Control Protocol (TCP)
TCP user
(application)
user transmit buffers
COMPLETE
TCP send buffers (optional)
SEND
user receive buffers
COMPLETE
TCP rcv buffers (optional)
TCP
implementation
transmit
queue
get
receive
queue
data
data
checksum
ack
outbound
packets
window
flow
flow
TCP/IP protocol sender
RECEIVE
TCB
ack
inbound
packets
TCP/IP protocol receiver
checksum
network path
Integrity: packets are covered by a checksum to detect errors.
Reliability: receiver acks received data, sender retransmits if needed.
Ordering: packets/bytes have sequence numbers, and receiver reassembles.
Flow control: receiver tells sender how much / how fast to send (window).
Congestion control: sender “guesses” current network capacity on path.
Illustration only
Who governs the Internet?
IANA: a department of ICANN.
ICANN: a US nonprofit organization
that is responsible for the coordination
of…unique identifiers related to the
namespaces of the Internet, and
ensuring the network's stable and
secure operation.
I/O syscalls: quick primer 1
read(fd, buf, len): read len bytes from the I/O object named by the descriptor fd
and store it in the process VAS at address buf.
Block until the data is available:
•
If fd is a file: wait for disk I/O to complete; e.g., the calling thread blocks
awaiting a notify triggered by the disk interrupt handler.
•
If fd is a socket: block until the data arrives from the network peer; e.g., the
calling thread blocks awaiting a notify triggered by the NIC interrupt handler
after the packet arrives. (The recv syscall is equivalent.)
•
If fd is a pipe: block until a writer writes the data into the write end of the
pipe; e.g., the calling thread blocks awaiting a notify triggered by the write
(like soda machine).
If the object named by fd will never produce the data, then read returns an
EOF: end of file. A read returns the number of bytes transferred: zero  EOF.
•
If fd is a file, and all bytes of the file have already been read through fd.
•
If fd is a stream socket or a pipe, and the writer(s) closed the other end.
I/O syscalls: quick primer 2
write(fd, buf, len): write len bytes to the I/O object named by the descriptor fd,
fetching them from the process VAS at address buf.
Generally, the write call is asynchronous / nonblocking: it returns
immediately, and completes when the data arrives at the destination some time
later. However, uncompleted writes require buffering in the system, and write
blocks when some bound on the buffer memory is reached:
•
If fd is a file: block if there are “too many” pending writes in progress; e.g.,
the calling thread blocks until “enough” of those writes reach the disk.
•
If fd is a stream socket: block if there are “too many” packets in transit to
the receiver; e.g., the calling thread blocks until the receiving process
consumes “enough” of that data. (The send syscall is equivalent.)
•
If fd is a pipe: block if the kernel’s bounded buffer for the pipe is exhausted;
e.g., the calling thread blocks until a reader consumes “enough” bytes,
freeing up sufficient kernel buffer space for the write to complete.
If bytes written to the object named by fd will never be consumed, then write
returns an error. E.g., this may occur if the receiver(s) closed its end of the
pipe or socket, or the network is unreachable.
Servers in “classic” Unix
• Single-threaded processes
• Blocking system calls
– Synchronous I/O: calling process blocks until is “complete”.
• Each blocking call waits for only a single kind of a event
on a single object.
– Process or file descriptor (e.g., file or socket)
• Add signals when that model does not work.
– Oops, that didn’t really help.
• With sockets: add select system call to monitor I/O on
sets of sockets or other file descriptors.
– select was slow for large poll sets. Now we have various
variants: poll, epoll, pollet, kqueue. None are ideal.
Services
Network services run as
processes that listen for requests
arriving on ports.
Before
Each network service receives
requests at a designated port
number (assigned and
standardized by IANA).
See /etc/services
Inetd forks service processes on
demand (lazily) on incoming
connect requests for their ports.
After
What if no process is listening on the
destination port for a request?
inetd
• Classic Unix systems run an
inetd “internet daemon”.
• Inetd receives requests for
standard services.
– Standard services and ports
listed in /etc/services.
– inetd listens on all the ports
and accepts connections.
• For each connection, inetd
forks a child process.
• Child execs the service
configured for the port.
• Child executes the request,
then exits.
[Apache Modeling Project: http://www.fmc-modeling.org/projects/apache]
Request/reply messaging
client
server
Remote Procedure
Call (RPC) is one
common example of
this pattern.
request
compute
reply
The Web is another.
P2: break a simple web server
• The web server is based on:
– */c-samples/buggyserver.c
• This server has a bug that makes it vulnerable to a stack
smash attack (previously discussed).
• Stack smash attacks may enable remote execution of
code chosen by the attacker, to “own” the web server.
• Each group gets their own instance to attack. If you
crack it you get the points.
• Test your talents, but please do not abuse them.
• These attacks have unleashed untold pain into the
world…and it never stops.
Stack smash defenses
• Modern systems have various defenses.
– NX: no-execute segments. The classic attack injects code
onto a buffer that resides on the stack, and overwrites a return
address to branch to the injected code. NX makes this harder by
disabling execute privilege on the stack segment. Any attempt
to execute the attacker’s code (or any code) on the stack
generates a fault.
– ASLR: address space layout randomization. The attacker
guesses where the stack resides in order to overwrite a frame’s
return address to branch to injected code. Randomizing the
layout makes this harder.
• These are disabled in the web server instances or p2.
Server listens on a socket
struct sockaddr_in socket_addr;
sock = socket(PF_INET, SOCK_STREAM, 0);
int on = 1;
setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &on, sizeof on);
memset(&socket_addr, 0, sizeof socket_addr);
socket_addr.sin_family = PF_INET;
socket_addr.sin_port = htons(port);
socket_addr.sin_addr.s_addr = htonl(INADDR_ANY);
if (bind(sock, (struct sockaddr *)&socket_addr, sizeof socket_addr) < 0) {
perror("couldn't bind");
exit(1);
}
listen(sock, 10);
Illustration only
Accept loop: trivial example
while (1) {
int acceptsock = accept(sock, NULL, NULL);
char *input = (char *)malloc(1024*sizeof (char));
recv(acceptsock, input, 1024, 0);
int is_html = 0;
char *contents = handle(input,&is_html);
free(input);
…send response…
close(acceptsock);
}
If a server is listening on only one
port/socket (“listener”), then it can
skip the select/poll/epoll.
Illustration only
Send HTTP/HTML response
const char *resp_ok = "HTTP/1.1 200 OK\nServer: BuggyServer/1.0\n";
const char *content_html = "Content-type: text/html\n\n";
send(acceptsock, resp_ok, strlen(resp_ok), 0);
send(acceptsock, content_html, strlen(content_html), 0);
send(acceptsock, contents, strlen(contents), 0);
send(acceptsock, "\n", 1, 0);
free(contents);
Illustration only
Servers and concurrency
• Servers receive requests from many different clients.
– Many clients send requests “at the same time”.
• Servers should handle those requests concurrently.
– Don’t leave a server CPU idle if there is a request to work on.
• But how to do that with the classic Unix process model?
– Unix had single-threaded processes and blocking syscalls.
– If a process blocks it can’t do anything else until it wakes up.
• Systems with GUIs also face them.
• What to do?
Event-driven programming
• Event-driven programming is a design
pattern for a thread’s program.
• The thread receives and handles a
sequence of typed messages or events.
– Handle one event at a time, in order.
• In its pure form the thread never blocks,
except to get the next event.
events
– Blocks only if no events to handle (idle).
• We can think of the program as a set of
handler routines for the event types.
– The thread upcalls the handler to
dispatch or “handle” each event.
• A handler should not block: if it does, the
thread becomes unresponsive to events.
Dispatch events by invoking
handlers (upcalls).
Handling a Web request
Accept Client
Connection
may block
waiting on
network
Read HTTP
Request Header
Find
File
may block
waiting on
disk I/O
Send HTTP
Response Header
Read File
Send Data
We want to be able to process requests concurrently.
Web server (serial process)
 Option 1: could handle requests serially
Client 1
WS
Client 2
R1 arrives
Receive R1
Disk request 1a
R2 arrives
1a completes
R1 completes
Receive R2
 Easy to program, but painfully slow (why?)
Web server (“pure” event-driven)
 Option 2: use asynchronous I/O
 Fast, but hard to program (why?)
Client 2 Client 1
WS
Disk
R1 arrives
Receive R1
Disk request 1a
R2 arrives
Receive R2
1a completes
R1 completes
Start 1a
Finish 1a
Web server (multiprogrammed)
 Option 3: assign one thread per request
Client 1
WS1
WS2
Client 2
R1 arrives
Receive R1
Disk request 1a
R2 arrives
Receive R2
1a completes
R1 completes
 Where is each request’s state stored?
Events vs. threading
• Classic Unix system call APIs are blocking. Requires multiple
processes/threads to build responsive/efficient systems.
• Even so, kernel networking and I/O stacks are mostly event-driven
(interrupts, callbacks, event queues, etc.). Example: Windows I/O
driver stack is a highly flexible event-driven system.
• Some system call APIs may be non-blocking, i.e., asynchronous
I/O. E.g., polling APIs like waitpid() with WNOHANG. This is an
event-driven model: notify thread by an event when op completes.
• Modern systems combine events and threading
– Event-driven model is natural for GUIs, servers.
– But to use multiple cores effectively, we need multiple threads.
And every system today is a multicore system.
– Multi-threading also enables use of blocking APIs without
compromising responsiveness of other threads in the program.
Multi-process server architecture
Process 1
Accept
Conn
Read
Request
Find
File
Send
Header
Read File
Send Data
…
separate address spaces
Process N
Accept
Conn
Read
Request
Find
File
Send
Header
Read File
Send Data
Multi-threaded server architecture
Thread 1
Accept
Conn
Read
Request
Find
File
Read File
Send Data
Send
Header
Read File
Send Data
…
Send
Header
Thread N
Accept
Conn
Read
Request
Find
File
This structure might have lower cost than the multi-process architecture
if threads are “cheaper” than processes.
Thread pool: idealized
Magic elastic worker pool
Resize worker pool to match
incoming request load:
create/destroy workers as
needed.
idle workers
Workers wait here for next
request dispatch.
Workers could be
processes or threads.
worker
loop
dispatch
Incoming
request
queue
Handle one
request,
blocking as
necessary.
When request
is complete,
return to
worker pool.
Ideal event poll API for thread pooling
Poll()
1. Delivers: returns exactly one event (message or
notification), in its entirety, ready for service (dispatch).
2. Idles: Blocks iff there is no event ready for dispatch.
3. Consumes: returns each posted event at most once.
4. Combines: any of many kinds of events (a poll set) may
be returned through a single call to poll.
5. Synchronizes: may be shared by multiple processes or
threads ( handlers must be thread-safe as well).
Event/request queue
We can synchronize an event
queue with a monitor: a
mutex/CV pair.
Protect the event queue data
structure itself with the mutex.
threads waiting on CV
Workers wait on the CV for
next event if the event queue
is empty. Signal the CV when
a new event arrives. This is a
producer/consumer
problem.
worker
loop
handler
dispatch
Incoming
event
queue
handler
handler
Handle one
event,
blocking as
necessary.
When handler
is complete,
return to
worker pool.
But what’s an “event”?
• A system can use an event-driven design pattern to
handle any kind of asynchronous event.
– Arriving input (e.g., GUI clicks/swipes, requests to a server)
– Notify that an operation started earlier is complete
• E.g., I/O completion
– Subscribe to events published/posted by other threads
– Including status of children: stop/exit/wait, signals, etc.
• You can use an “event” to represent any kind of message that
drives any kind of action in the receiving thread.
– In Android: intents, binder RPC, UI events
• But the system must be designed for it, so that operations the
thread requests do not block; the request returns immediately
(“asynchronous”) and delivers a completion event later.
Android: threading model
• An app is launched as a process when
any of its components is first instantiated.
• The process main thread is event-driven,
e.g., by User Interface (UI) events.
– Also called the “UI thread”.
– UI toolkit code is not thread-safe, so it
should execute only on the UI thread.
– UI thread should not block (except for next
event), or app becomes unresponsive.
• UI thread also receives incoming intents,
launches and tears down components,
and receives various upcalls.
• An app may spawn other background
threads (workers) for other uses.
• Binder RPC manages a thread pool.
events
Threads in Android
Three examples/models for use of threads in Android.
1. Main thread (UI thread): receives UI events and other upcall
events on a single incoming message queue. Illustrates eventdriven pattern: thread blocks only to wait for next event.
2. ThreadPool: an elastic pool of threads that handle incoming calls
from clients: Android supports “binder” request/response calls from
one application to another. When a request arrives, a thread from
the pool receives it, handles it, responds to it, and returns to the
pool to wait for the next request.
3. AsyncTask: the main thread can create an AsyncTask thread to
perform some long-running activity without blocking the UI thread.
The AsyncTask thread sends progress updates to the main thread
on its message queue.
These patterns are common in many other systems as well.
Adapted from
http://developer.android.com/guide/components/processes-and-threads.html
Summary: By default, all components of the same application run in the same process and
thread (called the "main" thread). The main thread controls the UI, so it is also called the
UI thread. If the UI thread blocks then the application stops responding to the user. You
can create additional background threads for operations that block, e.g., I/O, to avoid doing
those operations on the UI thread. The background threads can interact with the UI by
posting messages/tasks/events to the UI thread.
Details: When an application component starts and the application does not have any
other components running, the Android system starts a new Linux process for the
application with a single thread of execution called the main thread.
All components that run in the same process are initialized by its main thread, and system
upcalls to those components (onCreate, onBind, onStart,…) run on the main thread.
The main thread is also called the UI thread. It is in charge of dispatching events to user
interface widgets and interacting with elements of the Android UI toolkit. For instance,
when the user touches a button on the screen, your app's UI thread dispatches the touch
event to the widget, to set its pressed state and redraw itself.
If you have operations that might require blocking, e.g., to perform I/O like network
communication or database access, you should run them in separate threads. A thread
that is not the UI thread is called a background thread or "worker" thread.
Adapted from
http://developer.android.com/guide/components/processes-and-threads.html
Your app should never block the UI thread. When the UI thread is blocked, no events can
be dispatched, including drawing events. From the user's perspective, the application
appears to “hang” or “freeze”.
Even worse, if the app blocks the UI thread for more than a few seconds, Android presents
the user with the infamous "application not responding" (ANR) dialog. The user might
then decide to quit your application and uninstall it.
In a correct Android program the UI thread blocks only to wait for the next event, when it
has nothing else to do (it is idle). If you have an operation to perform that might block for
any other reason, then you should arrange for a background/worker thread to do it.
Additionally, the Android UI toolkit is not thread-safe: if multiple threads call a module
that is not thread-safe, then the process might crash. A correct app manipulates the user
interface only from a single thread, the UI thread. So: your app must not call UI widgets
from a worker thread.
So how can a worker thread interact with the UI, e.g., to post status updates?
offers several ways for a worker to post operations to run on the UI thread.
Android
Note: this concept of a single event-driven main/UI thread appears in other systems too.
Android: AsyncTask
[http://techtej.blogspot.com/2011/03/android-thread-constructs-part-3.html]
Summary/recap/segue
• Servers need concurrency. The standard model today
is multi-threaded servers with a thread pool. We can
implement it with our thread primitives.
• Clients need concurrency. It isn’t 1974 anymore, and
byte streams just aren’t enough. Modern apps have
GUIs and likely interact with one or more servers. And
they need to be responsive.
• Android apps “are servers”. They accept
upcalls/events from the GUI, from the OS, and from
other apps. So their threading models are similar to
servers.
• Android uses RPC to communicate across app
boundaries. RPC is an important structuring element in
modern distributed systems.
Remote Procedure Call (RPC)
• “RPC is a canonical structuring paradigm for client/server
request/response services.”
• Used in .NET, Android, RMI, distributed component frameworks
• First saw wide use in 1980s client/server systems for workstation
networks (e.g., Network File System).
client
server
Auto-generate this “stub”
code from API spec (IDL).
“glue”
[sockets]
Humans focus on
getting this code right.
[sockets]
This code is “canned”,
independent of the
specific application.
RPC
Remote Procedure Call (RPC) is request/response interaction
through a published API, using IPC messaging to cross an interprocess boundary.
API stubs generated from an Interface
Description Language (IDL)
Establishing an RPC connection
to a named remote interface is
often called binding.
RPC is used in many standard Internet services. It is also the basis for
component frameworks like DCOM, CORBA, and Android. Software is
packaged into named “objects” or components. Components may publish
interfaces and/or invoke published interfaces of other components.
Components may execute in different processes and/or on different nodes.
Threads and RPC
Q: How do we manage these
“call threads”?
A: Create them as needed,
and keep idle threads in a
thread pool.
When an RPC call arrives,
wake up an idle thread from
the pool to handle it.
On the client, the client thread
blocks until the server thread
returns a response.
[OpenGroup, late 1980s]
RPC: a classic picture
Implementing RPC
Birrell/Nelson 1984
RPC: Language integration
Stubs link with the client/server code to “hide” the boundary crossing.
– They “marshal” args/results
– i.e., translate to/from some standard
network stream format
– Also known as linearize, serialize
– …or “flatten”
– Propagate PL-level exceptions
– Stubs are auto-generated from an
Interface Description Language
(IDL) file by a stub compiler tool at
software build time, and linked in.
– Client and server must agree on the
protocol signatures in the IDL file.
Stubs
• RPC stubs are procedures linked into the client and server.
– RPC stubs are similar to system call stubs, but they do more than just
trap to the kernel.
– The RPC stubs construct/deconstruct a message transmitted through a
messaging system.
– Binder is an example of such a messaging system, implemented as a
Linux kernel plug-in module (a driver) and some user-space libraries.
• The stubs are generated by a tool that takes a description of the
application’s RPC API written in an Interface Description
Language.
– Looks like any interface definition…
– List of method names and argument/result types and signatures.
– Stub code marshals arguments into request message, marshals results
into a reply message.
Marshaling: a metaphor
Android Architecture and Binder
Dhinakaran Pandiyan
Saketh Paranjape
RPC Failure
Do RPC calls ever fail?
Yes. This is a key difference between RPC calls and local procedure calls. Complete
network transparency is neither achievable no desirable. If the client cannot reach the
server, you may want it to fail, so the app can decide if/how to report the problem to the
user, as opposed to (say) hanging forever. Generally, the problem of stalled network
communication is a major reason why thread systems provide thread alert APIs and
timeouts on CV wait.
RPC systems built over sockets (such as Java RMI) can be expected to reflect any socketlevel errors back to the process. For example, if a server process has failed (its port is no
longer bound), but its host is still up, then its host kernel rejects an incoming TCP
connection request with "connection refused”. The RPC client should report this and other
errors back to the app. For example, a socket connect request also fails if the network is
unreachable. Similarly, an established connection may timeout after a time if one end fails:
the RPC system may retry or it may report the error.
Timeouts are a configuration issue. Defaults should be reasonable, but they vary.
Binder: object-based RPC channels
Activity
Manager
Service
etc.
Services
register to
advertise for
clients.
JVM+lib
Bindings are
reference-counted.
A client binds
to a service.
JVM+lib
Android binder
an add-on kernel driver for
/dev/binder object RPC
Linux kernel
Android services and libraries communicate by sending messages
through shared-memory channels set up by binder.
Extra slides and resources
• Some useful reading on sockets and socket
programming, covering most of the ground in this deck:
– http://www.cs.dartmouth.edu/~campbell/cs50/socketprogramming.html
• For event-based concurrency, see OSTEP #33.
– http://pages.cs.wisc.edu/~remzi/OSTEP/threads-events.pdf
• For RPC, I recommend ch2 of Andy Tanenbaum’s text,
Distributed Systems: Principles and Paradigms.
– http://www.cs.vu.nl/~ast/books/ds1/02.pdf
Multi-process server architecture
• Each of P processes can execute one request at a time,
concurrently with other processes.
• If a process blocks, the other processes may still make
progress on other requests.
• Max # requests in service concurrently == P
• The processes may loop and handle multiple requests
serially, or can fork a process per request.
– Tradeoffs?
• Examples:
– inetd “internet daemon” for standard /etc/services
– Design pattern for (Web) servers: “prefork” a fixed number of
worker processes.
High-throughput servers
• Various server systems use various combinations
models for concurrency.
• Unix made some choices, and then more choices.
• These choices failed for networked servers, which
require effective concurrent handling of requests.
• They failed because they violate properties for “ideal”
event handling.
• There is a large body of work addressing the resulting
problems. Servers mostly work now. We skip over the
noise.
Concurrency/Asynchrony in Unix
Some partial answers and options
Shells face similar problems in tracking their children,
which execute independently (asynchronously).
1. Nonblocking (asynchronous) syscalls.
– Example: wait*(WNOHANG). But you have to keep asking to
know when a child changes state. (polling)
– What about starting asynchronous operations, like a read? How
to know when it is done without blocking?
2. Asynchronous notification messages (events).
– Signals? E.g., SIGCHLD: “Your child has died.”
– Interrupted syscalls w/ EINTR: “Look: something happened.”
3. Threads etc. (or multiple processes with
messaging)…but how to structure their interactions?