Outline
• Communication Primitives - continued
– Message passing model
– Remote procedure calls
• Theoretical Foundations
5/29/2016
COP5611
1
Communication Primitives
• Communication primitives are the high-level
constructs
– Programs use the underlying network by calling
these primitives
– Communication primitives play a significant role
in the effective usage of distributed systems
5/29/2016
COP5611
2
The Message Passing Model
• The message passing model provides two
basic communication primitives
– Send and receive
– Send has two logical parameters, a message and
its destination
– Receive has two logical parameters, the source
and a buffer for storing the message
5/29/2016
COP5611
3
Semantics of Send and Receive Primitives
• There are several design issues regarding SEND
and RECEIVE primitives
– Blocking vs. non-blocking primitives
• With blocking primitives, the SEND does not return control
until the message has been sent or received and the RECEIVE
does not return control until a message is copied to the buffer
• With non-blocking primitives, the SEND returns control as
the message is copied and the RECEIVE signals its intention
to receive a message and provide a buffer for it
5/29/2016
COP5611
4
Semantics of Send and Receive Primitives – cont.
• Synchronous vs. asynchronous primitives
– With synchronous primitives, a SEND primitive
is blocked until a corresponding RECEIVE
primitive is executed
– With asynchronous primitives, a SEND primitive
does not block even if there is no corresponding
execution of a RECEIVE primitive
• Buffered or un-buffered
– This is largely determined by the other two
choices
5/29/2016
COP5611
5
General Organization
5/29/2016
COP5611
6
Asynchronous SEND
• Persistent communication of letters back in the days
of the Pony Express.
5/29/2016
COP5611
7
Semantics of Send and Receive Primitives – cont.
5/29/2016
COP5611
8
Semantics of Send and Receive Primitives – cont.
5/29/2016
COP5611
9
Semantics of Send and Receive Primitives – cont.
5/29/2016
COP5611
10
Problems with Message Passing Model
• While it is highly flexible, programmers must
handle the details using such a model
– Pairing of responses with request messages
– Data representation
– Naming (the address of the remote machine or the
server)
– Taking care of communication and system failures
– The programs can be time-dependent, making it
impossible to reproduce errors and debug
5/29/2016
COP5611
11
Remote Procedure Call
• RPC is designed to hide all the details from
programmers
– Overcome the difficulties with message-passing
model
• It extends the conventional local procedure
calls to calling procedures on remote computers
5/29/2016
COP5611
12
Conventional Procedure Call
a)
b)
Parameter passing in a local procedure call: the stack before the call to read
The stack while the called procedure is active
5/29/2016
COP5611
13
Client and Server Stubs
• Principle of RPC between a client and server program.
5/29/2016
COP5611
14
Steps of a Remote Procedure Call
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Client procedure calls client stub in normal way
Client stub builds message, calls local OS
Client's OS sends message to remote OS
Remote OS gives message to server stub
Server stub unpacks parameters, calls server
Server does work, returns result to the stub
Server stub packs it in message, calls local OS
Server's OS sends message to client's OS
Client's OS gives message to client stub
Stub unpacks result, returns to client
5/29/2016
COP5611
15
Steps of a Remote Procedure Call – cont.
5/29/2016
COP5611
16
Remote Procedure Call – cont.
• Design issues
– Structure
• Mostly based on stub procedures
– Binding
• Through a binding server
• The client specifies the machine and service required
– Parameter and result passing
• Representation issues
• By value and by reference
5/29/2016
COP5611
17
Passing Value Parameters (1)
• Steps involved in doing remote computation through RPC
2-8
5/29/2016
COP5611
18
Passing Value Parameters (2)
a)
b)
c)
Original message on the Pentium
The message after receipt on the SPARC
The message after being inverted. The little numbers in
boxes indicate the address of each byte
5/29/2016
COP5611
19
Parameter Specification and Stub Generation
a)
b)
A procedure
The corresponding message.
5/29/2016
COP5611
20
Remote Procedure Call – cont.
• Design issues – continued
– Error handling, semantics, and correctness
•
•
•
•
“At least once” semantics
“Exactly once” semantics
“At most once” semantics
Correctness conditions
– Other issues
5/29/2016
COP5611
21
Error Handling
•
a)
b)
c)
A server in client-server communication
Normal case
Crash after execution
Crash before execution
5/29/2016
COP5611
22
Asynchronous RPC (1)
a)
b)
The interconnection between client and server in a traditional RPC
The interaction using asynchronous RPC
5/29/2016
COP5611
23
Asynchronous RPC (2)
• A client and server interacting through two
asynchronous RPCs
5/29/2016
COP5611
24
Example: DCE RPC
• Distributed Computing Environment (DCE)
– By Open Software Foundation (OSF, now called
Open Group)
– DCE is a true middleware system
• Designed as a layer of abstraction between existing
operating systems and distributed applications
• Provide a number of services
– Distributed file service, directory service, security service,
distributed time service
• Support UNIX, Windows NT
– A highly representative RPC system
5/29/2016
COP5611
25
Writing a Client and a Server
5/29/2016
COP5611
26
Binding a Client to a Server
• Client-to-server binding in DCE.
2-15
5/29/2016
COP5611
27
Remote Object Invocation
• Extend RPC principles to objects
– The key feature of an object is that it encapsulates
data (called state) and the operations on those data
(called methods)
– Methods are made available through an interface
– The separation between interfaces and the objects
implementing these interfaces allows us to place an
interface at one machine, while the object itself
resides on another machine
5/29/2016
COP5611
28
Distributed Objects
• Common organization of a remote object with
client-side proxy.
5/29/2016
COP5611
29
Binding a Client to an Object
Distr_object* obj_ref;
obj_ref = …;
obj_ref-> do_something();
//Declare a systemwide object reference
// Initialize the reference to a distributed object
// Implicitly bind and invoke a method
(a)
Distr_object obj_ref;
Local_object* obj_ptr;
obj_ref = …;
obj_ptr = bind(obj_ref);
obj_ptr -> do_something();
//Declare a systemwide object reference
//Declare a pointer to local objects
//Initialize the reference to a distributed object
//Explicitly bind and obtain a pointer to the local proxy
//Invoke a method on the local proxy
(b)
a)
b)
An example with implicit binding using only global
references
An example with explicit binding using global and local
references
5/29/2016
COP5611
30
Parameter Passing
• The situation when passing an object by reference or by
value.
2-18
5/29/2016
COP5611
31
Java RMI
• Java distributed-object model
– Aimed for high degree distribution transparency
but not entirely
• Java remote method invocation
– There are differences between local and remote
objects
• Local objects are passed by value while remote
objects are passed by reference
5/29/2016
COP5611
32
Distributed Systems
• A distributed system is a collection of
independent computers that appears to its
users as a single coherent system
– Independent computers mean that they do not
share memory or clock
– The computers communicate with each other by
exchanging messages over a communication
network
• The messages are delivered after an arbitrary
transmission delay
5/29/2016
COP5611
33
Inherent Limitations of a Distributed System
• Absence of a global clock
– In a centralized system, time is unambiguous
– In a distributed system, there exists no system
wide common clock
• In other words, the notion of global time does not
exist
– Impact of the absence of global time
• Difficult to reason about temporal order of events
• Makes it harder to collect up-to-date information on
the state of the entire system
5/29/2016
COP5611
34
Absence of Global Time
• When each machine has its own clock, an event
that occurred after another event may
nevertheless be assigned an earlier time.
5/29/2016
COP5611
35
Inherent Limitations of a Distributed System
• Absence of shared memory
– An up-to-date state of the entire system is not
available to any individual process
• This information, however, is necessary to reason
about the system’s behavior, debugging, recovering
from failures
5/29/2016
COP5611
36
Absence of Shared Memory – cont.
5/29/2016
COP5611
37
Physical Clocks
5/29/2016
COP5611
38
Physical Clocks – cont.
• TAI seconds are of constant length, unlike solar
seconds. Leap seconds are introduced when
necessary to keep in phase with the sun.
5/29/2016
COP5611
39
Clock Synchronization Algorithms
• The relation between clock time and UTC when clocks tick
at different rates.
5/29/2016
COP5611
40
Cristian's Algorithm
• Getting the current time from a time server.
5/29/2016
COP5611
41
The Berkeley Algorithm
a)
b)
c)
The time daemon asks all the other machines for their clock values
The machines answer
The time daemon tells everyone how to adjust their clock
5/29/2016
COP5611
42
Logical Clocks
• There are technical issues with the clock
synchronization approaches
– Due to unpredictable message transmission delays,
two processes can observe a global clock value at
different instants
– The physical clocks can drift from the physical time
and thus we cannot have a system of perfectly
synchronized clocks
• For many purposes, it is sufficient that all
machines agree on the same time
5/29/2016
COP5611
43
Lamport’s Logical Clocks
• Logical clocks
– For a wide of algorithms, what matters is the
internal consistency of clocks, not whether they
are close to the real time
– For these algorithms, the clocks are often called
logical locks
• Lamport proposed a scheme to order events
in a distributed system using logical clocks
5/29/2016
COP5611
44
Lamport’s Logical Clocks – cont.
• Definitions
– Happened before relation
• Happened before relation () captures the causal
dependencies between events
• It is defined as follows
– a  b, if a and b are events in the same process and a
occurred before b.
– a  b, if a is the event of sending a message m in a process
and b is the event of receipt of the same message m by
another process
– If a  b and b  c, then a  c, i.e., “” is transitive
5/29/2016
COP5611
45
Lamport’s Logical Clocks – cont.
• Definitions – continued
– Causally related events
• Event a causally affects event b if a  b
– Concurrent events
• Two distinct events a and b are said to be concurrent
(denoted by a || b) if a  b and b  a
• For any two events, either a  b, b  a, or a || b
5/29/2016
COP5611
46
Lamport’s Logical Clocks – cont.
5/29/2016
COP5611
47
Lamport’s Logical Clocks – cont.
• Logical clocks
– There is a clock at each process Pi in the system
• Which is a function that assigns a number to any event
a, called the timestamp of event a at Pi
• The numbers assigned by the system of the clocks
have no relation to physical time
• The logical clocks take monotonically increasing
values and can be implemented as counters
5/29/2016
COP5611
48
Lamport’s Logical Clocks – cont.
• Conditions satisfied by the system of clocks
– For any two events, if a  b, then C(a) < C(b)
– [C1] For any two events a and b in a process Pi,
if a occurs before b, then
Ci(a) < Ci(b)
– [C2] If a is the event of sending a message m in
process Pi and b is the event of receiving the
same message m at process Pj, then
Ci(a) < Cj(b)
5/29/2016
COP5611
49
Lamport’s Logical Clocks – cont.
• Implementation rules
– [IR1] Clock Ci is incremented between any two
successive events in process Pi
Ci := Ci + d ( d > 0)
– [IR2] If event a is the sending of message m by
process Pi, then message m is assigned a
timestamp tm = Ci(a). On receiving the same
message m by process Pj, Cj is set to
Cj := max(Cj, tm + d)
5/29/2016
COP5611
50
An Example
5/29/2016
COP5611
51