Outline
• Announcement
• Distributed file systems – continued
• Distributed shared memory
Announcement
• Programming project #1 demonstration
– You can come on April 1 or April 3 from 9:30-11:00
in my office to do the demonstration
– Or you can set up an appointment with me
– You need to have the demonstration before April 17
– What you need to prepare
• I may ask you some questions based on your report and
your implementation
• I will ask you to demonstrate at least one configuration
with and without mutual exclusion
May 29, 2016
COP 5611 - Operating Systems
2
Distributed Shared Memory
• Distributed computing is mainly based on the
message passing model
– Client/server model
– Remote procedure calls
• Distributed shared memory
– is a resource management component that
implements the shared memory model in
distributed systems, where there is no physically
shared memory
May 29, 2016
COP 5611 - Operating Systems
3
Memory Hierarchies
May 29, 2016
COP 5611 - Operating Systems
4
Virtual memory
May 29, 2016
COP 5611 - Operating Systems
5
Page Table Mapping
May 29, 2016
COP 5611 - Operating Systems
6
Distributed Shared Memory – cont.
• This is a further extension of the virtual
memory management on a single computer
– When a process accesses data in the shared
address space, a mapping manager maps the
shared memory address to the physical memory,
which can be local or remote
May 29, 2016
COP 5611 - Operating Systems
7
Distributed Shared Memory – cont.
May 29, 2016
COP 5611 - Operating Systems
8
Distributed Shared Memory – cont.
A mapping manager is a layer of software to map addresses in user
space into the physical memory addresses
May 29, 2016
COP 5611 - Operating Systems
9
Distributed Shared Memory – cont.
• With DSM, application programs can access
data in the shared space as they access data in
traditional virtual memory
– Mapping manager can move data among the
local main memory, local disk, and another nodes
May 29, 2016
COP 5611 - Operating Systems
10
Distributed Shared Memory – cont.
• Advantages of DSM
– It is easier to design and develop algorithms with
DSM than message passing models
– DSM allows complex structures to be passed by
reference, simplifying the distributed application
development
– DSM can cut down the overhead of communication
by exploiting locality in programs
– DSM can overcome some the architectural
limitations of shared memory machines
May 29, 2016
COP 5611 - Operating Systems
11
Distributed Shared Memory – cont.
• Central implementation issues in DSM
– How to keep track of the location of remote data
– How to overcome the communication delays and
high overhead associated with communication
protocols
– How to improve the system performance
May 29, 2016
COP 5611 - Operating Systems
12
The Central-Server Algorithm
• A central server maintains all the shared data
– It serves the read requests from other nodes or
clients by returning the data items to them
– It updates the data on write requests by clients
May 29, 2016
COP 5611 - Operating Systems
13
The Central-Server Algorithm – cont.
May 29, 2016
COP 5611 - Operating Systems
14
The Migration Algorithm
• In contrast to the central-server algorithm,
data in the migration algorithm is shipped to
the location of data access request
– Subsequent accesses can then be performed
locally
• Thrashing can be a problem
May 29, 2016
COP 5611 - Operating Systems
15
The Migration Algorithm – cont.
May 29, 2016
COP 5611 - Operating Systems
16
The Migration Algorithm – cont.
May 29, 2016
COP 5611 - Operating Systems
17
The Read-Replication Algorithm
• The read-replication algorithm allows
multiple node to have read access or one
node to have read-write access
May 29, 2016
COP 5611 - Operating Systems
18
The Read-Replication Algorithm
May 29, 2016
COP 5611 - Operating Systems
– cont.
19
The Read-Replication Algorithm
May 29, 2016
COP 5611 - Operating Systems
– cont.
20
The Full-Replication Algorithm
• This is a further extension of the readreplication algorithm
– It allows multiple nodes to have both read and
write access to shared data blocks
• Data consistency issue
May 29, 2016
COP 5611 - Operating Systems
21
The Full-Replication Algorithm
May 29, 2016
COP 5611 - Operating Systems
– cont.
22
Memory Coherence
• Memory consistency models
– Many consistency models have been proposed
– These models differ in
•
•
•
•
Restrictiveness
Implementation complexity
Ease of programming
Performance
May 29, 2016
COP 5611 - Operating Systems
23
Memory Coherence – cont.
• Strict consistency
– Any read on a data item x returns a value
corresponding to the result of the most recent
write on x
• Sequential consistency
– The result of any execution is the same as if the
operations by all processes on the data store were
executed in some sequential order and the
operations of each individual process appear in
this sequence in the order specified by its
program
May 29, 2016
COP 5611 - Operating Systems
24
Memory Coherence – cont.
May 29, 2016
COP 5611 - Operating Systems
25
Memory Coherence – cont.
Process P1
Process P2
Process P3
x = 1;
print ( y, z);
y = 1;
print (x, z);
z = 1;
print (x, y);
May 29, 2016
COP 5611 - Operating Systems
26
Memory Coherence – cont.
x = 1;
print ((y, z);
y = 1;
print (x, z);
z = 1;
print (x, y);
x = 1;
y = 1;
print (x,z);
print(y, z);
z = 1;
print (x, y);
y = 1;
z = 1;
print (x, y);
print (x, z);
x = 1;
print (y, z);
y = 1;
x = 1;
z = 1;
print (x, z);
print (y, z);
print (x, y);
Prints: 001011
Prints: 101011
Prints: 010111
Prints: 111111
(a)
May 29, 2016
(b)
(c)
COP 5611 - Operating Systems
(d)
27
Memory Coherence – cont.
• General consistency
– All the copies of a memory location eventually contain the
same data when all the writes issued by every process have
completed
• Causal consistency
– Writes that are causally related must be seen by all the
processes in the same order; concurrent writes may be seen in
a different order on different machines
– Processor consistency is related to causal consistency but with
one more relaxation
May 29, 2016
COP 5611 - Operating Systems
28
Memory Coherence – cont.
• Weak consistency
– Accesses to synchronization variables are
sequentially consistent
– Before a synchronization access, all previous
regular data accesses must be completed
– Before a regular data access, all previous
synchronization accesses must be completed
May 29, 2016
COP 5611 - Operating Systems
29
Memory Coherence – cont.
May 29, 2016
COP 5611 - Operating Systems
30
Memory Coherence – cont.
• Release consistency
– Synchronization operations are broken down into
acquire and release operations
– Before a read or write operation on shared data is
performed, all previous acquires done by the process
must have completed successfully
– Before a release is allowed, all previous reads and
write done by the process must have been completed
May 29, 2016
COP 5611 - Operating Systems
31
Memory Coherence – cont.
• Coherence protocols
– A protocol to keep replicas coherent
• Write-invalidate protocol
– A write to a shared data causes the invalidation of
all copies except the one where the write occurs
• Write-update protocol
– A write to a shared data causes all copies of that
data to be updated
May 29, 2016
COP 5611 - Operating Systems
32
Memory Coherence – cont.
May 29, 2016
COP 5611 - Operating Systems
33
Memory Coherence – cont.
• Type-specific memory coherence
– Exploiting application specific semantics
information
– The system uses different kinds of coherence
mechanisms for different kinds of classes
• Write-one objects
• Write-many objects
• Read-mostly objects
May 29, 2016
COP 5611 - Operating Systems
34
Design Issues
• Granularity
– The size of the shared memory unit
– Advantages of large sizes
• A page size that is a multiple of the size provided by
the underlying memory management system allows
for the integration of DSM and the memory
management system
• Better utilization of locality of reference
– Disadvantages
• The greater the chance for contention
• False sharing
May 29, 2016
COP 5611 - Operating Systems
35
False Sharing
May 29, 2016
COP 5611 - Operating Systems
36
Design Issues – cont.
• Page replacement
– Traditional methods such as LRU can not be used
directly
– Page access modes must be taken into
consideration
• For example, private pages may be replaced before
shared pages
May 29, 2016
COP 5611 - Operating Systems
37
Case Studies
• IVY
– Integrated Shared Virtual Memory at Yale
– The granularity is a page
– The address space is divided into shared virtual
memory address space and private space
• The coherence protocol
– Write fault handling
– Read fault handing
May 29, 2016
COP 5611 - Operating Systems
38
IVY – cont.
• Coherence protocol
– The centralized manager scheme
– Fixed distributed manager scheme
– Dynamic distributed manager scheme
May 29, 2016
COP 5611 - Operating Systems
39
IVY – cont.
May 29, 2016
COP 5611 - Operating Systems
40
IVY – cont.
• Double fault
• Memory allocation
• Process synchronization
May 29, 2016
COP 5611 - Operating Systems
41
Case Studies – cont.
• Mirage
– Thrashing control
• Clouds
– The RA kernel
– Distributed shared memory controller
May 29, 2016
COP 5611 - Operating Systems
42
Summary
• Distributed shared memory tries to provide an
easy-to-use interface for distributed applications
– Distributed programs access data in the shared
address space as they access data in local memory
• However, the performance is a critical issue
– Replication is used to reduce the high cost of
communications
• Memory coherence is however difficult and expensive to
achieve
May 29, 2016
COP 5611 - Operating Systems
43