Outline
• Distributed shared memory – continued
Distributed Shared Memory
• Distributed computing is mainly based on the
message passing model
– Client/server model
– Remote procedure calls
• Distributed shared memory
– is a resource management component that
implements the shared memory model in
distributed systems, where there is no physically
shared memory
May 29, 2016
COP 5611 - Operating Systems
2
Memory Hierarchies
May 29, 2016
COP 5611 - Operating Systems
3
Virtual memory
May 29, 2016
COP 5611 - Operating Systems
4
Page Table Mapping
May 29, 2016
COP 5611 - Operating Systems
5
Distributed Shared Memory – cont.
• This is a further extension of the virtual
memory management on a single computer
– When a process accesses data in the shared
address space, a mapping manager maps the
shared memory address to the physical memory,
which can be local or remote
May 29, 2016
COP 5611 - Operating Systems
6
Distributed Shared Memory – cont.
May 29, 2016
COP 5611 - Operating Systems
7
Distributed Shared Memory – cont.
A mapping manager is a layer of software to map addresses in user
space into the physical memory addresses
May 29, 2016
COP 5611 - Operating Systems
8
Distributed Shared Memory – cont.
• With DSM, application programs can access
data in the shared space as they access data in
traditional virtual memory
– Mapping manager can move data among the
local main memory, local disk, and another nodes
May 29, 2016
COP 5611 - Operating Systems
9
Distributed Shared Memory – cont.
• Advantages of DSM
– It is easier to design and develop algorithms with
DSM than message passing models
– DSM allows complex structures to be passed by
reference, simplifying the distributed application
development
– DSM can cut down the overhead of communication
by exploiting locality in programs
– DSM can overcome some the architectural
limitations of shared memory machines
May 29, 2016
COP 5611 - Operating Systems
10
Distributed Shared Memory – cont.
• Central implementation issues in DSM
– How to keep track of the location of remote data
– How to overcome the communication delays and
high overhead associated with communication
protocols
– How to improve the system performance
May 29, 2016
COP 5611 - Operating Systems
11
The Central-Server Algorithm
• A central server maintains all the shared data
– It serves the read requests from other nodes or
clients by returning the data items to them
– It updates the data on write requests by clients
May 29, 2016
COP 5611 - Operating Systems
12
The Central-Server Algorithm – cont.
May 29, 2016
COP 5611 - Operating Systems
13
The Migration Algorithm
• In contrast to the central-server algorithm,
data in the migration algorithm is shipped to
the location of data access request
– Subsequent accesses can then be performed
locally
• Thrashing can be a problem
May 29, 2016
COP 5611 - Operating Systems
14
The Migration Algorithm – cont.
May 29, 2016
COP 5611 - Operating Systems
15
The Migration Algorithm – cont.
May 29, 2016
COP 5611 - Operating Systems
16
The Read-Replication Algorithm
• The read-replication algorithm allows
multiple node to have read access or one
node to have read-write access
May 29, 2016
COP 5611 - Operating Systems
17
The Read-Replication Algorithm
May 29, 2016
COP 5611 - Operating Systems
– cont.
18
The Read-Replication Algorithm
May 29, 2016
COP 5611 - Operating Systems
– cont.
19
The Full-Replication Algorithm
• This is a further extension of the readreplication algorithm
– It allows multiple nodes to have both read and
write access to shared data blocks
• Data consistency issue
May 29, 2016
COP 5611 - Operating Systems
20
The Full-Replication Algorithm
May 29, 2016
COP 5611 - Operating Systems
– cont.
21
Memory Coherence
• Memory consistency models
– Many consistency models have been proposed
– These models differ in
•
•
•
•
Restrictiveness
Implementation complexity
Ease of programming
Performance
May 29, 2016
COP 5611 - Operating Systems
22
Memory Coherence – cont.
• Strict consistency
– Any read on a data item x returns a value
corresponding to the result of the most recent
write on x
• Sequential consistency
– The result of any execution is the same as if the
operations by all processes on the data store were
executed in some sequential order and the
operations of each individual process appear in
this sequence in the order specified by its
program
May 29, 2016
COP 5611 - Operating Systems
23
Memory Coherence – cont.
May 29, 2016
COP 5611 - Operating Systems
24
Memory Coherence – cont.
Process P1
Process P2
Process P3
x = 1;
print ( y, z);
y = 1;
print (x, z);
z = 1;
print (x, y);
May 29, 2016
COP 5611 - Operating Systems
25
Memory Coherence – cont.
x = 1;
print ((y, z);
y = 1;
print (x, z);
z = 1;
print (x, y);
x = 1;
y = 1;
print (x,z);
print(y, z);
z = 1;
print (x, y);
y = 1;
z = 1;
print (x, y);
print (x, z);
x = 1;
print (y, z);
y = 1;
x = 1;
z = 1;
print (x, z);
print (y, z);
print (x, y);
Prints: 001011
Prints: 101011
Prints: 010111
Prints: 111111
(a)
May 29, 2016
(b)
(c)
COP 5611 - Operating Systems
(d)
26
Memory Coherence – cont.
• General consistency
– All the copies of a memory location eventually contain the
same data when all the writes issued by every process have
completed
• Causal consistency
– Writes that are causally related must be seen by all the
processes in the same order; concurrent writes may be seen in
a different order on different machines
– Processor consistency is related to causal consistency but with
one more relaxation
May 29, 2016
COP 5611 - Operating Systems
27
Memory Coherence – cont.
• Weak consistency
– Accesses to synchronization variables are
sequentially consistent
– Before a synchronization access, all previous
regular data accesses must be completed
– Before a regular data access, all previous
synchronization accesses must be completed
May 29, 2016
COP 5611 - Operating Systems
28
Memory Coherence – cont.
May 29, 2016
COP 5611 - Operating Systems
29
Memory Coherence – cont.
• Release consistency
– Synchronization operations are broken down into
acquire and release operations
– Before a read or write operation on shared data is
performed, all previous acquires done by the process
must have completed successfully
– Before a release is allowed, all previous reads and
write done by the process must have been completed
May 29, 2016
COP 5611 - Operating Systems
30
Memory Coherence – cont.
• Coherence protocols
– A protocol to keep replicas coherent
• Write-invalidate protocol
– A write to a shared data causes the invalidation of
all copies except the one where the write occurs
• Write-update protocol
– A write to a shared data causes all copies of that
data to be updated
May 29, 2016
COP 5611 - Operating Systems
31
Memory Coherence – cont.
May 29, 2016
COP 5611 - Operating Systems
32
Memory Coherence – cont.
• Type-specific memory coherence
– Exploiting application specific semantics
information
– The system uses different kinds of coherence
mechanisms for different kinds of classes
• Write-once objects
• Write-many objects
• Read-mostly objects
May 29, 2016
COP 5611 - Operating Systems
33
Design Issues
• Granularity
– The size of the shared memory unit
– Advantages of large sizes
• A page size that is a multiple of the size provided by
the underlying memory management system allows
for the integration of DSM and the memory
management system
• Better utilization of locality of reference
– Disadvantages
• The greater the chance for contention
• False sharing
May 29, 2016
COP 5611 - Operating Systems
34
False Sharing
May 29, 2016
COP 5611 - Operating Systems
35
Design Issues – cont.
• Page replacement
– Traditional methods such as LRU can not be used
directly
– Page access modes must be taken into
consideration
• For example, private pages may be replaced before
shared pages
May 29, 2016
COP 5611 - Operating Systems
36
Case Studies
• IVY
– Integrated Shared Virtual Memory at Yale
– The granularity is a page
– The address space is divided into shared virtual
memory address space and private space
– It supports strict consistency using writeinvalidation protocol
May 29, 2016
COP 5611 - Operating Systems
37
IVY – cont.
• The coherence protocol
– Write fault handling
• The owner is identified first
• Then the owner sends the page and its copy-set to the
requested processor
• The faulting processor sends out invalidation messages to
all the processors contained in the copy-set
– Read fault handing
• The owner is identified first
• The owner sends the page and adds the requested site in
the copy-set of the page
May 29, 2016
COP 5611 - Operating Systems
38
IVY – cont.
• Coherence protocol
– The centralized manager scheme
– Fixed distributed manager scheme
– Dynamic distributed manager scheme
May 29, 2016
COP 5611 - Operating Systems
39
IVY – cont.
May 29, 2016
COP 5611 - Operating Systems
40
IVY – cont.
• Double fault
– A double fault occurs if a page not available
locally is read and written successively
• The page is transferred first due to the read fault
• Then the same page is transferred again due to the
write fault
• Memory allocation
• Process synchronization
May 29, 2016
COP 5611 - Operating Systems
41
Case Studies – cont.
• Mirage
– Thrashing control
• Clouds
– The RA kernel
• Segmentation with paging (The size of a segment is a
multiple of the physical page size)
– Distributed shared memory controller
• Four modes: read-only, read-write, weak-read, and
none
May 29, 2016
COP 5611 - Operating Systems
42
Summary
• Distributed shared memory tries to provide an
easy-to-use interface for distributed applications
– Distributed programs access data in the shared
address space as they access data in local memory
• However, the performance is a critical issue
– Replication is used to reduce the high cost of
communications
• Memory coherence is however difficult and expensive to
achieve
May 29, 2016
COP 5611 - Operating Systems
43