Part six:
Distributed systems
Natalia Shatokhina
Ashwini Mundya
Outline
• Reasons to build distributed system
• Network OS vs Distributed OS
• Data/process/communication migration in
distributed OS
• Communication Structure
• Communication Protocols
• Robustness
• Design Issues
• An Example: Networking
Distributed system
Site A
Site C
server
Network
Resources
client
Site B
Why distributed systems?
4 major reasons for building distributed
systems:
• Resource Sharing
• Computational speedup
• Reliability
• Communication
Reason 1: Resource sharing
• User on one site may be able to use the
resources available at another site
– Using a printer at remote site
– Access a file
– Access a database
Reason 2: Computational
Speedup
 If particular computation can be split into
subcomputations that can run in parallel, the
distributed system allows to distribute
computations among various sites
 Speedup can be achieved when running
computations in parallel
 Speedup= “serial computation time”/”parallel
computation time”
 In addition, load sharing helps to achieve
speedup: move the jobs from the site overloaded
with jobs to other with lighter load
 Automated load sharing is not yet common
Reason 3: Reliability
• If one site fails in distributed system remaining
sites can continue to operate and get the job done
• The function of failed site can be taken over by
another site
To provide reliability:
• The system must ensure the correct transfer of
function
• Failure of a site must be detected by the system
and the services of that site should not be longer
used
• Mechanisms must be available to integrate the
recovered site back to the system
Reason 4: Communication
• Sites can communicate over a network
by sending messages to one another.
• The advantage of distributed system is
that the messages can be carried over a
great distance (two geographically
distant sites). That provides for efficient
collaboration.
Types of network-based
operating systems
• Network operating systems
• Distributed operating systems
• What is the difference?
Network operating systems (NOS)
Definitions:
• From book: A network operating system provides an
environment in which users, who are aware of the
multiplicity of machines, can access remote resources by
either logging in to the appropriate machine or transferring
data from the remote machine to their own machines.
• From wikipedia: System designed to allow shared file and
printer access among computers in a network, typically a
local area network (LAN), a private network or Intranet.
Hardware architectures
(source:
http://www.cs.jhu.edu/~yairamir/cs437/week7/sld005.htm):
Parallel architecture
Multiprocessors
• Tightly coupled
• Shared memory
Multicomputers
CPU
cache
CPU
cache
Memory
Distributed architecture
• Loosely coupled
• Private memory
• Autonomous
Memory
Memory
Memory
CPU
CPU
CPU
Network OS vs Distributed OS
(source:
http://www.cs.jhu.edu/~yairamir/cs437/week7/sld005.htm):
• Network OS: Contains N copies of operating systems,
communication between machines is via shared files.
• Distributed OS: Contains N copies of operating systems,
communication between nodes is via messages over a
network . These messages pass the necessary parameters
for the task and on completion messages return the results.
Its as if the computers email each other with a request and
an answer.
Network Operating systems
Remote login:
• Network OS (for example Unix) provides an Internet service
that allows the user to log in remotely
• telnet or ssh is used
• After logging in user can compute on remote machine just
as any local user.
• Ssh is used to establish encrypted connection
Remote file transfer (ftp):
• Protocol allows to transfer a file from remote machine to
local computer
• User must know the commands and the exact location of
the file he wants
• Anonymous method provides access to files in directory tree
“anonymous” to anyone
• Since ftp is not secure, scp is currently used for file transfer
Distributed OS characteristics
• In a distributed OS users access remote
resources in the same way they access local
resources
• Data and process migration is under control of
OS
Examples of distributed systems
(source: Wikipedia)
• Network OS- UNIX; Windows 2000 Server; Netware.
• Distributed OS
– Mosix2
• implemented as an OS virtualization layer that provides to users
and applications a single system image with the Linux run-time
environment
• Definition: single system image (SSI) cluster is a cluster of
machines that appears to be one single system.
– Plan 9 (developed for research purposes, planned as UNIX
successor);
– Amoeba (development is frozen). Project lead: Andrew Tannenbaum
(MINIX author)
Examples of distributed systems
• Clusters:
– a group of linked computers often connected in LAN, working
together closely thus in many respects forming a single computer
• Implementations of cluster systems (commercial and free) exists
for many OS
• Categories of cluster systems:
–
–
–
–
• Web
high availability( Linux-HA, Lander Cluster),
load-balancing,
compute (Beowulf , OSCAR)
grid (Rocks).
Rocks cluster architecture
Cluster components
• Front-end node
• Compute node
• Ethernet network
• Application Message Passing Network (optional)
Data migration approaches in
distributed OS
• Transfer entire file (automated FTP system?). It was
used in Andrew file system but found inefficient
• Transfer only portions of the file that are
necessary for immediate task. Another portion requires
another transfer. Protocols that are using this method:
NFS, newer versions of Andrew, SMB (running on top
of TCP/IP or Microsoft NetBEUI)
• The system may also perform data translation (if sites
are using different character code representations)
Computation migration in
distributed OS
• Example: to obtain a summary of large files that reside on different
sites.
• Rather then copy files to one location and obtain a summary, it is
more efficient to access the files at the sites where they reside and the
return result to the initiating site
– Process P wants to access file at site A. Access is initiated by RPC.
Through UDP RPC executes a routine on remote site. P ivokes a
pre-defined procedure at A, the results are returned to P
– P sends a message to A. OS creates a new process Q that executes
the task. When done Q sends the results back to P via the
message system. This way P may execute concurrently with Q.
Either method can be used to access files at different sites
RPC might result in invocation of another RPC
Process migration in distributed
system
• Migration: the process is not get executed at the site where it was
initiated
• Reasons:
– For load balancing
– For speedup: if a process may be divided into subprocesses
– Hardware preference: process is more suitable for execution on some
specialized processor
– Software preference: process requires software that is available only at
particular site
– Data access: the same as in computation migration - for big chunks of data
• If load balancing is needed then a system itself can move processes
• when the process must satisfy hardware -software preference then the
user has to specify how process should migrate
• Examples: Web server/database, Java Applets
Networks for distributed systems
Local area networks (LAN) :
• Composed of processors distributed over small area (single
building)
• Emerged in 1970
• More economical then large mainframe computer
• Data sharing in a single enterprise
• Communication link: high quality cable
• Communication speed: 1Mbit/sec - 1 Gbit/sec
• Typical LAN consist of computers, shared peripheral
devices (printers, tapes) and one or more gateways that
provide access to other networks
• Ethernet scheme is used to construct LAN
• Wireless LAN networks can be built with only one wireless
router
• Compared to Ethernet systems WiFi networks run at slower
speed
Networks for distributed systems
Wide area networks
(WAN):
• Composed of
autonomous
processors
distributed over a
large area (such as US
territory)
• Emerged in 1960s as
academic research
project
• First WAN - Arpanet
- a cluster of four
computers
• Communication link:
phone lines,
microwave links,
satellite channels
Networks topology
Configurations used for
distributed system
network:
–
–
–
–
–
Partially connected network
Fully connected network
Tree-structured network
Star network
Ring network
• Criteria to compare the
configurations:
– Installation cost
– Communication cost
– Availability
Network communication
structure
Communication network design issues:
•
•
•
•
•
Naming and name resolution
Routing strategies
Packet strategies
Connection strategies
Contention
Naming and name resolution
• How could processes at different sites specify each
other to communicate?
– Process identifier (PID) is used for processes
within computer system, but since networked
hosts share no memory different approach is
needed
• Pair <host name, identifier> is used for process
identification
– Host name is alphanumeric
• Host name is human readable, but computers use
host-id instead. There is a mechanism to resolve host
name into host-id - Domain name system
• Hosts in Internet are logically addressed with with
multipart names (IP addresses)
Naming and name resolution
•
•
•
Each component in IP name has name server
A system that needs to use the DNS is configured with the known
addresses of the root servers
Example: bob.cs.brown.ed
How to get IP address for IP name bob.cs.brown.edu ?
1.
Kernel of system A issues a request to the root server for edu domain asking
for the address of the name server for brown.edu
2.
The edu root server returns the address of the host on which brown.edu name
server resides
3.
The kernel on system A then queries the name server at this address and asks
about cs.brown.edu
4.
An address is returned ; and a request to that address returns host-id
128.148.31.100
•
•
•
Java provides API for mapping IP names to IP addresses
Generally, OS is responsible for accepting a message from its
processes for <host name, identifier> and transferring it to
appropriate host
Kernel of destination host transfers the message to the process named
by identifier
Naming and name resolution
Routing strategies
• When a process from site A wants to
communicate with process at site B how is a
message sent?
• Use physical path from A->B
• If multiple paths exists then use Routing
table
– May include info about speed & cost of
communication paths
Routing strategies comparison
• Routing schemes
– Fixed routing
• Path is specified in advance (shortest path) and doesn’t change
– Virtual routing
• Path is fixed for the duration of one session. Examples: file
transfer, remote login period etc
– Dynamic routing
• Path is chosen when message is sent. The message is sent over
the link that is the least used at that particular time
• Tradeoffs
– Order of messages
• Fixed & virtual scheme deliver messages in order
– Environment
• Dynamic scheme performs better in complicated environment
Usually the combination of these schemes is used - for example:
within a site each host has a static route to the gateway, but the
gataway uses dynamic routing to reach any host on the network.
Communication Structure
(Cont.)
Packet Strategies
• Variable length message divided into fixedlength message called packets/frames
• Packets transferred to destination using 2 kinds
of services :
• Reliable service
• Unreliable service
Connection Strategies
• Circuit switching -
A permanent physical link is
established for the duration of the communication (i.e.,
telephone system)
• Message switching - A temporary link is established
for the duration of one message transfer (i.e., post-office
mailing system)
• Packet switching -
Messages of variable length are
divided into fixed-length packets which are sent to the
destination
• Each packet may take a different path through the network
•
The packets must be reassembled into messages as they arrive
Connection Strategies(Cont.)
• Circuit switching requires setup time, but
incurs less overhead for shipping each message,
and may waste network bandwidth
• Message and packet switching require less
setup time, but incur more overhead per
message
Contention
• Several sites may want to transmit information
over a link simultaneously. Techniques to avoid
repeated collision include
• CSMA/CD
• Token Passing
Contention (Cont.)
• CSMA/CD - Carrier sense with multiple access
(CSMA); collision detection (CD)
• A site determines whether another message is currently being
transmitted over that link. If two or more sites begin
transmitting at exactly the same time, then they will register a
CD and will stop transmitting
• When the system is very busy, many collisions may occur, and
thus performance may be degraded
• CSMA/CD is used successfully in the Ethernet
system, the most common network system
Contention (Cont.)
• Token passing - A unique message type,
known as a token, continuously circulates in the
system (usually a ring structure)
• A site that wants to transmit information must wait
until the token arrives
• When the site completes its round of message passing,
it retransmits the token
• A token-passing scheme is used by some IBM and
HP/Apollo systems
Communication Protocols
Communication Protocols
• Simplify the design problem by partitioning the
problem into multiple layers.
• The 2 types of reference models
• ISO's (International Standard Organization)
OSI (Open System Interconnection) Model
• TCP/IP Model
OSI Reference Model
• Defines 7 network layers
Physical Layer
• Handles the mechanical and electrical details of
the physical transmission of a bit stream
• This layer is implemented in the hardware of the
networking device
Data Link Layer
• This layer ensures that messages are delivered to
the proper device on LAN using hardware
address
• Handles the frames, or fixed-length parts of
packets, including any error detection and
recovery that occurred in the physical layer
Network Layer
• Responsible for tracking the location of devices
in the network and determine the best way to
move the data
• Routers work at this level
• Check if the packet belongs to that network else
forward the packet to the destination address
using routing table
Hop-to-hop delivery
Transport Layer
• Segments and reassembles the data into a data stream
• Provides the end-to-end transport services – using
TCP/UDP
• Also called as Reliable Networking – acknowledgment,
maintaining packet order (sequencing) and flow control
Session Layer
• Responsible for setting up, managing and
tearing down the sessions between presentation
layer entities
• Performs message synchronization
Presentation Layer
• Presents the data to the application layer
• Translator – provides coding and conversion
functions (Before transmit the data it has to be in standard
format. Computers are configured to receive this data and convert
onto native format).
• By providing the translation service presentation
layer ensures that data sent from the
Application layer of one system is read by the
Application layers of another one.
Presentation Layer (Cont.)
Application Layer
• Provides services to the end user
• Examples – File Transfer, E-Mail, Remote Login
OSI Reference Model
An OSI network message
Summary of layers
TCP/IP Protocol Layers
Robustness
• Failure Detection
• Reconfiguration
Robustness(Cont.)
• Distributed system may suffer from various
types of failure
• Failure of a link
• Failure of a site
• Loss of message
Failure Detection
• Detecting hardware failure is difficult
• To detect a link failure, a handshaking
protocol can be used
• The conclusion can be :
•
•
•
•
Other Site is down
The direct link between the two Site is down
The alternate link between the two Site is down
The message has been lost
Reconfiguration
• When Site A determines a failure has occurred, it
must reconfigure the system:
• If the link from A to B has failed, this must be
broadcast to every site in the system
• If a site has failed, every other site must also
be notified indicating that the services offered
by the failed site are no longer available
Recovery from failure
• Once the link is repaired both the systems must
be notified (handshake)
• Recover about the site has been notified to all
other sites (failed site must also receive
information from other sites)
Design Issues
• Transparency - the distributed system should appear as a
conventional, centralized system to the user
• Fault tolerance – the distributed system should continue to
function in the face of failure
• Clusters – a collection of semi-autonomous machines that acts
as a single system
• Scalability – as demands increase, the system should easily
accept the addition of new resources to accommodate the
increased demand
Silicon Graphics Cluster
An Example: Networking
• The transmission of a network packet between hosts on an Ethernet
network
• Every host has a unique IP address and a corresponding Ethernet
(MAC) address
• Communication requires both addresses
• Domain Name Service (DNS) can be used to acquire IP addresses
• Address Resolution Protocol (ARP) is used to map MAC addresses to
IP addresses
• If the hosts are on the same network, ARP can be used
• If the hosts are on different networks, the sending host will send the
packet to a router which routes the packet to the destination network
An Ethernet Packet
References
• Operating System Concepts 8th Edition by Abraham Silberschatz,
Peter B. Galvin, Greg Gagne
• http://en.wikipedia.org/wiki/OSI_model
• http://en.wikipedia.org/wiki/TCP/IP_model
• http://en.wikipedia.org/wiki/Distributed_computing
• citeseerx.ist.psu.edu/viewdoc
• http://cs.gmu.edu/~setia/cs475-S03/slides/Client-server.pdf
• http://www.rocksclusters.org
• http://www.cs.jhu.edu/~yairamir/cs437/week7/sld005.htm
Thank you !