Implementing
Remote Procedure Call
Landon Cox
February 12, 2016
Modularity so far
• Procedures as modules
• What is private and what is shared between
procedures?
• Local variables are private
• Stack, heap, global variables are shared
Module 1
Module 2
Code
Code
Private
state
Shared
state
Private
state
Modularity so far
• Procedures as modules
• How is control transferred between procedures?
• Caller adds arguments and RA to stack, jumps into callee code
• Callee sets up local variables, runs code, jumps to RA
Module 1
Module 2
Code
Code
Private
state
Shared
state
Private
state
Modularity so far
• Procedures as modules
• Is modularity between procedures enforced?
• No, either module can corrupt the other
• No guarantee that callee will return to caller either
Module 1
Module 2
Code
Code
Private
state
Shared
state
Private
state
Modularity so far
• MULTICS processes as modules
• What is private, shared btw MULTICS processes?
• Address spaces are private
• Segments can be shared
Module 1
Module 2
Code
Code
Private
state
Shared
state
Private
state
Modularity so far
• MULTICS processes as modules
• How is control transferred between MULTICS
processes?
• Use synchronization primitives from supervisor
• Lock/unlock, wait/notify
Module 1
Module 2
Code
Code
Private
state
Shared
state
Private
state
Modularity so far
• MULTICS processes as modules
• Is modularity btw MULTICS processes enforced?
• Yes, modules cannot corrupt private state of the other
• Isolate shared state inside common segments
Module 1
Module 2
Code
Code
Private
state
Shared
state
Private
state
Modularity so far
• UNIX processes as modules
• What is private and what is shared btw UNIX
processes?
• Address spaces are private
• File system and pipes are shared
Module 1
Module 2
Code
Code
Private
state
Shared
state
Private
state
Modularity so far
• UNIX processes as modules
• How is control transferred between UNIX processes?
• Use synchronization primitives from supervisor
• Block by reading from pipe, notify by writing to pipe
Module 1
Module 2
Code
Code
Private
state
Shared
state
Private
state
Modularity so far
• UNIX processes as modules
• Is modularity between UNIX processes enforced?
• Yes, modules cannot corrupt private state of the other
• Protect shared state using pipe buffer and FS access control
Module 1
Module 2
Code
Code
Private
state
Shared
state
Private
state
Network programming
• Now say two modules are on different machines
• What is the standard abstraction for
communication?
• Sockets
• Each end of socket is bound to an <address, port > pair
Module 1
Module 2
Code
Code
Private
state
Shared
state
Private
state
Network programming
• Now say two modules are on different machines
• Which approach to comm. are sockets most like?
• Most like pipes
• Use read/write primitives for synchronized access to buffer
• What are the downsides of socket programming?
•
•
•
•
Adds complexity to a program
Blocking conditions depend on data received
Data structures copied into and out of messages or streams
All of this work can be tedious and error-prone
• Idea: programmers are used to local procedures
• Try to make network programming as easy as procedure calls
Remote procedure call (RPC)
• RPC makes request/response look
local
• Provides the illusion of a function call
• RPC isn’t a really a function call
• In a normal call, the PC jumps to the function
• Function then jumps back to caller
• This is similar to request/response
though
• Stream of control goes from client to server
• And then returns back to the client
The RPC illusion
• How to make send/recv look like a function
call?
• Client wants
• Send to server to look like calling a function
• Reply from server to look like function returning
• Server wants
• Receive from client to look like a function being called
• Wants to send response like returning from function
Implementing RPC
• Primary challenges
• How to name, locate the remote code to invoke?
• How to handle arguments containing pointers?
• How to handle failures?
RPC architecture
Import
Client
Server
Client
code
Server
code
Interface
Export
Interface
Client
stub
RPC
runtime
Export
Server
stub
Network
Who imports and who exports the interface?
RPC
runtime
Import
RPC architecture
Import
Client
Server
Client
code
Server
code
Interface
Export
Interface
Client
stub
RPC
runtime
Export
Server
stub
Network
Who defines the interface?
The programmer
RPC
runtime
Import
RPC architecture
Import
Client
Server
Client
code
Server
code
Interface
Export
Interface
Client
stub
RPC
runtime
Export
Server
stub
Network
RPC
runtime
Who writes the client and server code?
The programmer
Import
RPC architecture
Import
Client
Server
Client
code
Server
code
Interface
Export
Interface
Client
stub
RPC
runtime
Export
Server
stub
Network
Import
RPC
runtime
Who writes the stub code?
An automated stub generator (rmic in Java)
RPC architecture
Import
Client
Server
Client
code
Server
code
Interface
Export
Interface
Client
stub
RPC
runtime
Export
Server
stub
Network
Import
RPC
runtime
Why can stub code be generated automatically?
Interface precisely defines behavior
What data comes in, what is returned
RPC architecture
Import
Client
Server
Client
code
Server
code
Interface
Export
Interface
Client
stub
RPC
runtime
Export
Server
stub
Network
Import
RPC
runtime
Where else have we seen automated control transfer?
Compilers + procedure calls
RPC stub functions
call
return
return
call
Client
stub
Serve
r
stub
send
recv
send
recv
RPC stub functions
• Client stub
1) Builds request message with server function name and
parameters
2) Sends request message to server stub
• Transfer control to server stub: clients-side code is paused
8) Receives response message from server stub
9) Returns response value to client
• Server stub
3) Receives request message
4) Calls the right server function with the specified parameters
5) Waits for the server function to return
6) Builds a response message with the return value
7) Sends response message to client stub
Binding
• What is binding?
• Establishing map from symbolic name  object
• In an RPC system what needs to be bound?
• Client code uses interface as a symbolic name
• RPC system must bind those names to real code instances
• In Cedar what managed this mapping?
• The Grapevine distributed database
• Types are listed as symbolic names
• Instances are listed as machine addresses
Binding
• Is anyone allowed to export any interface?
• No, this is regulated through Grapevine access controls
• Users allowed to export an interface are explicit in group
• Only group owner can allow someone to export
• Is anyone allowed to import an interface?
• Yes, authentication of clients at higher level
• What other distributed database is Grapevine
like?
• Domain name service (DNS)
• Contains mapping from
symbolic names to IP addrs
Grapevine
Group map: interfaces  user ids
Individual map: user id 
network address
Binding
• Is anyone allowed to export any interface?
• No, this is regulated through Grapevine access controls
• Users allowed to export an interface are explicit in group
• Only group owner can allow someone to export
• Is anyone allowed to import an interface?
• Yes, authentication of clients at higher level
• Are permissions same or different than DNS?
• Basically the same
• DNS updates are controlled
• DNS retrievals are not
Grapevine
Group map: interfaces  user ids
Individual map: user id 
network address
Shared state
• What is the shared state of the RPC abstraction?
• Arguments passed through function call
• What is the actual shared state in RPC?
• The underlying messages between client and server
Client
Server
Code
Code
Private
state
Shared
state
Private
state
Shared state
• Why is translating arguments into messages tricky?
• Data structures have pointers
• Client and server run in different address spaces
• Need to ensure that pointer on client = pointer on server
Client
Server
Code
Code
Private
state
Shared
state
Private
state
Shared state
• How do we ensure that a data structure is safely transferred?
•
•
•
•
Must know the semantics of data structure (typed object references)
Must then replace pointers on client with valid pointers on server
Requires explicit help of programmer to get right
Cannot just pass arbitrary C-style structs and hope to work correctly
Client
Server
Code
Code
Private
state
Shared
state
Private
state
Shared state
• What about after server code completes?
• Must synchronize updates to arguments
• Changes by server must be reflected in client before returning
Client
Server
Code
Code
Private
state
Shared
state
Private
state
Shared state
• What about multi-threaded client code?
•
•
•
•
Must be handled carefully in RPC systems
Client and server cannot synchronize at fine-grain
Client threads must not update state accessible to remote server code
Have to either partition data or properly lock around RPC call
Client
Server
Code
Code
Private
state
Shared
state
Private
state
Faults
• With procedures, what happens if a module faults?
• No isolation, program crashes
• Result of sharing the same address space
• With pipes, what happens if a module faults?
• Faulting module (process) crashes
• OS makes pipe unreadable and unwritable
• Cannot just return an error code through client stub
• Bad idea to overload errors
• Want to distinguish network failures from incorrectness
Faults
• How are RPC faults handled in practice?
•
•
Usually through a software exception
Often supported by language
• So how “pure” is the RPC abstraction?
• Not totally pure
• Programmer still knows which calls are local vs remote
• Have to write code for handling failures
• So is RPC a good abstraction?
• In some cases yes, hides a lot of the complexity
• However, it often comes at a steep performance penalty
• What part of RPC is slowest?
• Argument packing and unpacking
• Java class introspection for shipping data structures is particularly
painful
Structuring a concurrent system
• Talked about two ways to build a
system
Alternative structure
• Can also give cooperating threads own address
spaces
• Each thread is basically a separate process
• Use messages instead of shared data to communicate
• Why would you want to do this?
• Protection
• Each module runs in its own address space
• Reasoning behind micro-kernels
• Each service runs as a separate process
• Mach from CMU (influenced parts Mac OS X)
• Vista/Win7’s handling of device drivers
Course administration
• Project 2
• Write a shell
• Was released yesterday
• Due in two weeks
• Questions?