CSCE 313
Introduction to Computer Systems
Instructor: Amin Hassanzadeh
Fall 2013
http://people.tamu.edu/~hassanzadeh/csce
313.htm
Inter-Process Communication (IPC)
• IPC concept & principle
• POSIX IPC
– Message Queues
– Shared Memory
– Semaphore sets
• Accessing IPC resources from shell
• Reading: Unix Systems Programming, Ch 15
2
Interprocess Communication
• Processes within a system may be independent or
cooperating
• Cooperating process can affect or be affected by other
processes, including sharing data
• Reasons for cooperating processes:
– Information sharing
– Computation speedup
– Modularity
– Convenience
• Cooperating processes need interprocess
communication (IPC)
3
Fundamental Communication Models
4
Interprocess Communication – Message Passing
• A mechanism for processes to communicate and to
synchronize their actions
• Message system – processes communicate with each other
without resorting to shared variables
• IPC facility provides two operations:
– send(message) – message size fixed or variable
– receive(message)
• If P and Q wish to communicate, they need to:
– establish a communication link between them
– exchange messages via send/receive
• Implementation of communication link
– physical (e.g., shared memory, hardware bus)
– logical (e.g., logical properties)
5
Implementation Questions
• How are links established?
• Can a link be associated with more than two processes?
• How many links can there be between every pair of
communicating processes?
• What is the capacity of a link?
• Is the size of a message that the link can accommodate
fixed or variable?
• Is a link unidirectional or bi-directional?
6
Direct Communication
• Processes must name each other explicitly:
– send (P, message) – send a message to process P
– receive(Q, message) – receive a message from process Q
• Properties of communication link
– Links are established automatically
– A link is associated with exactly one pair of
communicating processes
– Between each pair there exists exactly one link
– The link may be unidirectional, but is usually bidirectional
7
Indirect Communication
• Messages are directed and received from mailboxes
(also referred to as ports)
– Each mailbox has a unique id
– Processes can communicate only if they share a
mailbox
• Properties of communication link
– Link established only if processes share a common
mailbox
– A link may be associated with many processes
– Each pair of processes may share several
communication links
– Link may be unidirectional or bi-directional
8
Indirect Communication
• Operations
– create a new mailbox
– send and receive messages through mailbox
– destroy a mailbox
• Primitives are defined as:
send(A, message) – send a message to mailbox A
receive(A, message) – receive a message from
mailbox A
9
Indirect Communication
• Mailbox sharing
– P1, P2, and P3 share mailbox A
– P1, sends; P2 and P3 receive
– Who gets the message?
• Solutions
– 1…
– 2…
– 3…
10
Indirect Communication
• Mailbox sharing
– P1, P2, and P3 share mailbox A
– P1, sends; P2 and P3 receive
– Who gets the message?
• Solutions
– Allow a link to be associated with at most two
processes
– Allow only one process at a time to execute a receive
operation
– Allow the system to select arbitrarily the receiver.
Sender is notified who the receiver was.
11
Synchronization
• Message passing may be either blocking or nonblocking
• Blocking is considered synchronous
– Blocking send has the sender block until the
message is received
– Blocking receive has the receiver block until a
message is available
• Non-blocking is considered asynchronous
– Non-blocking send has the sender send the
message and continue
– Non-blocking receive has the receiver receive a
valid message or null
12
Buffering
• Queue of messages attached to the link;
implemented in one of three ways
1. Zero capacity – 0 messages
Sender must wait for receiver (rendezvous)
2. Bounded capacity – finite length of n messages
Sender must wait if link full
3. Unbounded capacity – infinite length
Sender never waits
13
POSIX IPC: Overview
primitive
POSIX function
description
message queues
msgget
msgctl
msgsnd/msgrcv
create or access
control
send/receive message
semaphores
semget
semctl
semop
create or access
control
wait or post operation
shared memory
shmget
shmctl
shmat/shmdt
create and init or access
control
attach to / detach from
process
Accessing IPC resources from the shell: ipcs [-a]
14
xxGET: It’s all about Naming!
• Condition variables, mutex locks:
– Based on a memory variable concept.
– Does not work across memory spaces!!
• Pipes
– Uses file descriptors
– Works across memory spaces.
– Relies on inheritance of file descriptors -> does not work for
unrelated processes.
• Named Pipes
– Uses file system as name space for pipe.
– Works for unrelated processes.
– Carry the overhead of the file system.
• IPC Objects
– Use system-global integer keys to refer to objects.
15
IPC Object Creation: Message Queues
Object key identifies object across processes. Can be assigned
as follows:
-- Create some unknown key (IPC_PRIVATE) by the system
-- Pass explicit key (beware of collisions!)
-- Use file system to consistently hash key (using ftok)
#include <sys/msg.h>
int msgget(key_t key, int msgflg);
/* create a message queue with given key and flags. */
Object id is similar to file descriptor.
-- It can be inherited across fork() calls.
16
ftok()
• ftok - generate an IPC key
• The ftok()function shall return a key based on path and
id that is usable in subsequent calls to msgget(), semget(),
and shmget(). The application shall ensure that the path
argument is the pathname of an existing file that the
process is able to stat().
• Upon successful completion, ftok() shall return a key.
Otherwise, ftok() shall return (key_t)-1 and set errno to
indicate the error.
17
Operations on Message Queues
#define PERMS (S_IRUSR | S_IWUSR)
int msqid;
if ((msqid = msgget(IPC_PRIVATE, PERMS)) == -1)
perror(“msgget failed);
struct mymsg { /* user defined! */
long msgtype; /* first field must be a long identifier */
char mtext[1]; /* placeholder for message content */
} mymsg_t;
int msgsnd(int msqid, const void *msgp,
size_t msgsz, int msgflg)
msgtyp
0
action
ssize_t msgrcv(int msqid, void *msgp, size_t msgsz,
long msgtyp, int msgflg);
remove first message from queue
>0
remove first message of type msgtyp from the queue
<0
remove first message of lowest type that is less than or
equal to absolute value of msgtyp
18
Operations on Message Queues - Send
19
Operations on Message Queues (cont.)
int msgctl(int msqid, int cmd, struct msgid_ds *buf)
Cmd
IPC_RMID
IPC_SET
IPC_STAT
description
remove the message queue msqid and destroy the
corresponding msqid_ds
Set members of the msqid_ds data structure from buf
Copy members of the msqid_ds data structure into buf
20
POSIX Message Queues Example
#define MSGSZ 128
#define MSGKY 1234
typedef struct the_msg {
long mtype;
char mtext[MSGSZ];
} message_buf;
21
POSIX Message Queues Example
main () {
int msqid;
int msgflg = IPC_CREAT | 0666;
message_buf sbuf;
memset(&sbuf.mtext, 0, (size_t) MSGSZ);
if((msqid = msgget((key_t) MSGKY, msgflg)) < 0) {
perror("msgget");
exit(1);
}
sbuf.mtype = 1;
strcpy(sbuf.mtext, "Did you get this?\n\0");
if(msgsnd(msqid, &sbuf, (size_t)(strlen(sbuf.mtext)), 0) < 0) {
perror("msgsnd");
exit(1);
}
printf("Message sent...\n");
exit(0);
}
22
POSIX Message Queues Example
main () {
int msqid;
message_buf rbuf;
memset(&rbuf.mtext, 0, (size_t) MSGSZ);
if((msqid = msgget((key_t) MSGKY, 0666)) < 0) {
perror("msgget");
exit(1);
}
if (msgrcv(msqid, &rbuf, MSGSZ, 1, 0) < 0) {
perror("msgrcv");
exit(1);
}
printf("%s\n", rbuf.mtext);
exit(0);
}
23
Message Queue Program Example
• Program 15.9 msgqueuelog.c (ch15)
– Utility functions that access and output to a message
queue
• Program 15.10 msgqueuesave.c (ch15)
– A program that copies messages from a message
queue to standard output
24
POSIX Shared Memory
• A range of addresses created by the IPC facility for one process
• Other processes can “attach” the same shared memory segment into
their own address space.
• Applications use this memory just as if it had been allocated by malloc
• When one application writes to shared memory in this range, the
change is immediately available to any other process that has access
to the same shared memory
• Shared memory doesn’t provide any synchronization facilities.
• It is up to the programmer to synchronize access.
25
POSIX Shared Memory
#include <sys/shm.h>
int shmget(key_t key, size_t size, int shmflg);
Ok, we have created a shared-memory segment. Now what?
address space of
calling process
P1
shared-memory
segment mapped by
shmat
void *shmat(int shmid, const void *shmaddr,
int shmflg);
system memory
shared-memory
segment created by
shmget
address space of
calling process
P2
shared-memory
segment mapped by
shmat
26
Use POSIX Shared Memory
– Process first creates shared memory segment
segment id = shmget(IPC PRIVATE, size, S IRUSR | S IWUSR);
– Process wanting access to that shared memory must
attach to it
shared memory = (char *) shmat(id, NULL, 0);
– Now the process could write to the shared memory
sprintf(shared memory, "Writing to shared memory");
– When done a process can detach the shared memory
from its address space
shmdt(shared memory);
27
1. shmget()
this function is used to create a shared memory segment
the amount of memory required, in bytes
#include <sys/shm.h>
int shmget(key_t key, size_t size, int shmflg);
returns a shared
memory identifier
or -1
a programmer defined key
permission bits
and IPC_CREAT
28
2. shmctl()
#include <sys/shm.h>
int shmctl(int shm_id, int command, struct shmid_ds *buf);
returns 0 if successful
or -1 if fails
the shared memory
segment’s identifier
struct shmid_ds
{
uid_t shm_perm.uid;
uid_t shm_perm.gid;
mode_t shm_perm.mode;
}
IPC_STAT
Sets the data in the shmid structure to reflect
the values associated with the shared memory.
IPC_SET
Sets the values associated with the shared
memory to those provided in the data structure.
IPC_RMID
Delete the shared memory
29
3. shmat()
this function attaches the shared memory segment to the process’s
address space.
#include <sys/shm.h>
void *shmat(int shm_id, const void *shm_addr, int shmflg);
returns a pointer to
the shared memory.
the shared memory identifier
gotten from shmget()
if non-zero, the segment is
attached for read-only. If 0,
it is attached read/write.
address in the process’s address
space where the shared memory
is to be attached. Normally this
is a NULL pointer, allowing the
system to determine where.
This is the recommended form.
30
3. shmdt()
this function detaches the shared memory segment
#include <sys/shm.h>
int shmdt(const void *shm_addr);
returns 0 if successful
or -1 if not
pointer to the shared memory
segment to be detached.
31
POSIX Shared Memory Example
a
b
c
d
e
.
.
.
Producer
\n
\0
Consumer
shared
memory
32
POSIX Shared Memory - Producer
#define SHMSZ 27
// the size of the shared memory segment
#define SHMKY 5678
// the key of the shared memory segment
main ()
{
char c;
int shm_id;
char *shm, *s;
// create a shared memory segment.
if ((shm_id = shmget((key_t)SHMKY, SHMSZ, IPC_CREAT | 0666)) < 0 )
{
perror("shmget");
exit(1);
}
// attach the shared memory segment, shm points to it
if ((shm = shmat(shm_id, NULL, 0)) == (char *)-1)
{
perror("shmat");
exit(1);
}
33
POSIX Shared Memory - Producer
// Now write out a string of characters
// Add a line feed and Null terminate the string.
s = shm;
for (c = 'a'; c <= 'z'; c++)
{
*s++ = c;
}
*s++ = '\n';
*s = '\0';
// Finally, wait until the consumer writes a '*'
// in the first character position to signal that
// it is done.
while(*shm != '*')
{
sleep(1);
}
printf("All done ... data has been read!\n");
exit(0);
}
34
POSIX Shared Memory - Consumer
#define SHMSZ 27 // the size of the shared memory block
#define SHMKY 5678
// the key of the shared memory block
main ()
{
int shm_id;
char *shm, *s;
if ((shm_id = shmget((key_t)SHMKY, SHMSZ, 0666)) < 0 ) {
perror("shmget");
exit(1);
}
if ((shm = shmat(shm_id, NULL, 0)) == (char *)-1)
{
perror("shmat");
exit(1);
}
for (s = shm; *s != '\0'; s++)
{
getchar(*s);
}
*shm = '*';
exit(0);
}
35
Accessing IPC Resources From the Shell
• POSIX:XSI defines shell extensions for examining and
deleting IPC resources.
ipcs [-qms] [ a | -bcot ]
• With no parameters, the ipcs command displays an
abbreviated set of information about all message queues,
shared memory, and semaphore sets .
ipcrm [ -q msgid | -Q msgkey | -s semid | -S semkey |
-m shmid | -M shmkey ]
removes an individual IPC object.
36
Shared Memory Program Example
• Program 15.5 monitorshared.c
– Monitor two file descriptors and keep information in
shared memory. The parent waits for the child, to
ensure mutual exclusion
• Program 15.6 sharedmemsum.c
– A function that keeps a synchronized sum and count
in shared memory
• Program 15.7 showshared.c
– Display the shared count and sum when it receives a
SIGUSR1 signal
37
POSIX:XSI Semaphores
• POSIX:XSI semaphores are part of the general
POSIX interprocess communication facility
• A POSIX:XSI semaphore is created by executing a
semget() system call
• The semget() creates a semaphore data structure in
the kernel and returns an integer handle to the
semaphore
• Processes cannot access semaphore data
structures directly –only through system calls
• Semaphore ids or handles are analogous to file
descriptors
38
POSIX:SEM vs POSIX:XSI Semaphores
• POSIX:XSI data structures are created and kept in the
kernel and are referenced through integer handles
• In POSIX:SEM, a program declares a variable of type
sem_t and passes a pointer to that variable
39
Semaphore Sets
• A semaphore set is an array of semaphore elements
• A process can perform operations on the entire set in a
single system call
• The internal representation of semaphore sets and
semaphore elements is not directly accessible
• Each semaphore includes at least the following:
– A nonnegative integer representing semaphore value
– Process ID of the last process to manipulate the
semaphore element
– Number of processes waiting for the semaphore
element to increase
– Number of processes waiting for the semaphore
element value to equal 0
40
1. semget( )
the key is like a file name. Processes can cooperate
using a shared semaphore if they agree on a
common key.
int semget ( key_t key, int num_sems, int sem_flags);
semget creates a new semaphore or
obtains the key of an existing semaphore.
The semaphore identifier is returned. The
identifier is like a file descriptor, and is
used within the process to access the
semaphore. Returns -1 if the call fails.
the number of semaphores
required.
like the open flags on a file:
IPC_PRIVATE: only this process can use the semaphore
IPC_CREAT: create a new semaphore if one doesn’t exist
IPC_EXCL: make sure semaphore is unique
If a process attempts to create a semaphore that already exists,
it receives a handle to the existing semaphore, unless sem_flags
specifies both IPC_CREAT and IPC_EXCL.
41
POSIX Semaphore Sets
#include <sys/sem.h>
int semget(key_t key, int nsems, int
semflg);
/* Create semaphore set with nsems
semaphores. If set exists, nsems can be
#include <sys/sem.h> zero. */
#define PERMS
(S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP|S_IROTH|S_IWOTH)
#define SET_SIZE 2
int main (int argc, char * argv[]) {
key_t mykey;
int
semid;
mykey = ftok(argv[1], atoi(argv[2]));
semid = semget(mykey, SET_SIZE, PERMS | IPC_CREAT)
return 0;
}
42
semget() example
#include <sys/sem.h>
#define PERMS S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP|S_IROTH|S_IWOTH
#define SET_SIZE 1
#define KEY ((key_t) 99887)
void main(void) {
int semid;
if ((semid=semget(KEY, SET_SIZE, PERMS | IPC_CREAT)) < 0)
fprintf(stderr, "Error creating semaphore with key %d:
%s\n“,(int)KEY,strerror(errno));
else
printf("Semaphore with ID %d created for key
%d\n",semid,(int)KEY);
}
43
semget() example
#include <sys/sem.h>
#define PERMS S_IRUSR|S_IWUSR
#define SET_SIZE 3
int semid;
void main(void) {
int semid;
if ((semid = semget(IPC_PRIVATE, SET_SIZE, PERMS)) < 0)
perror("Could not create new private semaphore");
else
printf("Semaphore created with ID %d\n",semid);
}
44
Semaphore Set Control semctl()
#include <sys/sem.h>
int semctl(int semid, int semnum, int cmd, …);
command
description
SETVAL
set value of a specific semaphore element to arg.val
SETALL
set values of semaphore set from arg.array
GETVAL
return value of specific semaphore element
GETALL
return values of the semaphore set in arg.array
GETPID
return process id of last process to manipulate element
GETNCNT
return number of processes waiting for element to increment
GETZCNT
return number of processes waiting for element to become 0
etc….
45
2. semctl()
Each element in a semaphore set must be initialized with semctl before it is used.
int semctl ( int sem_id, int sem_num, int command, union semun arg);
the semaphore set
identifier from semget
The Semaphore to
set
union semun
{
int val; // value for SETVAL
struct semid_ds *buf; // buffer for IPC_STAT, I
unsigned short int *array; // array for GETALL,
struct seminfo * _buf; // buffer for IPC_INFO
};
Commonly Used comands:
GETVAL – return the value of the semaphore
GETPID – return last process ID that manipulated semaphore
GETNCNT – return number of processes waiting for an element to increase
GETZCNT – return number of processes waiting for element to become 0
SETVAL – initialize a semaphore to a known value
IPC_RMID – delete a semaphore ID
IPC_STAT – copy info from the semaphore set data structure into _buf
IPC_SET – write values from _buf into the semaphore set data structure
46
semctl() example
#include <sys/sem.h>
int initelement(int semid, int semnum, int semvalue)
{
union semun {
int val;
struct semid_ds *buf;
unsigned short *array;
} arg;
arg.val = semvalue;
return semctl(semid, semnum, SETVAL, arg);
}
sets the value of the specified semaphore element to semvalue
47
semctl() example
#include <sys/sem.h>
int removesem(int semid) {
return semctl(semid, 0, IPC_RMID);
}
deletes the semaphore specified by semid
48
3. semop()
#include <sys/sem.h>
int semop(int semid, struct sembuf *sops, size_t nsops);
/* The operations are defined in the array pointed to
by ‘sops’. */
struct sembuf contains approximately the following members:
short sem_num
number of semaphore element
short sem_op
operation to be performed
IPC_NOWAIT
short sem_flg
specific options for the operation SEM_UNDO.
sem_op > 0
add the value to the semaphore element and awaken
all processes that are waiting for element to increase
(sem_op == 0)
block calling process (waiting for 0) and increment
&& (semval != 0) count of processes waiting for 0.
sem_op < 0
add sem_op value to semaphore element value
provided that result would not be negative.
If result negative, block process on event that
semaphore value increases.
If result == 0, wake processes waiting for 0.
49
3. semop()
#include <sys/sem.h>
int semop(int semid, struct sembuf *sops, size_t nsops);
/* The operations are defined in the array pointed to
by ‘sops’. */
struct
short
short
short
sembuf contains approximately the following members:
sem_num
number of semaphore element
sem_op
operation to be performed
IPC_NOWAIT
sem_flg
specific options for the operation SEM_UNDO.
IPC_NOWAIT -- Can be set for any operations in the array. Makes the
function return without changing any semaphore value if any operation for
which IPC_NOWAIT is set cannot be performed. The function fails if it tries
to decrement a semaphore more than its current value, or tests a nonzero
semaphore to be equal to zero.
SEM_UNDO -- Allows individual operations in the array to be undone when
the process exits.
50
Semaphore Set Operations: Example
Consider the following function written to set up the semop
structure sembuf:
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/sem.h>
void setsembuf (struct sembuf *s, int num, int op, int flg)
{
s->sem_num = (short)num;
s->sem_op = op;
s->sem_flg = flg;
return;
}
51
Semaphore Operations: Example
Mutually-Exclusive Access to Two Tapes:
/* pseudo code */
struct sembuf get_tapes[2];
struct sembuf release_tapes[2];
setsembuf(&(get_tapes[0]),
setsembuf(&(get_tapes[1]),
setsembuf(&(release_tapes[0]),
setsembuf(&(release_tapes[1]),
0,
1,
0,
1,
-1,
-1,
+1,
+1,
0);
0);
0);
0);
/* Process 1: */
semop(S, get_tapes, 1);
<use Tape 0>
semop(S, release_tapes, 1);
/* Process 2: */
semop(S, get_tapes, 2);
<use both tapes 0 and 1>
semop(S, release_tapes, 2);
52
Semaphore Operations: Another Example
Semaphore to control access to critical section:
int main(int argc, char * argv[]) {
int semid;
struct sembuf sem_signal[1];
struct sembuf sem_wait[1];
semid = semget(IPC_PRIVATE, 1, PERMS);
setsembuf(sem_wait, 0, -1, 0);
setsembuf(sem_signal, 0, 1, 0);
if(-1 == initelement(semid, 0, 1) /* Set value of element 0 to 1 */
removesem(semid);
for (int i = 1; i < n; i++) if fork() break;
semop(semid, sem_wait, 1); /* enter critical section */
/* in critical section */
semop(semid, sem_signal, 1); /* leave critical section */
/* in remainder section */
return 0;
}
53