Lock and IPC (Inter-Process Communication)
(Chap 12, 14 in the book “Advanced Programming in the UNIX Environment”)
Acknowledgement : Prof. Y. Moon at Kangwon Nat’l Univ.
Lock

a synchronization mechanism for enforcing limits on
access to a resource in an environment where there are
many threads of execution

Advisory lock



Other processes can still access (read or write) to a locked resource
There is a way to check whether a resource is locked
Mandatory lock

Prevent access to a locked resource
2
Advisory Locking
#include <fcntl.h>
int flock(int fd, int operation);
Returns: 0 if OK, -1 otherwise


Applies or removes an advisory lock on the file associated with the file
descriptor fd
Operation can be





LOCK_SH
LOCK_EX
LOCK_NB : non-blocking mode. Use together with other lock modes by OR ‘|’
LOCK_UN
Locks entire file
3
Advisory Record Locking
#include <fcntl.h>
int fcntl(int fd, int cmd, struct flock* lock);
Returns: depends on cmd if OK, -1 on error
struct flock {
short l_type;
off_t l_start;
short l_whence;
off_t l_len;
pid_t l_pid;
}

/*
/*
/*
/*
/*
F_RDLCK, F_WRLCK, or F_UNLCK */
offset in bytes from l_whence */
SEEK_SET, SEEK_CUR, SEEK_END */
length, in bytes; 0 means “lock to EOF */
returned by F_GETLK */
Lock types are :



F_RDLCK : non-exclusive (read) lock; fails if write lock exists
F_WRLCK : exclusive (write) lock; fails if any lock exists
F_UNLCK : releases our lock on specified range
4
Setting and Clearing a lock
struct flock fl;
int fd;
fl.l_type = F_WRLCK; /* F_RDLCK, F_WRLCK, F_UNLCK */
fl.l_whence = SEEK_SET; /* SEEK_SET, SEEK_CUR, SEEK_END */
fl.l_start = 0; /* Offset from l_whence */
fl.l_len = 0; /* length, 0 = to EOF */
fl.l_pid = getpid(); /* our PID */
fd = open("filename", O_WRONLY);
fcntl(fd, F_SETLKW, &fl); /* F_GETLK, F_SETLK, F_SETLKW */
fl.l_type = F_UNLCK; /* tell it to unlock the region */
fcntl(fd, F_SETLK, &fl); /* set the region to unlocked */
• cmd
-F_SETLKW: obtain lock
-F_SETLK: obtain lock but no wait if it cannot obtain the lock
-F_GETLK: only check to see if there is a lock
5
Non-Blocking(Asynchronous) I/O

A form of I/O that permits subsequent processing
to continue before current processing has finished.
 Non Blocking I/O function returns immediately
 Two ways for specifying non-blocking I/O


When calling “open” and specify O_NONBLOCK flag
For a file descriptor that is already open, call fcntl to
turn on O_NONBLOCK file status flag.
6
IPC
(Inter-Process Communication)





Pipes
FIFOs
Message Queue
Shared Memory
Semaphores
Page 7
Contents
APUE (Interprocess Communication
Pipes
FIFOs
System V IPC
•
Message Queues
•
Shared Memory
•
Semaphores
Page 8
IPC using Pipes
• IPC using regular files
• unrelated processes can share
• fixed size
• lack of synchronization
• IPC using pipes
• for transmitting data between related processes
• can transmit an unlimited amount of data
• automatic synchronization on open()
Page 9
Pipes in a UNIX Shell
In a UNIX shell, the pipe symbol is: | (the vertical bar)
In a shell, UNIX pipes look like:
$ ls -al | more
• where the standard output of the program at the left (i.e., the producer) becomes the standard input of the
program at the right (i.e., the consumer).
We can have longer pipes:
$ ls –l | grep *.c | wc > output.file
Page 10
Example (1/2)
$ who | sort
who
write pointer of
another process
pipe
sort
read pointer of
one process
Page 11
Example (2/2)
Page 12
IPC using Pipes
APUE (Interprocess Communication
• Data transmitting
• data is written into pipes using the write() system call
• data is read from a pipe using the read() system call
• automatic blocking when full or empty
• Types of pipes
• (unnamed) pipes
• named pipes (FIFOs)
Page 13
Pipes (1/4)
• In UNIX, pipes are the oldest form of IPC.
• Limitations of Pipes:
• Half duplex (data flows in one direction)
• Can only be used between processes that have a common ancestor
(Usually used between the parent and child processes)
• Processes cannot pass pipes and must inherit them from their parent
• If a process creates a pipe, all its children will inherit it
Page 14
Pipes (2/4)
APUE (Interprocess Communication
#include <unistd.h>
int pipe(int fd[2])
Returns: 0 if OK, -1 on error
• Two file descriptors are returned through the fd argument
•
fd[0]: can be used to read from the pipe, and
•
fd[1]: can be used to write to the pipe
• Anything that is written on fd[1] may be read by fd[0].
•
This is of no use in a single process.
•
However, between processes, it gives a method of communication
• The pipe() system call gives parent-child processes a way to communicate with
each other.
Page 15
Pipes (3/4)
parent  child:
parent closes fd[1]
child closes fd[0]
parent  child:
parent closes fd[0]
child closes fd[1]
parent
fd[1]
child
parent
fd[0]
fd[0]
child
fd[1]
pipe
pipe
kernel
kernel
Page 16
Pipes (4/4)
• Read from a pipe with write end closed: (fd[1]이 close된 경우)
• returns 0 to indicate EOF
• Write to a pipe with read end closed: (fd[0]가 close된 경우)
• SIGPIPE generated,
• write() returns error (errno == EPIPE)
Page 17
example: pipe.c (1/2)
#include <stdio.h> // pipe.c
#define READ 0
#define WRITE 1
char* phrase = "Stuff this in your pipe and smoke it";
main( ) {
int fd[2], bytesRead;
char message[100];
pipe(fd);
if (fork() == 0) { // child
close(fd[READ]);
write(fd[WRITE], phrase, strlen(phrase)+1);
fprintf(stdout, "[%d, child] write completed.\n", getpid());
close(fd[WRITE]);
}
else { // parent
close(fd[WRITE]);
bytesRead = read(fd[READ],
message, 100);
fprintf(stdout, "[%d, parent] read completed.\n", getpid());
printf("Read %d bytes: %s\n", bytesRead,message);
close(fd[READ]);
}
}
Page 18
example: pipe.c (2/2)
실행 결과
Page 19
Contents
Pipes
FIFOs
System V IPC
•
Message Queues
•
Shared Memory
•
Semaphores
Page 20
FIFOs (1/3)
APUE (Interprocess Communication
• Pipes can be used only between related processes.
(e.g., parent and child processes)
• FIFOs are "named pipes" that can be used between unrelated
processes.
• A type of file
• stat.st_mode == FIFO
• Test with S_ISFIFO() macro
Page 21
FIFOs (2/3)
APUE (Interprocess Communication
#include <sys/types.h>
#include <sys/stat.h>
int mkfifo(const char *pathname, mode_t mode);
Returns: 0 if OK, -1 on error
• Creating FIFOs is similar to creating a file.
• pathname: filename
• mode: permissons, same as for open() function
• Using a FIFO is similar to using a file.
• we can open, close, read, write, unlink, etc., to the FIFO.
Page 22
FIFOs (3/3)
• if FIFO opened without O_NONBLOCK flag
• an open for read-only blocks until some other process opens the FIFO for writing
• an open for write-only blocks until some other process opens the FIFO for reading
• if O_NONBLOCK is specified (nonblocking)
• an open for read-only returns immediately if no process has the FIFO open for writing
• an open for write-only returns an error (errno=ENXIO) if no process has the FIFO open for reading
• Like a pipe, if we write to a FIFO that no process has open for
reading, the signal SIGPIPE is generated.
• When the last writer for a FIFO closes the FIFO, an end of file (EOF)
is generated for the reader of the FIFO.
Page 23
Uses of FIFOs
• Used by shell commands to pass data from one shell pipeline to another,
without creating intermediate files.
• Used in client-server application to pass data between clients and server.
Page 24
Using FIFOs to Duplicate Output Streams
tee(1) copies its standard input to both its standard output and to the file named
on its command line.
$ mkfifo fifo1
$ prog3 < fifo1 &
$ prog1 < infile | tee fifo1 | prog2
fifo1
prog3
infile
prog1
tee
prog2
Page 25
An Example using a FIFO
APUE (Interprocess Communication
Page 26
Client-Server Communication Using a FIFO
Server creates a “well-known” FIFO to communicate with clients.
client
write request
.
.
.
well-known
FIFO
read
request
server
write request
client
Problem: Server can't reply clients using a single “well-known” FIFO
Page 27
Contents
•
Pipes
•
FIFOs
•
System V IPC
•
Message Queues
•
Shared Memory
•
Semaphores (간단히)
Page 28
System V IPC
• Message Queues
• Send and receive amount of data called “messages”.
• The sender classifies each message with a type.
• Shared Memory
• Shared memory allows two or more processes to share a given region of memory.
• Readers and writers may use semaphore for synchronization.
• Semaphores
• Process synchronization and resource management
• For example, a semaphore might be used to control access to a device like printer.
Page 29
Identifiers & Keys
• Identifier: each IPC structure has a nonnegative integer
• Key: when creating an IPC structure, a key must be specified (key_t)
id = xxxget(key, …)
• How to access the same IPC?  key in a common header
• Define a key in a common header
• Client and server agree to use that key
• Server creates a new IPC structure using that key
• Problem when the key is already in use
−
(msgget, semget, shmget returns error)
−
Solution: delete existing key, create a new one again!
Page 30
IPC System Calls
APUE (Interprocess Communication
• msg/sem/shm get
• Create new or open existing IPC structure.
• Returns an IPC identifier
• msg/sem/shm ctl
• Determine status, set options and/or permissions
• Remove an IPC identifier
• msg/sem/shm op
• Operate on an IPC identifier
• For example(Message queue)
−
add new msg to a queue (msgsnd)
−
receive msg from a queue (msgrcv)
Page 31
Permission Structure
• ipc_perm is associated with each IPC structure.
• Defines the permissions and owner.
struct ipc_perm {
uid_t uid;
/*
gid_t gid;
/*
uid_t cuid; /*
gid_t cgid; /*
mode_t mode; /*
ulong seq;
/*
key_t key;
/*
};
owner's effective user id */
owner's effective group id */
creator's effective user id */
creator’s effective group id */
access modes */
slot usage sequence number */
key */
Page 32
Message Queues (1/2)
• Linked list of messages
• Stored in kernel
• Identified by message queue identifier (in kernel)
• msgget
• Create a new queue or open existing queue.
• msgsnd
• Add a new message to a queue
• msgrcv
• Receive a message from a queue
• Fetching order: based on type
Page 33
Message Queues (2/2)
• Each queue has a structure
struct msqid_ds {
struct ipc_perm msg_perm;
struct msg *msg_first; /*
struct msg *msg_last; /*
ulong msg_cbytes;
/*
ulong msg_qnum;
/*
ulong msg_qbytes;
/*
pid_t msg_lspid;
/*
pid_t msg_lrpid;
/*
time_t msg_stime;
/*
time_t msg_rtime;
/*
time_t msg_ctime;
/*
};
ptr to first msg on queue */
ptr to last msg on queue */
current # bytes on queue */
# msgs on queue */
max # bytes on queue */
pid of last msgsnd() */
pid of last msgrcv() */
last-msgsnd() time */
last-msgrcv() time */
last-change time */
• We can get the structure using msgctl() function.
• Actually, however, we don’t need to know the structure in detail.
Page 34
msgget()
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/msg.h>
int msgget(key_t key, int flag);
Returns: msg queue ID if OK, -1 on error
• Create new or open existing queue
• flag : ipc_perm.mode
• Example
msg_qid = msgget(DEFINED_KEY, IPC_CREAT | 0666);
Page 35
msgctl()
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/msg.h>
int msgctl(int msqid, int cmd, struct msqid_ds *buf);
Returns: 0 if OK, -1 on error
• Performs various operations on a queue
• cmd = IPC_STAT:
fetch the msqid_ds structure for this queue, storing it in buf
• cmd = IPC_SET:
set the following four fields from buf: msg_perm.uid, msg_perm.gid,
msg_perm.mode, and msg_qbytes
• cmd = IPC_RMID:
remove the message queue.
Page 36
msgsnd()
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/msg.h>
int msgsnd(int msqid, const void *ptr, size_t nbytes, int flag);
Returns: 0 if OK, -1 on error
• msgsnd() places a message at the end of the queue.
• ptr: pointer that points to a message
• nbytes: length of message data
• if flag = IPC_NOWAIT: IPC_NOWAIT is similar to the nonblocking I/O flag for file
I/O.
• Structure of messages
struct mymesg {
long mtype;
char mtext[512];
};
/* positive message type */
/* message data, of length nbytes */
Page 37
msgrcv()
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/msg.h>
int msgrcv(int msqid, void *ptr, size_t nbytes, long type, int flag);
Returns: data size in message if OK, -1 on error
• msgrcv() retrieves a message from a queue.
• type == 0: the first message on the queue is returned
• type > 0: the first message on the queue whose message type equals type is
returned
• type < 0: the first message on the queue whose message type is the lowest value
less than or equal to the absolute value of type is returned
• flag may be given by IPC_NOWAIT
Page 38
example: sender.c receiver.c (1/4)
#include
#include
#include
#include
<stdio.h> // sender.c
<sys/types.h>
<sys/ipc.h>
<sys/msg.h>
#define DEFINED_KEY 0x10101010
main(int argc, char **argv)
{
int msg_qid;
struct {
long mtype;
char content[256];
} msg;
fprintf(stdout, "=========SENDER==========\n");
if((msg_qid = msgget(DEFINED_KEY, IPC_CREAT | 0666)) < 0) {
perror("msgget: "); exit(-1);
}
}
msg.mtype = 1;
while(1) {
memset(msg.content, 0x0, 256);
gets(msg.content);
if(msgsnd(msg_qid, &msg, sizeof(msg.content), 0) < 0) {
perror("msgsnd: "); exit(-1);
}
}
Page 39
example: sender.c receiver.c (2/4)
#include
#include
#include
#include
<stdio.h> // receiver.c
<sys/types.h>
<sys/ipc.h>
<sys/msg.h>
#define DEFINED_KEY 0x10101010
main(int argc, char **argv)
{
int msg_qid;
struct {
long mtype;
char content[256];
} msg;
fprintf(stdout, "=========RECEIVER==========\n");
if((msg_qid = msgget(DEFINED_KEY, IPC_CREAT | 0666)) < 0) {
perror("msgget: "); exit(-1);
}
}
while(1) {
memset(msg.content, 0x0, 256);
if(msgrcv(msg_qid, &msg, 256, 0, 0) < 0) {
perror("msgrcv: "); exit(-1);
}
puts(msg.content);
}
Page 40
example: sender.c receiver.c (3/4)
APUE (Interprocess Communication
Page 41
example: sender.c receiver.c (4/4)
Page 42
Shared Memory
• Allows multiple processes to share a region of memory
• Fastest form of IPC: no need of data copying between client & server
• If a shared memory segment is attached
• It become a part of a process data space, and shared among multiple processes
• Readers and writers may use semaphore to
• synchronize access to a shared memory segment
Page 43
Shared Memory Segment Structure
APUE (Interprocess Communication
• Each shared memory has a structure
struct shmid_ds {
struct ipc_perm shm_perm;
struct anon_map *shm_amp; /* pointer in kernel */
int shm_segsz;
/* size of segment in bytes */
ushort shm_lkcnt;
/* # of times segment is being locked */
pid_t shm_lpid;
/* pid of last shmop() */
pid_t shm_cpid;
/* pid of creator */
ulong shm_nattch;
/* # of current attaches */
ulong shm_cnattch; /* used only for shminfo() */
time_t shm_atime;
/* last-attach time */
time_t shm_dtime;
/* last-detach time */
time_t shm_ctime;
/* last-change time */
};
• We can get the structure using shmctl() function.
• Actually, however, we don’t need to know the structure in detail.
Page 44
shmget()
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
int shmget(key_t key, int size, int flag);
Returns: shared memory ID if OK, -1 on error
• Obtain a shared memory identifier
• size: is the size of the shared memory segment
• flag: ipc_perm.mode
• Example
shmId = shmget(key, size, PERM|IPC_CREAT|IPC_EXCL|0666);
Page 45
shmctl()
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
int shmctl(int shmid, int cmd, struct shmid_ds *buf);
Returns: 0 if OK, -1 on error
• Performs various shared memory operations
• cmd = IPC_STAT:
fetch the shmid_ds structure into buf
• cmd = IPC_SET:
set the following three fields from buf: shm_perm.uid, shm_perm.gid, and
shm_perm.mode
• cmd = IPC_RMID:
remove the shared memory segment set from the system
Page 46
shmat()
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
void *shmat (int shmid, void *addr, int flag);
Returns: pointer to shared memory segment if OK, -1 on error
• Attached a shared memory to an address
• flag = SHM_RDONLY: the segment is read-only
• addr==0: at the first address selected by the kernel (recommended!)
• addr!=0: at the address given by addr
Page 47
Memory Layout
high address
command-line arguments
and environment variables
stack
0xf7fffb2c
0xf77e86a0
0xf77d0000
shared memory of 100,000 bytes
heap
0x0003d2c8
0x00024c28
malloc of 100,000 bytes
uninitialized data
(bss)
0x0003d2c8
0x00024c28
array[] of 40,000 bytes
shared memory
initialized data
low address
text
Page 48
shmdt()
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
void shmdt (void *addr);
Returns: 0 if OK, -1 on error
• Detach a shared memory segment
Page 49
example: tshm.c (1/2)
#include
#include
#include
#define
#define
#define
<sys/types.h> // tshm.c
<sys/ipc.h>
<sys/shm.h>
ARRAY_SIZE
MALLOC_SIZE
SHM_SIZE
100000
100000
100000
err_sys(char *p) { perror(p); exit(-1); }
char
array[ARRAY_SIZE];
int main(void) {
int shmid; char
/* uninitialized data = bss */
*ptr, *shmptr;
printf("array[] from %x to %x\n", &array[0], &array[ARRAY_SIZE]);
printf("stack around %x\n", &shmid);
if ((ptr = malloc(MALLOC_SIZE)) == NULL) err_sys("malloc error");
printf("malloced from %x to %x\n", ptr, ptr+MALLOC_SIZE);
if ((shmid = shmget(0x01010101, SHM_SIZE, IPC_CREAT | 0666)) < 0)
err_sys("shmget error");
if ((shmptr = shmat(shmid, 0, 0)) == (void *) -1) err_sys("shmat error");
printf("shared memory attached from %x to %x\n", shmptr, shmptr+SHM_SIZE);
// if (shmctl(shmid, IPC_RMID, 0) < 0) err_sys("shmctl error");
}
exit(0);
Page 50
example: tshm.c (2/2)
Page 51
Semaphore

a special variable (sv)
 upon which only two operations are allowed

P(sv) : wait



V(sv) : signal





if sv is greater than zero, decrement sv.
If sv is zero, suspend execution of the process
If some other process has been suspended for waiting sv, make it
resume execution
If no process is suspended waiting for sv, increment sv.
They are used to ensure that a single executing process has
exclusive access to a resource.
Critical Section
Binary Semaphore
52
Semaphore
semaphore sv = 1;
loop forever {
P(sv);
critical section code;
V(sv);
non-critical code section
}
53
Semaphores
• A counter to provide access to shared data object for multiple
processes
• To obtain a shared resource:
• 1. Test semaphore that controls the resource
• 2. If value > 0, value--, grant use
• 3. If value == 0, sleep until value > 0
• 4. Release resource, value ++
• Step 1, 2 must be an atomic operation
Page 54
Semaphore Structure
• Each semaphore has a structure
struct semid_ds {
struct ipc_perm sem_perm;
struct sem *sem_base; /*ptr to first semaphore in set */
ushort sem_nsems;
/* # of semaphors in set */
time_t sem_otime;
/* last-semop() time */
time_t sem_ctime;
/* last-change time */
};
struct sem {
ushort semval;
pid_t sempid;
ushort semncnt;
ushort semzcnt;
};
/*
/*
/*
/*
semaphore value, always >= 0 */
pid for last operation */
# processes awaiting semval > currval */
# processes awaiting semval = 0 */
• We can get the structure using semctl() function.
• Actually, however, we don’t need to know the structure in detail.
Page 55
semget()
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/sem.h>
int semget(key_t key, int nsems, int flag);
Returns: semaphore ID if OK, -1 on error
• Obtain a semaphore ID
• nsems: sem_nsens (# of semaphores in set)
• flag: ipc_perm.mode
Page 56
semctl()
APUE (Interprocess Communication
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/sem.h>
int semctl(int semid, int semnum, int cmd, union semun arg);
union semun {
int
val;
/* for SETVAL */
struct semid_ds *buf;
/* for IPC_START and IPC_SET */
ushort
*array; /* for GETALL and SETALL */
};
• To use semaphore, please refer to the textbook and manuals related semaphore.
Page 57
ipcs, ipcrm
APUE (Interprocess Communication
• ipcs: checking status of System V IPC
• $ ipcs
// check information of IPC (q, m, s)
• $ ipcs –q ($ ipcs –qa)
// check information of Message Queue
• $ ipcs –m ($ ipcs –ma)
// check information of Shared Memory
• $ ipcs –s ($ ipcs –sa)
// check information of Semaphore
• ipcrm: delete a defined IPC
• $ ipcrm –q id
// Delete Message Queue
• $ ipcrm –m id
// Delete Shared Memory
• $ ipcrm –s id
// Delete Semaphore
Page 58
Memory mapped file I/O
APUE (Interprocess Communication
#include <sys/types.h>
#include <sys/mman.h>
caddr_t mmap (caddr_t addr, size_t len, int prot, int flag, int filedes, off_t off);
Returns: starting address of mapped region if OK, -1 on error
int munmap(caddr_t addr, size_t len);
Returns : 0 if OK, -1 on error
• Memory mapped I/O lets us map a file on disk into a buffer in memory so that
when we fetch bytes from the buffer, the corresponding bytes of the file are read.
Similarly, when we store data in the buffer, the corresponding bytes are
automatically written to the file. This lets us perform I/O without using read or
write.
• To use this feature we have to tell the kernel to map a given file to a region in
memory.
Page 59
Memory mapped file I/O
flag argument
• MAP_FIXED : the return value must equal addr
• MAP_SHARED : store operation modify the mapped file. Thus shared by others.
• MAP_PRIVATE: make a copy of original file and modify the copied file.
therefore, it does not affect others.
high address
command-line arguments
and environment variables
stack
length
Starting address
Memory-mapped
portion of file
heap
uninitialized data
(bss)
Memory-mapped
portion of file
file :
offset
initialized data
low address
text
60