Operating Systems (234123) – Spring 2011
(Homework Wet 3)
Homework Wet 3
Due date: Thursday, 26/05/2011, 12:30 noon
Teaching assistant in charge:
•
Michael Kuperstein
E-mails regarding this exercise should be sent only to the following email:
cs234120@cs.technion.ac.il with the subject line: cs234123hw3_wet.
Introduction
In the lectures and recitations you have seen several models of communication between
processes and threads. In this exercise, we introduce an additional model called “messagepassing”. It will be your goal to implement a library to support this model, and use the library to
implement a sample application (a “message router”).
Message Passing
The message passing model is one of the most intuitive models for inter-thread communication.
In this model threads communicate by sending units of information called messages through
queues. When thread A wants to send a piece of information to thread B, thread A encapsulates
the information in a message and adds it to a queue. At some later point in time, thread B
removes the message from the queue, and processes the information. During the time thread A
sends its message and the time thread B decides to receive it, both threads may continue
performing unrelated work.
This model is often used for communication between threads of different processes, or even
between processes on different physical machines. However, in this exercise, you will only need
to implement message passing between threads of the same process. This means you will use
shared memory and POSIX synchronization primitives to implement the message-passing
mechanism.
Message Router
An NxM message router is an application that has N input streams, and M output streams. The
router reads messages from the input streams and writes them to the output streams. For each
received message the router must make a “routing decision”: which of the output streams
should it write the message to.
In this exercise, the input and output streams are all going to be represented by files. Note that
those do not have to be “regular” files – they can be, for example, FIFOs (“named pipes”). The
message router will read messages from the input files, and write them to the output files. The
Operating Systems (234123) – Spring 2011
(Homework Wet 3)
routing decision algorithm for this exercise is very simple: each incoming message is tagged with
the number of the output stream it should go to.
Message Passing Library
Interface
The interface of the library that you will implement is provided in the file mp_interface.h that
can be found on the course website.
In all of the following functions, you may assume the con parameter points to an initialized
context_t structure.
The operations you should implement are:
Initialization
context_t* mp_init();
Description: Allocates and initializes a new “context” structure. The context_t type is declared in
mp_interface.h, as a typedef of struct context_t. It is your responsibility to define the structure
in your code.
The context_t structure is a process-wide structure required for operation of the messagepassing library. mp_init() must be called once by the process (not by each thread) before any
other calls to the library are made. You may assume mp_init() is called by the process at most
once.
Return value: A pointer to the newly allocated context, or NULL on error
int mp_register(context_t* con);
Description: Registers the current thread as a target for messages. This function must be called
by every thread that will send or receive messages, before it can call any other mp functions
except mp_init(). Note that the thread that called mp_init() must also call mp_register() if it
wants to send or receive messages.
Return value: 0 on success, -1 otherwise. In particular, if a thread has already registered, -1
should be returned.
Destruction
void mp_unregister(context_t* con);
Description: Unregisters the current thread as a target for messages. A thread should call this
function before it exits. An attempt to send a message to a thread while it is being deregistered
should fail. You may assume that mp_unregister() is not called while there is a pending
Operating Systems (234123) – Spring 2011
(Homework Wet 3)
mp_send() call to the unregistering thread with the SEND_SYNC flag turned on (the meaning of
SEND_SYNC is explained later).
Return value: None.
void mp_destroy(context_t* con);
Description: Destroys and frees the context structure pointed to by con. After this call, the
process should not use the con context in any other mp call. You may assume mp_destroy() is
not concurrent with any other mp calls.
Return value: None.
Barriers
barrier_t* mp_initbarrier(context_t* con, int N);
Description: Allocates and initializes a new “barrier” structure. The barrier_t type is declared in
mp_interface.h, as a typedef of struct barrier_t. It is your responsibility to define the structure
in your code. The N parameter is an attribute of the barrier, which is used in the mp_barrier()
call.
Return value: A pointer to the newly allocated barrier, or NULL on error.
void mp_destroybarrier(context_t* con, barrier_t* bar);
Description: Destroys and frees the barrier structure pointed to by bar.
Return value: None
int mp_barrier(context_t* con, barrier_t* bar);
Description: Perform a “barrier” operation. An mp_barrier() call blocks until at least N threads
performed an mp_barrier() call on the structure pointed to by bar. Here, N is the parameter
given to the mp_initbarrier() call when bar was initialized. When N threads performed the call,
all threads currently waiting for that barrier are released and should proceed.
The barrier you should implement should be designed to be used exactly once. So, you may
assume a thread calls mp_barrier() at most once for each barrier. An mp_barrier() call for a
barrier that has already been “used” should result in an error.
Return value: 0 if the function returned because N threads called mp_barrier(). Otherwise (on
error), -1.
Sending Messages
int mp_send(context_t* con, pthread_t* target, char* buf, int
len, int flags)
Operating Systems (234123) – Spring 2011
(Homework Wet 3)
Description: Send a message to the thread pointed to by target. The message contents are
pointed to by buf, and the length of the message by len. Messages should be delivered to the
receiving thread in FIFO order, except as specified below. If several threads send messages to
the same thread concurrently, then those messages may be delivered in any order.
The flags field is a bitwise-or of the following values (defined in mp_interface.h):


SEND_URGENT
If the flag is set, then this message should “jump in front” of all messages already sent
to that thread, but not yet delivered, and should be delivered before the waiting
messages. Note that the ordering between urgent messages is LIFO. E.g. if an urgent
message A is sent, but before it is delivered another urgent message B is sent, then B
should be delivered first.
If the flag is not set, the message obeys normal FIFO ordering rules.
SEND_SYNC
If the flag is set, the mp_send() call only returns after the message has been delivered to
the target thread. That is, it may return only after the target thread made an mp_recv()
call which will return this message. Until that happens, the mp_send() call must block.
If the flag is not set, the call returns immediately and does not wait for the message to
be delivered.
Return value: 0 if the message was successfully sent, -1 otherwise.
Broadcasting Messages
int mp_broadcast(context_t* con, char* buf, int len, int flags)
Description: Send a message all registered threads except the thread that called
mp_broadcast(). If the only registered thread is the thread that called mp_broadcast(), the
function should return immediately.
The message contents are pointed to by buf, and the length of the message by len. The flags
parameter has the same meaning as it does for mp_send() with the following modifications:


SEND_URGENT
If the flag is set, the message is considered urgent for all receiving threads.
If not, it is not considered urgent for any of them.
SEND_SYNC
If the flag is set, mp_broadcast() returns only of the message has been delivered to all
receiving threads. Otherwise it returns immediately.
Return value: 0 if the message was successfully sent to all receivers, -1 if sending to at least one
receiver failed. The mp_broadcast() should have “fail-early” semantics. That is, once sending the
Operating Systems (234123) – Spring 2011
(Homework Wet 3)
message to some thread failed for any reason, it should return without trying to send the
message to any further threads.
Receiving Messages
int mp_recv(context_t* con, char* buf, int maxlen, int* msglen,
int flags)
Description: Receive a waiting message. buf points to an allocated buffer into which the
message is written. maxlen is the maximum length of message (in bytes) that may be written
into the buffer. The flags parameter is either 0 or contains the following flag set:

RECV_SYNC
If the flag is set, mp_recv() blocks until there is an available message, and returns only
once a message is available.
If the flag is not set, mp_recv() returns immediately.
If the length of the message that should be returned exceeds maxlen, then mp_recv() should
return without copying the message. The correct return value for this case is described below.
After an mp_recv() call determines it will return a specific message, this message is considered
delivered. So if the message was sent with the SEND_SYNC flag, the mp_send() call that was
used to send the message should be unblocked.
Return value:
If a message was successfully received and written into buf, the return value is 0. After the
function returns, the integer pointed to by msglen should contain the length (in bytes) of the
received message.
If mp_recv() was called without the RECV_SYNC flag, and there are no waiting messages, the
return value is 0, and the integer pointed to by msglen should contain 0.
If a message was not received due to its length exceeding maxlen, the return value is the
required buffer length to receive the message.
In all other error cases, the return value is -1.
Concurrency
Your thread library should be as concurrent as possible. There should be no dependence
between send operations to different threads or receive operations of different threads. In
particular, you may not use a global lock to synchronize all sends and receives. You may use
locks to synchronize operations related to the same thread. However you should acquire as few
locks, and for as little time as possible.
Operating Systems (234123) – Spring 2011
(Homework Wet 3)
Deadlock
If you are not careful when using mp_send()/mp_broadcast() with the SEND_SYNC flag set, it is
very easy to arrive at a deadlock. For instance, if thread A sends a message to thread B, and
thread B sends a message to thread A concurrently, the mp_send() calls of both threads will
block. Since both threads are blocked, neither of the messages will ever be delivered, and the
result is a deadlock. This means that when you are debugging your library, a deadlock is possible
even if the library is implemented correctly, but used incorrectly by your test program.
Message Router
User Interface
Your router executable should be named router, and should accept 5 parameters:
$ ./router <inputnum> <inputprefix> <outputnum> <outputprefix>
<monitor>
e.g.
$ ./router 5 /tmp/in 3 /tmp/out /tmp/monitor
This particular instance of the router will have 5 input streams and 3 output streams. The names
of the input stream files are /tmp/in1, /tmp/in2, …, /tmp/in5 and the output files
/tmp/out1,/tmp/out2,/tmp/out3. The monitor stream file is /tmp/monitor.
You may assume the number of input and output streams is at most 255 each.
Messages
Each input stream consists of a sequence of variable-size messages. A message consists of three
fields and a data payload, in the following order:




Length – 2 bytes, the length of the entire message, including the length field and all
other fields. The legal values for data messages are between 4 and 65535. The length
field is “little-endian”: the first byte of the field is the least-significant byte.
Target – 1 byte, the target output stream of a message, between 1 and 255. If the target
field is the 0, the message should be broadcast to all output streams.
Msgflags – 1 byte. This field is a bitwise-or of the following flags:
o MSG_URGENT: If this flag is set, then the message is considered urgent. The
meaning of an urgent message is the same as in the message library. If the flag
is not set, the message is not considered urgent.
o MSG_FINAL: If this flag is set, the message is considered a final message.
Otherwise it is a normal data message.
Data – the data payload of the message. The length of the data field, in bytes, is length
– 4. Note that this means messages with a zero-length payload are legal.
Operating Systems (234123) – Spring 2011
(Homework Wet 3)
For example, the following is a legal message (bytes are written in hexadecimal, left-to-right):
09 00 05 00 48 45 4C 4C 4F
This message should be parsed as follows:
Field
Raw Data
Meaning
Length
09 00
Target
05
The target is output stream number 5
Msgflags
00
No flags are set
Data
48 45 4C 4C 4F
The total message length is 9 + 0 * 256 = 9 bytes
The data payload is the ASCII representation of the character
sequence HELLO.
Messages should be written into output streams in the same format they are received. The
target field should, however, be replaced with a source field of the same length. This new field
should contain the number of the source stream from which the message came. In addition, a
copy of the message (as written to the output stream) should also be written to the monitor
stream.
You may assume all messages are legal, that is the length field is at least 4 and correctly
describes the length of the message, and the flags field only has the defined flags set. Messages
with the target field set to 0 or to a value above outputnum should be sent only to the monitor
stream.
Final Message
A final message is a data message that has the MSG_FINAL flag set in the msgflags field. Once
the router has received a final message from an input stream it may assume no further message
will be received from that stream. Once final messages have been received from all input
streams, the router should exit. However, before exiting, the router must make sure all
messages it has received (including the final messages) have been written to the output
streams.
Concurrency and Ordering
You should try to make your router as concurrent as possible. This means that:

If reads from one of the input streams block, this should not affect routing of messages
from other input streams.
Operating Systems (234123) – Spring 2011
(Homework Wet 3)

If writes to one of the output streams block, this should not affect routing of messages
to other output streams.
Assuming you have implemented your message passing library correctly, and use it properly,
these requirements should not be hard to meet.
You should try to write messages to the output streams in approximate FIFO order. In particular:


If two non-urgent messages arrived on the same input stream and are directed to the
same output stream, they should be written to the output stream in order.
A solution which receives all messages from the first input, writes them to their
respective outputs, then handles the second input, etc. is not acceptable. Note that
solutions of this type violate the concurrency requirements.
Notes
•
•
•
•
•
•
You must implement this exercise in C. Java/C++/Assembly are prohibited.
You should use only POSIX threads and synchronization functions. However, you may
not use the pthread_barrier_* or pthread_rwlock_* functions, and must implement
the barrier and read-write lock (if you need it) primitives yourself.
You may assume pthread_t is an integer type.
Do not change the provided mp_interface.h header file.
You may not use the poll() and select() system calls or similar mechanisms.
You should try to make your implementation concurrent and efficient. However,
efficiency must not come at the cost of correctness! Therefore, we suggest you first
implement a working version, and only then optimize. The performance of your
solution may affect your grade, but we will not take performance into account for
incorrect submissions.
Operating Systems (234123) – Spring 2011
(Homework Wet 3)
Submission
•
•
•
You submission should consist of two parts: an electronic and a printed submission.
The printed submission should contain the documentation of your design. As part of the
documentation, we expect you to describe the way you used POSIX thread and
synchronization primitives to achieve the goal of maximum concurrency. Please
submit the printed part separately from the dry exercise.
The electronic submission should contain the implementation of your message passing
library and router.
You should create a zip file containing all the files that you have used. Make sure
your zip file contains these files without any directories in it.
1. All source and header files that are part of your implementation.
2. A makefile named "Makefile" that (when using the default target) creates a
library against which we can link our test code, and the router executable.
 The library should be compiled as a static library. The library filename
should be libmp.a. If you want to create a static library to contain
several object files you can do it in the following way:
ar rcs libmp.a obj1.o obj2.o
 The executable should be named router.
3. A file named submitters.txt which includes the ID, name and email of the
participating students. The following format should be used:
Bill Gates bill@t2.technion.ac.il 123456789
Linus Torvalds linus@gmail.com 234567890
Steve Jobs jobs@os_is_best.com 345678901
Important Note: Make the outlined zip structure exactly. In particular, the zip should contain
only regular files (no directories!). You can create the zip by running (inside VMware):
zip final.zip mp_interface.h <your source and header files> submitters.txt
The zip should look as follows:
zipfile -+
|
+- Makefile
|
+- mp_interface.h
|
+- …
|
+- submitters.txt