cs4411 – Operating Systems Practicum
Supplementary lecture 3
October 14, 2011
Zhiyuan Teo
Today’s lecture
•Administrative Information
•Machine endianness
•Network header generation
•General implementation hints
•Discussion
Administrative Information
•Project 3 updated on Sunday, 9 October.
- Please re-download if you used the initial version we released.
- Project 3 slides and the instructions have been amended.
•Groups may be reshuffled.
•Test cases for project 2 are in the project 3 bundle.
- Test your implementation to make sure preemption and alarms work.
- We will not be grading you on preemption and alarms, but it will be used
later on.
Machine endianness
•Memory is organized as an array of bytes.
•Multibyte data items such as integers and shorts
take up more than 1 byte of storage.
- 4 bytes for an integer.
- 2 bytes for a short.
•Machine endianness refers to the physical ordering
of bytes within the memory for the multibyte data
item.
- Actual ordering of bytes in memory does not change the value stored.
- Ordering of bytes does not change final result of arithmetic shift
operations.
Machine endianness
Big-endian (SPARC, DLX, etc):
32 bitvalue: 0x12345678
12
34
56
78
16 bitvalue: 0xdead
de
ad
Little-endian (Intel, VAX, etc):
32 bitvalue: 0x12345678
78
56
34
12
16 bitvalue: 0xdead
ad
de
•Humans write in big-endian form; it feels more
natural to read.
Which endianness is better?
•Short answer: nobody knows.
•Advantages of big-endian representation:
- Faster when determining an approximation to the stored value.
- Sometimes faster for checking if one value is bigger than another.
- Sign (+/-) can be checked very quickly since it is contained in the 1st byte.
•Advantages of little-endian representation:
- Simpler processor hardware implementation: first byte is 2560, second byte
is 2561, third byte is 2562 and so on.
- Addition of two little-endian values is faster.
Actual value
Big-endian
Little-endian
0x12 34 56 78
0x12 34 56 78
0x78 56 34 12
0x98 76 54 32
0x98 76 54 32
0x32 54 76 98
Endianness problems
•A copies his integer value into a network packet,
byte for byte.
- first byte in memory = first byte into packet.
•B copies the network packet into memory byte for
byte.
•What happens if A and B are of different endianness?
Endianness: Solution
•Big-endian format is the standard for networking.
- Frequently referred to as network byte order.
- Host byte order is the native representation for individual machines; it
could be big or little endian.
•Berkeley sockets API has functions for converting
between host and network byte order.
- htonl(), htons() : host to network long/short.
- ntohl(), nthos() : network to host long/short.
•But we will not be using Berkeley sockets functions!
- Use our own pack and unpack functions.
Network header generation
•Having a common header format will be fun later on.
•Use the pack and unpack functions that we provide.
- Do not use a simple byte copy of the network address, this is incorrect.
- Port numbers may be stored as ints in your program but must be converted
to unsigned shorts when packing.
- Reminder: do not pack port numbers as ints!
•The protocol field will be useful in the next project.
- Set the protocol char field to PROTOCOL_MINIDATAGRAM for this project.
Network header generation – the bad way
•Given the header specs:
- 1 byte protocol type
- 8 bytes source address
- 2 bytes source port
- 8 bytes destination address
- 2 bytes source port
•Allocate a (1+8+2+8+2) byte buffer and manually
fill in the contents.
char* header = (char*) malloc(sizeof(protocol_type) +
sizeof(source_address) +
sizeof(source_port) + ...);
memcpy(header, &protocol_type, sizeof(protocol_type));
memcpy(header + sizeof(protocol_type), &source_address,
sizeof(source_address));
Why is this a bad idea?
•Tedious to code.
- Lots of memcpy() and sizeof() operations.
- Code looks plain ugly.
•What if the header specs change later?
- Must manually change all offsets.
Iteration #2
•Idea: use a struct to store all fields so they are
arranged correctly in memory.
- compiler arranges a contiguous block of memory for the struct.
- memory layout of the struct follows the order declared by the struct.
struct header
{
char protocol_type;
network_address_t source_address;
unsigned short source_port;
network_address_t destination_address;
unsigned short destination_port;
};
struct header hdr;
network_send_pkt((char*) &hdr, sizeof(hdr), ...);
Iteration #2: close but no cigar…
•Padding!
- computers usually load in units of words.
- if a multibyte variable spans 2 words, then 2 loads are needed.
- align the variable to some word boundary so it requires exactly 1 load.
“Variables are aligned to certain power-of-2 numbered
boundaries for faster access.”
- (Getting ready for) Project 1 slides
•Padding is unpredictable and is a waste of resources
to transmit.
Third time’s the charm
•Idea: use a struct that cannot possibly have padding.
- chars require exactly 1 load no matter where they are located.
- Therefore consecutive char fields in a struct are not padded.
- Works regardless of compiler options for padding.
struct header
{
char protocol_type;
char source_address[8];
char source_port[2];
char destination_address[8];
char destination_port[2];
};
struct header hdr;
network_send_pkt((char*) &hdr, sizeof(hdr), ...);
•Use packing functions to convert and populate the char
arrays.
Implementation Hints
•Use an array for your ports.
- O(1) time when using unbound ports (since user specifies the port he wants).
- O(1) time when creating bound ports before a wraparound; O(n) time
afterwards us acceptable (since you need time to check each port).
•Use semaphore_P and semaphore_V for blocking
and unblocking threads.
- Remember how we did this in project 2; consider places where you need to
disable interrupts.
•Reuse your queue implementation.
- This is useful for storing data in FIFO order.
More Implementation Hints
•Perform sanity checks.
- Is the protocol type correct?
- Are you sure the received packet is meant for you?
- Is the packet malformed (header too short, invalid port numbers, etc)?
•Consider semantics for unused ports.
- Data sent to unused ports should not actually be transmitted.
- Data received on an unused port should not be queued.
•Consider reuse semantics for unbound ports.
- When an unbound port is destroyed and later re-created, any prior queued
data should no longer be there.
- Don’t forget to reset the counting semaphore too.
Project 3 FAQ
•Dynamic memory responsibilities.
- network interrupt handler passes you an network_interrupt_arg_t, which
you have to eventually free.
- The user-supplied buffer for both minimsg_send and minimsg_receive
should not be freed by you.
•Occasional lost packets across machines.
- This is normal.
- Try re-executing your program again.
•Unable to communicate between two machines.
- Make sure both machines can ping each other.
- Try running on two machines in the CSUG lab.
- Redrover is known to have problems with machine visibility.
Project 3 FAQ
•Mutexes and semaphores for unbound ports.
- You will need a counting semaphore.
- But technically you won’t need a mutex. (Why?)
struct miniport {
char port_type;
int port_number;
union {
struct {
queue_t incoming_data;
semaphore_t datagrams_ready;
} unbound;
struct {
network_address_t remote_address;
int remote_unbound_port;
} bound;
};
Q&A