TDTS06
The labs and x-kernel
Christian Vestlund, christian.vestlund@liu.se
IDA/ADIT, Linköping University
2008-09-03
Me
 christian.vestlund@liu.se
 PhD Student in the security engineering and networks group
 Office hours:
 Monday 13 - 15
 Thursday 13 - 15
 Friday
9 - 10
2
Outline
 The lab series
 What are you supposed to do?
 Lab environment
 x-kernel
 Message flow
 protocols
 sessions
 libraries
 Also, ask questions!
3
Just a short outline of what to expect of this session. Iʼm going to mention the labs and
what the labs are about. Then we are going to dive into the lab environment and
framework and talk about what it does and how it works.
What are the labs about?
 Understanding concepts in networking
 Application programmer’s view:
!"#$%#&'())*%+(,%-#
."+"%/%#&'())*%+(,%-#
2343
2343
."*%(0*"'+1(##"*
4
What we want you to achieve during the labs and the course is to understand
how the protocols and the protocol stack works in relation to the real-world
characteristics of transmission channels. Usually a programmer of an
application imagines the network to be like this. The developer uses some
library to create connections between hosts and the packets arrive at the
receiver using black magic.
This view has some disadvantages even though application programming
usually comes down to this. Juha has already mentioned some of the
application characteristics with regards to network properties so you should
already be aware that applications can be sensitive to network attributes such
as loss and delay. An application programmer utilizing networks needs to have
some understanding on how a network can behave and what can affect an
application.
What are the labs about?
 Understanding concepts in networking
 Characteristics of unreliable channel determines the complexity
of reliable data transfer protocols.
!"#$%#&'())*%+(,%-#
."+"%/%#&'())*%+(,%-#
2343
2343
5$,96"#$:;
2343
."*%(0*"'+1(##"*
."*%(0*"'$(,('
,5(#67"5')5-,-+-*
Your work
%)96"#$:;
)(+8",
5$,9$"*%/"5:;
2343
."*%(0*"'$(,(
,5(#67"5')5-,-+-*
5$,95+/:;
)(+8",
<#5"*%(0*"'''+1(##"*
5
This is however the real situation. You have an unreliable physical channel
which
is used to transfer the data but it drops, delays and distorts packets. On top of
that you often have some sort of reliable transfer protocol which takes care of
problems due to the nature of the unreliable channel and the higher layers
doesn’t have to know about stuff that goes wrong beneath them. And
depending on application requirements and the characteristics of the
transmission channel the complexity of the reliable transfer protocol varies.
In the labs it is your task to ensure the reliability of the transfer protocol by
implementing such functionality!
The lab series
 Lab 1: Handling corrupt packets
 Packet distortion
 Lab 2: NAK-free protocol
 Only ACKs
 Lab 3: Handling lost packets
 Retransmission
 Lab 4: MTU discovery and fragmentation
6
The lab series consist of four labs. Each lab is an incrementation of the previous
lab, i.e. you will add functionality to handle unreliable behavior in the lower
levels of transmission. In the end you will end up with a working and reliable
transfer protocol.
*In the first lab you will have to take care of possible packet distortion by using
ACKs, NAKs and checksums to ensure that a message has arrived at the
receiver.
*In the second lab we will remove the NAKs and the server will only respond
with ACKs, this will require the use of some sort of sequence number
*In the third lab we will add a virtual protocol which drops packets every now
and then, thus you will have to implement retransmission at the sender side.
*And in the last lab we you will have to deal with fragmentation and different
unit sizes.
The first three labs all correspond to a state machine in the textbook which
more or less gives you the solution in terms of what concepts that need to be
implemented. In the rest of this session we will cover how you can implement
them and what you will have to use in terms of x-kernel primitives.
Lab environment
 Solaris
 Refresh your Unix commands
 Setting up the environment
1. $ module initadd /home/TDTS06/modules/tdts06
2. log out
3. log in again
4. $ initTDTS06
 You will get a new directory (TDTS06) with several directories
and files
7
The labs will be done using x-kernel on the solaris systems in the computer
rooms in the B-building. We don’t have any support for running x-kernel on
your own system but you can try if you feel that you have time to spare. If you
haven’t used any unix system before you should read up a little on basic
commands for moving files etc. But you don’t have to worry, if you don’t know
how, just come to the lab sessions or my office hours and i’ll help you out.
To install x-kernel you need to follow these four steps on the slide. The
procedure is on the course web page so you don’t have to remember it now.
When you’ve completed the four steps you will get a TDTS06 directory that
contains several more files and directories in your home directory.
Lab environment
 Important files in /TDTS06
 Makefile (better left alone)
 graph.comp (we will look closer on this one later on)
 Protocol subdirectories
 asp, lab1, lab2, etc.
 Testing subdirectories
 Server (start server here)

rom file for server
 Client (start client here)

rom file for client
 Always run make first when you log in to do the labs, this
updates the real ip addresses in the rom files.
8
There are some files that are of more interest than others. We have the
Makefile, which you don’t need to do anything with. The graph.comp file
describes the network stack in use, we will look closer on that one later on.
The protocol directories asp, lab1, lab2, etc contain the files you will be
working with to implement your solutions. Exactly what you will be changing
where will be in the lab descriptions, and I will show you an example of some
code later. In the labs you will have to copy your files from the previous lab to
the next lab in order to increment on the code.
Lastly we have the testing directories where you will start the client and server,
the rom files in those directories describes the network addresses used and
simulated.
rom files
# The SIMETH line identifies the UDP port that this simulated host
# should use to simulate an ethernet device.
simeth 1234
...
# One of the real port numbers should correspond to the port number in
# the preceding SIMETH line.
#
# simulated
# IP address
Real
IP address
Real
Port number
arp 128.1.2.3
arp 128.1.2.4
130.236.177.13
130.236.177.13
1234
9876
9
The rom files are important to configure correctly, otherwise you will get into
trouble like many students before you.
The simeth line identifies the port number for *this* host, which means the
host in which directory you’re in.
The arp lines then identifies the ports and ip numbers for every other host
simulated.
In this case the first line corresponds to the server and the second to the client.
Thus you can see that the real port number and the simeth number are
identical.
All this is in the setup guide so you don’t have to memorize it, I just want to
highlight the importance of it.
The x-kernel framework
 Object-based framework for implementing, testing and simulate
protocols.
 Lots of code (you don’t have to learn all of it)
 Written in C
 Real-life language for implementing e.g. protocols.
 Fairly high-level language with access to internal hardware
 Learn your pointers
10
So further on to the actual framework. The framework is object-based, not to
be confused with object-oriented. The actual difference is that you always send
along session objects to the other layers instead of calling different layers for a
session object. The framework consists of a lot of code for the different
protocol layers, and you don’t have to learn all of it. But it is a good idea to
look on other example protocols for implementation hints.
x-kernel and the protocols are all written in C. This makes for a quite good
exercise since C is a “quick and dirty” programming language used in real life
for implementing for example, protocols and other fast data handling
functionality.
The most important part for these lab, however, is that you need to know how
to handle pointers.
C pointers
y
int y = 5;
int* yPtr;
5
#42
yPtr = &y;
yPtr
42
#1337
printf(“*yPtr = %d, y = %d”, *yPtr, y);
> *yPtr = 5, y = 5
printf(“yPtr = %d, &y = %d”, yPtr, &y);
> yPtr = 42, &y = 42
11
This is just a short description of C pointers. Suppose we have to variable, one
is an integer y with the value 5 and the other one is an unassigned integer
pointer. Each variable resides on their own address in memory, in this case 42
and 1337. When we assign the yPtr variable the address of the y variable the
yPtr will get the address of y as its value, i.e. it points to that address in
memory.
So if you dereference the yPtr variable and print it out you will print out the
value of the variable it’s pointing to, in this case 5.
And the other way around if you print out the yPtr variable’s value you will get
the address to the variable it’s pointing to.
x-kernel
 Main objects
 Collection of libraries
 Protocols
 Message library
 Sessions
 Event library
 Uniform interface for
protocol interaction
 xPush, xPop, etc.
 Map library
 Trace library
 Utility routines
12
Now we will dig deeper into x-kernel and describe the objects and functions
you will be working with.
The main objects in x-kernel are the protocols and the sessions. You will be
implementing the functionality in one of the protocols in the protocol stack.
The sessions can be regarded as an established connection between a pair of
hosts.
Further on we have a uniform interface for interacting with higher and lower
protocols which have caused confusion in the past. What it means is that you
don’t call specific Push and Pop functions to interact with each layer, you call
the xPush function with a session object which will identify which protocol’s
xPush function you want to call. I will show you some examples of the flow of
function calls in a short while.
Lastly we have the libraries with all the primitives you need to manipulate
messages, schedule
events and debug your solutions. We will look closer into each of these as well.
Flow of messages
 xPush is linked to the lower layer’s push function.
 i.e. if asptest.c calls xPush(...) the aspPush(...) function in asp.c
will be called.
 xDemux is linked to the higher layer’s xDemux function.
 xPop in aspDemux will call aspPop
 aspPop decides whether to call xDemux in the higher layer or
reply by calling xPush.
13
This is just a textual description of the protocol interaction functions. The
three functions you really have to care about are xPush, xPop and xDemux.
xPush is linked to the lower layer’s push function. xDemux is linked to the
higher layer’s demux functions. And lastly, a layer’s demux function will
call the pop function for the same layer and the pop function decides
whether to call xDemux for the next layer to deliver the message or to call
xPush to reply to the message.
Flow of messages
!"#$%"$&'
!"#$%"$&'
()*+%"$
,-)".
"%(4%(5%6),
!"#&'
!"#&' !"#5%6),
!"#-)".
!"#-7#
,-)".
/#&'
/#&'
/#-)".
/#5%6),
/#-7#
,-)".
,5%6),
,5%6),
0%1/!21%3'.!**%1
14
Graphically it can look like this. The runTest function in the test protocol
calls xPush, which invokes the aspPush function in the asp protocol.
aspPush calls xPush to forward the message to a lower layer, which in this
case is the ip layer. The ipPush function will call xPush to push the message
further down the stack. When the message then arrives at the receiving
host the lowest layer will call xDemux to pop it upwards in the protocol
stack. The ipDemux will be called, which calls the ipPop function which in
turn calls xDemux to deliver it up to aspDemux, and so on.
Example - aspPush
static XkHandle aspPush(Sessn self, Msg *msg){
SessnState *sstate = (SessnState *)self->state;
ASPhdr hdr;
char
*buf;
/* create a header by inserting length into header template */
hdr = sstate->hdr;
hdr.ulen = msgLength(msg) + HLEN;
/* attach header to message and pass it on down the stack */
buf = msgPush(msg, HLEN);
aspHdrStore(&hdr, buf, HLEN);
xTraceS3(self, TR_DETAILED,
"PUSH header(sport=%d, dport=%d, ulen=%d)",
hdr.sport, hdr.dport, hdr.ulen);
return xPush(xGetSessnDown(self, 0), msg);
}
15
Lastly I have some code excerpts from the asp protocol’s push and pop
function. The asp protocol does not really implement anything other than
just the absolute basics such as header manipulation. As you can see in the
push function here we get a message from the layer above, we create a
header, prepends the header to the message and the push it further down
the protocol stack. You can see that there are nothing whatsoever added to
the header except for the length of the message, no checksum, data type
or anything. Here is also an example of the trace library’s trace functions.
Example - aspPop
static XkReturn aspPop(Sessn self, Sessn lls, Msg *msg, void *hdr){
ASPhdr *h = (ASPhdr *)hdr;
/* truncate message to length shown in header */
if (h->ulen - HLEN < msgLength(msg))
msgTruncate(msg, (int)h->ulen);
xTraceS1(self, TR_DETAILED,
"Datagram now has size: %d", msgLength(msg));
if (DEBUG) {
print_msg(msg);
printf("\n");
}
/* pass the message to the next protocol up the stack */
return xDemux(xGetUp(self), self, msg);
}
16
The pop function is even simpler than the push function. We get a message
and the corresponding header from aspDemux, the message gets truncated
to remove any trailer still left, and then it gets demuxed to the higher layer.
Here we can see that there is no checking if the message is in order, is
correct or anything like that. It’s just a simple transfer protocol.
Protocols and sessions
 The protocols are statically loaded and initialized
 Sessions are “instances” of protocols which are linked to lower
and/or higher protocol layers.
 Sessions are either opened actively or passively, depending on
the hosts mode (client/server).
*'++)%&+
".%(%$%/#
+(,$0
,+-('+(
,+)-
!"#$%&&'$()%&
17
Each protocol in x-kernel is statically loaded and initialized. Each protocol
keeps a map of the different sessions it currently has. This means that
there can be several connections between different hosts in x-kernel, you
do not have to care about this since we will only be using two hosts and
one connection between them. The sessions can be seen as “instances” of
protocols, each session has a state which keeps track of its connection to
the other host.
Session state
 Each session has a session state struct (asp_internal.h)
 Useful to keep session related attributes in this struct
typedef struct sstate {
ASPhdr hdr;
} SessnState;
18
A session state structure is meant to keep tabs on variables necessary to
handle that session, in here it might be useful to add for example sequence
numbers or session events. The example here is for the asp protocol which
doesn’t really keep track of anything except for the header template.
Protocols and sessions
 Control functions are used to get or set variables in other
protocol layers.
 rdttest.c:
sessnRes = xControlSessn(ps->lls, RDT_SETPKTSIZE,
(char *)&pkt_size, sizeof(pkt_size));
 rdt.c:
case RDT_SETPKTSIZE:
checkLen(len, sizeof(short));
pktsize = *(short *)buf;
if (pktsize > RDT_MAX_PKT || pktsize <= HLEN)
{return -1;}
return sizeof(pktsize);
19
There are also control functions that you can use to set or get variables for
sessions in lower/higher layers. You are going to need this in at least the
last lab to get the MTU from the lower layer. This can look like this code
example.
In rdttest.c you will call the control function for the lower layer with the
argument RDT_SETPKTSIZE. This will invoke the control function in the
lower layer for the case RDT_SETPKTSIZE for session ps->lls which will
write the MTU to the variable value sent down with the function call.
Protocol headers
 Each protocol has its own protocol header (asp_internal.h)
typedef struct header {
ASPport sport; /* source port
*/
ASPport dport; /* destination port */
u_short ulen; /* ASP length
*/
} ASPhdr;
20
Each protocol also has its own protocol header which needs to be added
when the message is passed down the stack. In the header you will need to
add necessary information to ensure the transfer reliability for the
transmission channel in the lab. This might be, for example, the packet’s
sequence number and checksum.
The protocol graph
$cat graph.comp
@;
name=simeth;
name=eth protocols=simeth;
name=arp protocols=eth;
name=vnet protocols=eth,arp;
name=ip
protocols=vnet;
name=icmp protocols=ip;
name=vdelay
protocols=ip;
name=vdrop protocols=vdelay;
name=vdistort protocols=vdrop;
name=rdt files=rdt/rdt protocols=vdistort;
name=rdttest files=rdt/rdttest
protocols=rdt;
@;
21
The graph.comp file describes the current protocol stack in use in xkernel. The two lowest protocols are the ones you will be working with,
rdttest and rdt. You can see that each protocol is linked to another
protocol, this linkage defines the protocol stack. You will need to modify
this file depending on what the lab requires in terms of channel
characteristics.
You can also see three virtual protocols, which are protocols which you will
need to add to the graph.comp file in order to simulate the channel
characteristics. In other words, they are there to make the labs a bit more
interesting.
Virtual protocols
 vdrop, vdistort, vdelay
 Does not add/remove headers or information in a message
 Simulates real-world communication channel characteristics
',-.$,./*
',-.$,./*
#0"1$,.
230,+
,$#4$#5$602
',-/*
',-/* ',-5$602
',-30,+
',-37-
230,+
25$602
859:3
&-/*
&-/*
&-30,+
&-5$602
&-37-
230,+
25$602
!"#$%&'(%$)))*+'""$%
22
The virtual protocols are just to simulate the real-world issues in the
transmission channel. We have three virtual protocols which will be used in
the labs; vdrop, vdistort and vdelay. The protocols does not add/remove
headers or information in a message but for example the vdrop protocol
can remove entire messages from reaching the receiver.
In lab 1 and 2 you will only have to use the vdistort protocol which affects
the order of the bytes in a packet and in lab 3 and 4 you are going to use
all virtual protocols.
Message library
 Creating and manipulating messages
 Optimized to avoid excessive copying
 Examples of message objects; IP datagram, Ethernet frame,
TCP segment, etc.
23
Now we will go into the libraries used to implement your reliable transfer
protocol. The message library will be the one of most interest since this is
mostly about creating and manipulating messages. The message library is
optimized with regards to copying overheads, however we will not go any
deeper into that since it’s quite a complex structure. Typical examples of
real-life messages are IP datagrams, ethernet frames and TCP segments.
Message library - creating messages
 Creating messages
Msg *myMsg;
Msg myMsg;
char *bug;
char *bug;
...
...
myMsg = X_NEW(Msg);
MsgConstructBuffer(&myMs
g, buf, strlen(buf));
MsgConstructBuffer(myMsg,
buf, strlen(buf));
...
...
msgDestroy(&myMsg);
msgDestroy(myMsg);
24
This slide shows two different ways of creating a message. You can probably
spot that the main difference is in the line with X_NEW, which is a memory
allocation macro. Otherwise they are quite alike even though the difference in
declaration, so remember to be wary of allocation and pointers.
Message library - headers
Adding a header
m
DATA
buf
Stripping a header
hdr
m
msgPush(m, HLEN);
aspHdrStore(&buf, m, HLEN);
m
hdr+DATA
hdr+DATA
buf = msgPop(m, HLEN);
m
DATA
buf
hdr
RDThdr h;
aspHdrLoad(&h, buf, HLEN);
25
Other operations are to add and remove headers from a message. The removal
of headers is usually done in the xDemux function which you probably don’t
have to worry about. Adding a header is more important for you. There are two
functions that need to be used for this action; msgPush and aspHdrStore. The
msgPush function allocates memory of size HLEN in front of the message and
returns a pointer to that location. Then you use the HdrStore function to store
the header of length HLEN at location m.
Message library - fragmentation
Fragmenting
m
DATA
Reassembling
m1
DA
+
m2
TA
msgBreak(m, new, 2);
msgJoin(new, m1, m2);
new
DA
+
m
TA
new
DATA
26
Fragmentation will be an important part for the last lab. The actual use of the
fragmentation function is quite easy, you call the msgBreak function with the
message you want to fragment, the empty message new in which you want to
store the fragment and then the size of the fragment. in this example we have
message m which we want to split into chunks of 2 bytes length. Thus we call
msgBreak with m, our empty new message and 2 as arguments and we end up
with two smaller messages.
Reassembling a message is not any more difficult. Call msgJoin with your two
fragments and a new message which will consist of the assembled fragments
and voila, you have a new message with all the fragments.
Also note that the functions takes message pointers as arguments.
Message library
 Fragmentation
 When fragmenting you want to create empty fragments to
assign message fragment to.
Msg* frag_msg;
frag_msg = X_NEW(Msg);
msgConstructEmpty(frag_msg);
-(.%
!&&
!"#$%"$&'
)%""!*%
...
msgBreak(msg, frag_msg, MTU);
!"#&'
+!',%$
(#&'
27
Lastly, when dealing with fragmentation you will have to create empty
messages in which to put your fragments. The only difference here is the
usage of msgConstructEmpty which initializes the empty message. The
fragmentation is important since each layer might have its own predefined
MTU. In the x-kernel framework the test file reads a file and sends
messages down to your protocol. The protocol in turn might have its own
MTU and must divide the message into several smaller packets, then
append a header to each of those and send it down to the next layer.
Event library
 Scheduling protocol timeouts
 evSchedule
 evSchedule(retransmit, (void*)arg, timeout_length)
 The first argument is the function you want to be executed when
the timeout occurs.
 Definition of retransmit function:
static void retransmit(Event ev, void *arg){
//Cast arg to non-void type
...
}
 evCancel
 evCancel(Event ev)
28
Next we have the event library which is a critical part in any reliable
protocol. Since you don’t know whether a message has arrived in some
cases (lost ACKs or lost messages) you might have to resend a message.
Thus you have to schedule your retransmissions using events. The
scheduling of events is done by calling the function evSchedule with the
arguments; the retransmit function that should be called when the timeout
occurs, the argument for the retransmit function and in how long time the
event shall occur. evSchedule returns an event which you want to keep
track of in case you need to cancel the event.
The retransmit function has to be defined with the event as first argument
and then the retransmit arguments as second argument. Also not that the
arguments to the retransmit function must be casted to (void *) and then
cast to a non-void type in the retransmit function.
Cancelling an event is done by calling evCancel with the event as argument.
To be able to cancel an event you must remember to keep track of it using
for example the session state.
Trace library
 Useful when debugging and understanding x-kernel flow
29
The trace library is what you want to use if you need to follow the
execution of the protocols for testing and debugging. There are several
different levels of traces which you can see on the slide. the most useful
ones in my opinion are TR_FULL_TRACE and TR_DETAILED.
Trace library
 Used in graph.comp
@;
name=simeth;
name=eth protocols=simeth;
name=arp protocols=eth;
name=vnet protocols=eth,arp;
name=ip
protocols=vnet;
name=icmp protocols=ip;
name=vdelay
protocols=ip;
name=vdrop protocols=vdelay;
name=vdistort protocols=vdrop;
name=rdt files=rdt/rdt protocols=vdistort trace=TR_DETAILED;
name=rdttest files=rdt/rdttest
protocols=rdt;
@;
30
You activate the traces by adding an argument in the graph.comp file. just
add trace followed by the trace level and run make compose which update
x-kernel with the graph.comp file. This will affect the output when you run
the server and client since it will print out all predefined (and own defined)
traces.
Semaphores
 Each session will need a semaphore to wait
for ACKs.
Semaphore mySem;
 Initialized once when the session is created
semInit(&sstate->mySem, 0);
 A call to semWait will block others from
using the resource (e.g. when you have
done xPush and wait for an ACK).
semWait(&sstate->mySem);
 A call to semSignal will increment the
semaphore’s value and allow the use of
resources again.
semSignal(&sstate->mySem);
semInit(init)
semWait()
//init - 1
semSignal()
//init + 1
31
Semaphores are what you are going to use to be able to wait for
acknowledgements on a sent message. What’s important when you use
semaphores is the initiation value. In your labs you will only have to initiate
the semaphore to 0 because you are only going to have one message inthe-air at any one time. Then, for each time you push a message, you will
call semWait which decrements the value of the semaphore and causes it to
block all other use of this session. When for example an ACK arrives you
will have to increment the semaphore using semSignal in order to allow
continued execution.
Checksum
 Checksum variable is initialized to 0.
 Calculation of checksum is done over the whole message
(message + header).
chk = inCkSum(msg, &hdr, sizeof(ASPhdr));
 Expects an even number of bytes in msg and hdr.
32
In the first lab you will need to implement the use of checksums in order to
handle corrupt packets. To calculate the checksum for a message you need
to call the inCkSum function with the message, header and header size as
arguments. Be careful about the sizes of your messages and headers here.
The checksum function expects both the message and the header to be an
even number of bytes, so you should be smart when choosing your
message and header sizes.
Protocol examples
 Look in /home/TDTS06/xkernel/protocols for examples of
other implemented protocols.
 Very useful to see how different mechanisms are implemented.
 E.g. BLAST for fragmentation
33
For examples of already implemented protocols you can go to the TDTS06
xkernel directory. There are a lot of example protocols which all
implements various mechanisms. I know that I used to look at the CHAN
and BLAST protocols when I took this course.
Debugging
 printf is your friend
 Use the Trace library by adding trace lines in graph.comp
 Plan your development
 Small increments are often better
34
Now we are at the end of this tutorial, just some concluding remarks left.
When you implement your solution you can never print too much to see
how your protocol executes, either by using the trace library or your own
prints. And also, try to plan your development so you keep track of which
parts interacts with what.