MSR Tutorial Exercises
Group 3
jdd@arl.wustl.edu
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
Exercise Goals
• Get you familiar with the tools we use for
configuring, testing and analyizing the MSR
• Get you familiar with building and running an
MSR
• Not worry about a lot of MSR internal details
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
2
General Info
• Each group will be using 4 PCs
• Some machines are running Linux, some NetBSD
– All utilities work on both
– You must use NetBSD when building:
• MSR Kernels
• MSR Plugins
• Please pay attention to the diagrams and the hostnames!!!
– pink is Linux, green is NetBSD
• The exercises will start out by giving you very specific
directions.
• Later exercises leave out some details that you should be
able to fill in from what you learned in earlier exercises.
• MSRs are configured with serial cables to SPCs on Ports 4
and 5. We will be doing most of our work with Port 4.
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
3
General Info (continued)
• We are going to be configuring and testing a router
based on the WUGS and SPCs from the hardware
on up.
• There are a LOT of details involved.
• Prior to you doing the exercises we have already
run:
 cd ~/wu_arl/wugs/src
 gmake OSTYPE={Linux,NetBSD} depend
 ~/wu_arl/Make.script install
– these builds and installs the needed utilities
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
4
Group 3
•
User login: msr3
– password: Msr3Tut
– root password, where needed: TutR00T (those are zero’s)
•
Use nmvc1 (Linux) as your base machine
– I.e. log in there and open windows and ssh to other machines
•
MSR CP: nmvc1 (Linux)
– Serial Port for SPC on Port 4, host demand5 (NetBSD):
 tip spc4
– Serial Port for SPC on Port 5, host demand5 (NetBSD):
 tip spc5
•
Switch 1 CP: nmvc2 (NetBSD)
– Data Sources:
• host 1: nmvc2 (NetBSD)
• host 2: gussie (Linux)
•
•
Machine for running Monitoring GUI: nmvc1 (Linux)
Important stuff in ~/.cshrc:
– setenv MSR ~/wu_arl/msr
– setenv WUGSETCPATH ~/wu_arl/wugs/etc
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
5
MSR CP
nmvc1
nmvc2
50-53
SW1 50-53
50-53
CP
P1
P3
50-53
& 2nd 60-63
50-53
Data
70-73
50-53
Source
P2
50-53
50-53
50-53
50-53
P4
P5
50-53
P0
P0
P2
P5
50-53
50-53
50-53
Group 3
P1
P3
50-53
MSR-4
P4
50-53
Data
Switch3
Source 60-63
70-73
gussie
MSR-3
4:demand5 tip spc4
5:demand5 tip spc5
50-53
50-53
P6
50-53
P7 50-53
50-53
P7
P6
50-53
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
6
MSR Lab Part 1
•
•
•
•
•
Boot and Verify MSR (using Debug Kernel)
Generate and Monitor data packets
Use GUI for Traffic Monitoring
Rebuild the existing MSR Kernel
Build a Performance MSR Kernel
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
7

Exercise 1
• Boot and Verify MSR (using Debug Kernel)
– Reset your MSR (Hardware Reset)
• Identify which WUGS switch in the cabinet is your MSR
• Push reset button on front of WUGS
– Open serial connections to two SPCs
• open a window and log in to demand5 as root:
 tip spc4
• open a second window and log in to demand5 as root:
 tip spc5
• for now you should see nothing in these windows…
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
8
Exercise 1 (continued)
• Boot and Verify MSR (using Debug Kernel) (continued)
– Use MSR_Config to boot and configure the MSR
• login in to your MSR CP: nmvc1
 cd ~/wu_arl/msr/cp/configuration
 ./Linux/MSR_Config –f config.MSR
 < this will take a minute or two…>
– Check for booting kernel via serial ports (tip and/or cu)
• make sure it comes to a login prompt
• It occasionally hangs after “init: copying out path ‘/sbin/init’ 11
WHY?
– From nmvc1, use basic cmd protocol commands to check status
• wait for MSR_Config to finish and return to the command prompt.
• look on SPC console windows for output from these commands:
 ~/wu_arl/msr/bin/Linux/sendcmd -p 4 -c apic -s info -t 100000
 ~/wu_arl/msr/bin/Linux/sendcmd -p 4 -c stats -s get_all -t 10000
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
9

Exercise 2
• Configure ATM on your switch CP: nmvc2
– open a window and log in as root on nmvc2
 cd ~msr3/wu_arl/msr/TUTORIAL/GROUP3/cfgs/WUGS
 sh config.apic.nmvc2
 exit
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
10

Exercise 2 (continued)
• Configure connections in switches 1&2
– open a window and log in to your SWITCH CP: nmvc2
 cd ~/wu_arl/msr/TUTORIAL/GROUP3/cfgs/WUGS
 ~/wu_arl/wugs/bin/NetBSD/newGBNSC.init –P3551 –init
init.GBNSC.switch3 config.GBNSC.switch3 &
– this will take a minute or so, be patient…
• you should seem messages like this:
Switch controller for GBN switch 0.2
Controller is READY
Switch is ALIVE
Switch has 8 Ports
Switch has xyxyxyx.ababab HZ ClockRate
CP connected to IPP 1 OPP 1
Control path to the switch via port 1 on VPI/VCI 0/32
Control path from the switch via port 1 on VPI/VCI 0/32
GBNSC is listening on TCP Port 3553
 ~/wu_arl/wugs/bin/NetBSD/Jammer 0.1 nmvc2 3551 group3.js
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
11

Exercise 2 (continued)
• Generate and Monitor data packets
– Read the APIC cell counters on the SPC
• In the window to the Port 4 SPC Console, log in as root (no
password) and try:
 /usr/local/bin/readCounts
• Do you remember what the different ports of the APIC are?
• Remember, the APIC counters are read and clear so if you
repeat the above command the counters should be all zero
 /usr/local/bin/readCounts
– Configure gussie for IP/ATM operation
• As root in a window on gussie:
 sh ~msr1/wu_arl/msr/TUTORIAL/GROUP3/cfgs/IP/gussie.atm
– This will set up VCIs and routes
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
12

Exercise 2 (continued)
• Generate and Monitor data packets (continued)
– Use sendpkts utility to generate packets
• from gussie, as regular user:
 ~/wu_arl/utilities/bin/NetBSD/sendpkts -S -n 10 -a 1 –l 192.168.216.2 192.168.211.2
• Which VCI should these go out on?
– Verify that packets arrive at SPC using readCounts
• In the tip window to the SPC Console:
 /usr/local/bin/readCounts
• You should see Port 0 and Port Bus cells being counted
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
13

Exercise 2 (continued)
• Generate and Monitor data packets (continued)
– Monitor traffic using cmd protocol
• use sendcmd on MSR CP (nmvc1)
 ~/wu_arl/msr/bin/Linux/sendcmd -p 4 -c stats -s get_all -t 10000
• watch for messages on SPC console
• redo sendpkts from gussie and sendcmd from nmvc1
• redirect messages back to MSR CP and to SPC console, then
use monmsgs
 ~/wu_arl/msr/bin/Linux/monmsgs &
 ~/wu_arl/msr/bin/Linux/sendcmd -p 4 -c policy -s set_dflags -d
0x30 -t 10000
• redo sendpkts from gussie and stats sendcmd from nmvc1
– you should see the stats messages on both the SPC console and
in the nmvc1 window where you ran monmsgs.
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
14

Exercise 2 (continued)
• Generate and Monitor data packets (continued)
– IP, which sendpkts uses, and AAL5Generator, which uses APIC
User mode library, cannot coexist on the same VCIs, so we need
to undo the IP/ATM configuration on gussie
– To do this, as root:
 sh ~/wu_arl/msr/TUTORIAL/GROUP3/cfgs/IP/gussie.atm.undo
• This will tear down VCIs and routes
– Now use AAL5Generator to generate traffic from gussie:
 ~/wu_arl/utilities/bin/NetBSD/AAL5Generator -pacer 8 -src 192.168.216.2 frate 1000 -dst 192.168.211.2 -svci 50 -seconds 20 -packet 20
– Monitor traffic using all the previous methods:
• readCounts on the SPC
• sendcmd with messages directed just to SPC console
• monmsgs and sendcmd with messages directed to both CP and SPC console
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
15

Exercise 3
• GUI Traffic Monitoring
– Start newGBNSC on MSR CP (nmvc1)
 cd ~/wu_arl/msr/TUTORIAL/GROUP3/cfgs/WUGS
 ~/wu_arl/wugs/bin/Linux/newGBNSC –P3551 config.GBNSC.MSR
– Verify that newGBNSC.init (or newGBNSC) is still running on
nmvc2
 ps –auxwww | grep GBN
– Start monitoring gui on nmvc1
 cd ~/wu_arl/msr/TUTORIAL/GROUP3/cfgs/GUI/
 /pkg/jdk1.2.2/bin/java –jar ~/wu_arl/msr/apps/javaGUI/pubgui.jar
• when it comes up, use Option: Load from File and load Exercise3.txt
• what are each of the items being monitored:
• IPPBW-r 4: IPP Port bandwidth without recycling for Port 4
• OPPBW-r 0: OPP Port bandwidth without recycling for Port 0
• …
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
16

Exercise 3 (continued)
• GUI Traffic Monitoring (continued)
– Use AAL5Generator to generate traffic
– Move your mouse over the items listed on right of charts to bring
different lines to front
– Try changing options to AAL5Generator to change traffic load
• try different packet sizes
• try different packet rates
-frate #
• try different bit rates
-kbits #
• try different destination addresses
– try each of
» 192.168.203.2, 192.168.207.2, 192.168.211.2, 192.168.215.2,
192.168.219.2, 192.168.223.2 192.168.227.2, 192.168.231.2
• try different values for –seconds (especially, try –seconds 0)
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
17

Exercise 3 (continued)
• GUI Traffic Monitoring (continued)
– Change what you are monitoring
• Try the Options: Show Values on one of the charts
• add different parameters to monitor
– to do this you will have to select a switch to monitor
• remove parameters
• save to a new file
– you can save all the charts as one or each one individually
• quit and reload from new file
– For a specific set of test data (source and destination)
• combine the proper inputs and outputs on one chart
• Try to determine the maximum number of one cell packets that
the MSR can forward. This is a debug kernel, we will try the
same thing later for a performance kernel.
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
18

Exercise 3 (continued)
• GUI Traffic Monitoring (continued)
– Use sendCommand utility to change AAL5Generator’s output
• run your AAL5Generator with –seconds 0 to have it run forever while
you do this
– Try these examples from nmvc2:
 ~/wu_arl/utilities/bin/NetBSD/sendCommand –p 4444 gussie “Packet Rate
10000”
 ~/wu_arl/utilities/bin/NetBSD/sendCommand –p 4444 gussie “Stop”
 ~/wu_arl/utilities/bin/NetBSD/sendCommand –p 4444 gussie “Start”
 ~/wu_arl/utilities/bin/NetBSD/sendCommand –p 4444 gussie “DestAddr
192.168.221.2”
• Changing destination address may cause some dropped packets
– we are changing the IP header on the fly and must update the IP header
checksum as well. We can not update the destination address and the
checksum as an atomic operation so a packet could go out with a bad
checksum.
– Look in ~/wu_arl/utilities/APIC/AAL5Generator/commands.h
• the full list of command verbs
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
19

Exercise 4
• Rebuild the existing MSR Kernel
– On a NetBSD machine, nmvc2:
 cd ~/wu_arl/msr/usr/src/sys/arch/i386/conf
 config MSR_SPC
 cd ../compile/MSR_SPC
 make depend
 make
• As root: Place symbol-only version of kernel in file system, you need
to do this each time you build a new kernel to be downloaded:
 cd ~msr3/wu_arl/msr/TUTORIAL/GROUP3/FS
 vnconfig –t spc24MB –v –c /dev/vnd0d SPC24MB.fs
 mount /dev/vnd0d /mnt
 ~msr3/wu_arl/utilities/bin/NetBSD/mknl \
~msr3/wu_arl/msr/usr/src/sys/arch/i386/compile/MSR_SPC/netbsd \
/mnt/netbsd
 umount /mnt
 vnconfig –u /dev/vnd0d
– Watch for error messages when you do the above.
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
20

Exercise 4 (continued)
• Rebuild the existing MSR Kernel (continued)
– On your MSR CP machine
 cp ~/wu_arl/msr/usr/src/sys/arch/i386/compile/MSR_SPC/netbsd
/usr/MSR/netbsd.MSR
 cp ~/wu_arl/msr/TUTORIAL/GROUP3/ FS/SPC24MB.fs /usr/MSR
• We use /usr/MSR (a local disk, not NFS) so that the downloads
will go faster
– Download and boot
• You may need to kill the newGBNSC if it is still running on
your MSR CP machine
• Check your cu windows. Linux kills them when the other end
is reset.
– Retest with traffic generator and monitoring gui
– How many 1 cell packets can it forward?
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
21

Exercise 5
• Build a Performance Kernel
– We will use a different Kernel config file:
• MSR_SPC_PERF
• Compare it to MSR_SPC. What are the differences?
– Repeat building steps from Exercise 4 but this
time use MSR_SPC_PERF
• While the build is taking place, look at:
– ~/wu_arl/msr/cp/configuration/msrDownload.sh
– This is the shell script that handles the download, you can
change what files it downloads if you need to
– We will continue to refine the configuration and download
process over time.
– Repeat tests from Exercise 4
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
22
MSR Lab Part 2
•
•
•
•
Manually install routes and test
Working with existing Plugins
Modify and rebuild Plugin
Communicating with Plugins via sendcmd
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
23

Exercise 6
• Manually install routes
– Download a debug MSR Kernel and verify
• use traffic generator and GUI traffic monitor to verify SIPL operation
– Turn on FIPL
 ~/wu_arl/msr/bin/Linux/sendcmd –p 4 –c policy –s set_fipl –t 10000
– Install a new route:
 ~/wu_arl/msr/bin/Linux/fip –s –p 4
Enter command (h for help): a 192.168.211.0/24 131
• Do not quit.
• We will use this fip session to build up a set of useful routes.
– Verify FIPL operation
• First using sendpkts and sendcmd to read stats
– don’t forget about needing to configure IP/ATM on gussie
– use destination address of 192.168.211.2
• Then, using traffic generator and GUI traffic monitor
– don’t forget to tear down IP/ATM before trying to use AAL5Generator
– use destination address of 192.168.211.2
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
24

Exercise 6 (continued)
• Build up a set of useful routes
– place fip commands in a file, MSR.IN, something like:
a 192.168.200.0/24 0
a 192.168.201.0/24 1
…
a 192.168.231.0/24 451
q
– load all routes at once:
 ~/wu_arl/msr/bin/Linux/fip –s –q –p 4 < MSR.IN
– verify operation
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
25

Exercise 7
• Load a Plugin
– Download a debug MSR Kernel and verify
• use traffic generator and GUI traffic monitor to verify SIPL operation
– Turn on Debug for plugin and pcu
 ~/wu_arl/msr/bin/Linux/sendcmd …
– Log in to the SPC on Port 4 and load Plugin
• log in as root, no password
 cd /usr/local/rp/plugins/example
 make touch
 make load
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
26

Exercise 7 (continued)
• Instantiate and Test Plugin
– Turn on gate
• ~/wu_arl/msr/bin/Linux/sendcmd –p 4 –c policy –s set_gen –d 1 –t
10000
– Create an instance of the plugin
• ~/wu_arl/msr/bin/Linux/sendcmd –p 4 –c rp_pcu –s create –i 100 –
t 10000
– Add a filter
• ~/wu_arl/msr/bin/Linux/sendcmd –p 4 –c rp_pcu –s addfltr –g 0 –f
0 –F 0 –n 0 –n 0 –d 0 –n 192.168.211.0 –n 255.255.255.0 –d 0 –d 0 –t
10000
– Bind instance to filter
• ~/wu_arl/msr/bin/Linux/sendcmd –p 4 –c rp_pcu –s bind –i 0 –g 0
–f 0 –t 10000
– verify operation using sendpkt
 ~/wu_arl/utilities/bin/NetBSD/sendpkt…
• look for debug messages on SPC console
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
27

Exercise 8
• Modify the Plugin (on a NetBSD machine)
–
–
–
–
–
location: ~/wu_arl/msr/rp/plugins/example
Change the print output in the plugin
Rebuild it on your NetBSD machine
Load the rebuilt plugin into your File System
Download your debug kernel and new File System
• vnconfig and mount your file system
• copy your example/* files into the appropriate place in your
/mnt file system
– Turn on debug, load plugin, instantiate, add filter, …
• just like last exercise…
– verify operation using sendpkt
 sendpkt…
• look for debug messages on SPC console
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
28

Exercise 9
• Communicate with the Plugin
– Use cmd protocol to retrieve plugin packet count
 sendcmd –p 4 –d 0 –c rp_inst –d 5 –d10 –t 10000 –v 5
– Modify plugin to accept other command(s)
• use –d # to differentiate between your plugin “commands”
– Use cmd protocol to send new command(s)
– Modify file system to include new plugin
– Retest by alternating these two commands:
• from nmvc1 (replacing # with a “command” value)
 ~/wu_arl/msr/bin/Linux/sendcmd –p 4 –i 0 –c rp_inst –d # –t 10000 –v 5
• from gussie:
 ~/wu_arl/utilities/bin/NetBSD/sendpkts –S –n 1 –a 1 –l 192.168.216.2
192.168.211.2
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
29
Exercise 10 (Time Permitting)
• Route between MSRs
– Use Fiber connection between two MSRs
• via Port 3
– Set up routes to route between you and other Group
• coordinate with them…
• we’ll need to talk about IP address assignments…
– Verify
• use sendpkt and/or traffic generator.
• Use multiple traffic sources
–
–
–
–
feed data in to MSR on several ports
use AAL5Generator to generate traffic
use gui for monitoring traffic
test input and output side loading
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
30
The End
Now, wasn’t that fun?
John DeHart- 3/22/2016 1:31 AM
Washington
WASHINGTON UNIVERSITY IN ST LOUIS
31