604_Advanced Networking & Traffic Management using NetScaler

604: Advanced Networking & Traffic
Management using NetScaler 10.x
Hands-on Lab Exercise Guide
Contents .................................................................................................................................... 1
Overview .................................................................................................................................... 2
Exercise 1 .................................................................................................................................. 7
Dynamic Routing........................................................................................................................ 7
Exercise 2 .................................................................................................................................13
Net Profiles/Vserver IP Port and Client IP Insertion ...................................................................13
Exercise 3 .................................................................................................................................25
Inbound NAT (INAT)/Listen Policies/ACLs ................................................................................25
Exercise 4 .................................................................................................................................35
Policy Based Routing (PBR)/Link Load Balancing (LLB) ...........................................................35
Exercise 5 .................................................................................................................................43
Route Health Injection ...............................................................................................................43
Exercise 6 .................................................................................................................................47
Token based Load Balancing ....................................................................................................47
Exercise 7 .................................................................................................................................55
Direct Server Return (DSR).......................................................................................................55
Appendix I Acronym Definitions ................................................................................................62
| 1 |
Hands-on Training Module
Citrix NetScaler boasts an impressive set of networking and traffic management features that bring
flexibility and security to any network. In this lab, attendees will gain hands-on experience with
features such as failover interface sets, high availability, independent network configuration, access
control lists, policy-based routes and more.
Familiarity with the NetScaler user and command line interfaces.
Familiarity with Network Routing concepts and Protocols, for example OSPF.
Familiarity with Web Servers e.g. Apache HTTPD server and the HTTP protocol.
Familiarity with UNIX/Linux and the vi editor.
Citrix Partners, Sales Engineers, Consultants, Technical Support.
Lab Environment Details
Describe the lab environment. The system diagram of the lab is shown below:
| 2 |
The Student Desktop is accessed remotely using Citrix Receiver running on your laptop. All
windows applications such as XenCenter, (the XenServer GUI management tool), are accessed
from the Student Desktop.
| 3 |
Lab Guide Conventions
This symbol indicates particular attention must be paid to this step
Special note to offer advice or background information
Text the student enters or an item they select is printed like this
Filename mentioned in text or lines added to files during editing
Bold text indicates reference to a button or object
Focuses attention on a particular part of the screen (R:255 G:20 B:147)
Shows where to click or select an item on a screen shot (R:255 G:102 B:0)
List of Virtual Machines Used
VM Name
IP Address
Description / OS
Windows Domain Controller for Training domain
| 4 |
Apache web server 1 (Red)
Apache web server 2 (Green and Blue)
NetScaler 1
NetScaler 2
First Vyatta Router for 15.x and 16.x subnets
Second Vyatta Router for 17.x and 18.x subnets
Third Vyatta Router for 16.x, 18.x and 19.x subnets
Windows 7 Client
Required Lab Credentials
The credentials required to connect to the environment and complete the lab exercises.
VM Name
IP Address
| 5 |
How to Log into the Lab Environment
Follow the directions below to access the lab environment.
Note: Browser support: IE, Firefox, Chrome, Opera, Safari (on MAC).
Step-by-step login instructions
Launch your web browser and go to http://iltevents.citrixvirtualclassroom.com
On the website, type in the session code provided by your instructor and your
business email address. Click Get Started.
Once you’ve logged in, click the Start Lab button. This will launch a desktop.
Please leave this screen open as you will need these details during the exercises.
Note: Please allow time for the desktop to launch.
| 6 |
Double-click XenCenter on the desktop and then click Add.
On the Add New Server screen enter the XenServer IP address provided on the
website and in the Password field enter the password provided on the site.
Exercise 1
Dynamic Routing
In this exercise we will use the Dynamic Routing feature to inject network routes from routers in the
lab environment running the OSPF protocol into the NetScaler kernel. This functionality uses the
built-in ZebOS router on the NetScaler.
The NetScaler is in an un-configured state so is not aware of any internal or external networks. In
this exercise we will build up the NetScaler configuration to make the appliance aware of an internal
network behind some internal routers and of the external networks surrounding the externally facing
edge routers.
This exercise will be done using the NetScaler CLI.
Step by step guidance
Estimated time to complete this lab: 20 minutes.
Let’s test connectivity internally and externally. Logon to the Win7Client VM. Click the
Switch to Remote Desktop button.
Use the username training\administrator and password Citrix123:
Once in Windows open up the command prompt. Click Start > Run > cmd.
| 7 |
Try to ping the internal Apache web server on
This ping should fail with the message Request timed out. Retry the ping with an
external URL e.g. www.citrix.com
C:\Users\administrator.TRAINING>ping www.citrix.com
You should see a response like this:
Pinging www.gslb.citrix.com [] with 32 bytes of data:
Request timed out
So you can see Windows 7 workstation is not aware of internal or external networks. Let’s
check our interface IP configuration:
Check the default gateway in the IP configuration. What’s this address?
Check the IP table in this guide. It’s the NetScaler SNIP for the network!
We’ll make sure we can reach this address with a ping:
Reply from bytes=32 time=1ms TTL=255
So we’re using the NetScaler as a router. Let’s shift over to the NetScaler and see.
We need to access the NetScaler command line interface (CLI). Open up PuTTY from the
Desktop. Select NS1 from the Saved Sessions list and click Open.
Credentials: nsroot/nsroot
Enter the following command to show the routing table on the NetScaler:
> show route
We can only see one network connected, the network as it is directly
connected to the NetScaler via interface 1/1.
| 8 |
We also have other networks connected:
> show interface
There should be four interfaces: 1/1, 1/2, 1/3 and the loopback adapter. The 1/2 interface is
connected to the network and the 1/3 interface is connected to the network.
Let’s see what IPs we have setup on the NetScaler:
> show ip
Only the NSIP and the SNIP for the network? We will need extra Subnet IPs
for the other networks in the upcoming steps.
We are going to interface with Routers in the environment that are running the OSPF
protocol. Routing protocols are not enabled by default on the NetScaler and are controlled
using features.
Let’s find out if the OSPF Routing feature is enabled. Enter the following command to show
the features and pipe the output via the grep command:
> show feature | grep –i ospf
The OSPF feature is not on by default.
Using the grep utility with the –i switch ignores case sensitivity which is useful on NetScaler
as it is based on FreeBSD UNIX so is a case-sensitive Operating System.
Enable the OSPF Routing feature:
> enable ns feature ospfrouting
Add a Subnet IP for the 192.168.15.x network via interface 1/2:
> add ns ip –type SNIP
10. Enable Dynamic Routing for the newly created SNIP and create a new SNIP for the network via interface 1/3 while enabling dynamic routing in one go!
> set ns ip -dynamicRouting ENABLED
> add ns ip -type SNIP -dynamicRouting
| 9 |
11. ZebOS provides the router functionality on the NetScaler and has its own interface. The
interface is very similar to Cisco iOS. To get access into the ZebOS router functionality on
the NetScaler we issue the command:
> vtysh
12. Enter the ZebOS configuration mode:
ns# configure terminal
13. Enable OSPF routing configuration mode (the number denotes the process id you are
specifying for OSPF):
ns(config)# router ospf 10
14. Redistribute the routes to the NetScaler kernel:
ns(config-router)# redistribute kernel
15. Add the OSPF area 0 and networks from the Vyatta Routers:
ns(config-router)# network area 0
ns(config-router)# network area 0
16. Exit the OSPF routing configuration mode:
ns(config-router)# exit
17. Install the OSPF routes into the NetScaler kernel:
ns(config)# ns route-install ospf
18. Exit to the ZebOS command line:
ns(config)# exit
19. To check we got the previous set of ZebOS commands correct we can issue the command
to output the running configuration:
ns# show running-config
20. Commit the running configuration to disk by issuing the command:
ns# write
21. The NetScaler kernel should populate with routes from OSPF:
ns# exit
> show route
You should be able to see a route of type OSPF for the network.
| 10 |
22. Try to ping the Apache1 server at or Apache2 at from the
> ping
Reply from bytes=32 time=1ms TTL=255
23. Now we will take a look at the ZebOS configuration from the NetScaler CLI:
> shell cat /nsconfig/ZebOS.conf
24. Let’s test SSH access from the Win7Client to the Apache1 server.
Go to the Win7Client desktop and open up a new PuTTY session. Select Apache1 and
click Open.
If the NetScaler has all the correct OSPF routes available in its routing table then it should
connect successfully to the ApacLogon with credentials root/Citrix123.
25. Logon with credentials root/Citrix123.
26. Now that we have routes for our internal network, let’s add the routes from our upstream
router facing externally. The router also is part of the OSPF area 0 and will
have the routes needed to get external access from the lab.
Jump back to NS1 open in PuTTY and enter the following commands in order:
> vtysh
ns# configure terminal
ns(config)# router ospf 10
ns(config-router)# network area 0
ns(config-router)# exit
ns(config)# exit
ns# write
ns# exit
> show route
You should be able to see a route of type OSPF for the and some other bonus
| 11 |
27. Finally let’s ping an external host from the NetScaler:
> ping
Reply from bytes=32 time=25ms TTL=42
Success! The NetScaler is now configured to talk to internal and external networks that
connect to the Internet.
28. Save the configuration on the NetScaler:
> save config
29. To go one further, let’s check from the Win7Client VM that we can also ping externally.
Recall that the Windows 7 is using the NetScaler as its default gateway so therefore any
traffic that needs to route out of the subnet needs to go via the NetScaler. Click
Start > Run > cmd:
C:\Users\administrator.TRAINING>ping www.google.com
Pinging www.google.com [] with 32 bytes of data:
Reply from bytes=32 time=9ms TTL=49
Note: Google.com could respond with a different IP address!
Exercise Summary
In this exercise we:
| 12 |
Learned about the ZebOS router built into the NetScaler.
We configured OSPF Routing to work with 4 different Vyatta routers in the lab environment.
We installed routes learned from OSPF directly into the NetScaler kernel.
We learned about the ZebOS configuration file location and contents.
Exercise 2
Net Profiles/Vserver IP Port and Client IP Insertion
In this exercise we will use a Subnet IP for Apache1 Web Server web traffic and also a separate
SNIP for traffic from the health monitor. This will be achieved using the Net Profiles feature to bind
SNIPs at either the Virtual Server, Service or Monitor levels.
Furthermore we will insert the IP and Port of the Virtual server used along with the Client IP into
HTTP Headers that are sent to the backend Apache Web Server.
This table shows which Subnet IP is used for what purpose and the name we will give the Network
Profile to be created for each.
IP Address
Monitor Traffic
Web Traffic to Apache1
This exercise will be done using the NetScaler UI.
Step by step guidance
Estimated time to complete this lab: 30 minutes.
Open up the Internet Explorer browser. Navigate to the URL for NetScaler1 . If you receive a Security Warning please select Allow.
Use the credentials nsroot/nsroot.
| 13 |
We created a Subnet IP in the previous exercise. We need one more as
per the table above.
Navigate to System > Network > IPs. Click Add. Enter the following details:
IP Address:
IP Type: Subnet IP
Leave everything else default and select Create and Close!
Now we need a network profile for each Subnet IP. The Network Profile will allow us the
granularity of binding a SNIP to monitor, service or virtual server.
Go to System > Network > Net Profiles. Click Add. Enter the name subnet17_snip1,
select IP Address radio button and from the drop-down list.
Click Create. Do not click Close yet, you will need to do this once more!
| 14 |
Enter a new name subnet17_snip2 and select from the drop-down list:
Click Create and Close!
| 15 |
In this step we will create a new HTTP based monitor and bind the first network profile to it
so all monitor traffic will source from the NetScaler on SNIP
Navigate to Traffic Management > Load Balancing.
Now click Monitors, select the existing http monitor in the UI list and click Add. This will
copy the existing settings from the http monitor when adding a new one. Name this
mon_apache1 and select subnet17_snip1 from the Net Profile drop-down list:
Click Create and Close.
| 16 |
In this step we are adding a new service and binding the second network profile to it so all
web traffic will source from the NetScaler on SNIP
Click on Services and click Add. Enter the following details:
Service Name: svc_apache1
Protocol: HTTP
Port: 80
Configured monitor: mon_apache1 (created in the previous step!)
Before you click Create select the Profiles tab and select subnet17_snip2 from the Net
Profile drop-down.
Click Create and Close.
| 17 |
For the final step we need to create the virtual server that includes the newly created
monitor and service. Go to Traffic Management > Load Balancing > Virtual Servers
and click Add.
Enter the following details:
Name: lb_vs_apache
Protocol: HTTP
IP Address:
Port: 80
Services: svc_apache1
Click Create.
From your own laptop; Mac or the LandingVM (you are using this to access XenCenter)
use a browser of choice to navigate to Extra IP1.
This IP and FQDN was presented with in the ILT portal page, an example IP address is or the equivalent full example FQDN is http://75-126-148-171.
Note: You should see a sample Apache page.
| 18 |
There is a screenshot below of the HTTP headers when viewed by the Network Utility in
Firefox (version 23 or higher – accessible using Ctrl+Shift+Q or via menu Tools > Web
Developer > Network) which includes the Server header: ‘Apache’:
So how can we see the monitor and the production traffic being separated? Let’s logon to
the Apache1 box and comb the logs for evidence.
Open up VM Win7Client and double-click PuTTY on the Desktop. Enter
and click Open. Enter the credentials root/Citrix123. Once logged in change to the
apache2 logs directory:
# cd /var/log/apache2
Let’s look for the SNIP for monitor traffic first, which is We will check the
file access_log using the grep utility:
# grep access_log
You should get an output like this which is : - - [25/Nov/2013:16:13:36 +0000] “HEAD / HTTP/1.1” 200 – - - [25/Nov/2013:16:13:41 +0000] “HEAD / HTTP/1.1” 200 – - - [25/Nov/2013:16:13:46 +0000] “HEAD / HTTP/1.1” 200 – - - [25/Nov/2013:16:13:51 +0000] “HEAD / HTTP/1.1” 200 –
The NetScaler monitors the Apache1 web server via the Subnet IP The
monitor issues a HEAD request which should get a successful HTTP 200 response in
order to stay UP.
| 19 |
10. Let’s look at the other SNIP to see what kind of traffic is logged.
# grep access_log
You should get an output like this where we can see the GET requests from the client to
the web server: - - [26/Nov/2013:08:13:36 +0000] “GET / HTTP/1.1” 200 45 - - [26/Nov/2013:08:13:41 +0000] “GET / HTTP/1.1” 200 45
The logs contain some HTTP GET requests for the index.html page. The source is Subnet
IP Monitor traffic is sourced from one SNIP and web traffic from the other
11. So now that we are familiar with the logs on the Apache1 web server, let’s notch it up a
gear! What if we wanted to send the IP and port of the Virtual Server and even the client IP
in a HTTP Request header? This is easy to configure on the NetScaler Virtual server and
From the Win7Client VM, go back to the NetScaler UI again. From Traffic Management
> Load Balancing > Virtual Servers open up the virtual server lb_vs_apache we
created earlier in this exercise. Click on the Advanced tab and enter VServerIP under
Vserver IP Port Insertion with the default VIPADDR drop-down selected:
NOTE: this is case sensitive as we will be using this detail later on to configure logs on our
Apache server, which is hosted on Linux – a case-sensitive OS!
Click OK.
| 20 |
12. Next onto Services and open up the svc_apache1 service. Click on the Advanced tab
and check the Client IP check box, entering ClientIP in the Header text box:
NOTE: Like the last step this is case-sensitive too!
Click OK.
13. Save the configuration on the NetScaler by clicking the Save Icon in the top-right corner
The Apache servers in the environment are based on Gentoo Linux support command
completion using the Tab key in the command line interface.
| 21 |
14. Now that we will be sending the extra HTTP Headers to the Apache web server, we need
to configure Apache logging to capture these headers into a new logfile.
Switch back to the Apache1 session already open in PuTTY.
A configuration file has been provided already on the server. From the command line we
will swap out the older configuration file for the newer one using the following commands:
# cd /etc/apache2/modules.d/
# mv 00_mod_log_config.conf 00_mod_log_config.conf.old
# mv 00_mod_log_config.conf.new 00_mod_log_config.conf
To view the changes made in the logging configuration file, you can use the diff
command to compare the newer vs older files:
# diff 00_mod_log_config.conf 00_mod_log_config.conf.old
< LogFormat "VServer IP %{VServerIP}i" vserverip
< LogFormat "Client IP %{ClientIP}i" clientip
< CustomLog /var/log/apache2/headers_log clientip
< CustomLog /var/log/apache2/headers_log vserverip
The changes made instruct Apache to record all instances of the HTTP Headers
VServerIP and ClientIP to a new file called headers_log.
15. Now that the Apache logging configuration file has been changed, let’s restart the service
to reflect the change. Enter this command:
# /etc/init.d/apache2 restart
| 22 |
16. Let’s see if we have a new log file for our headers. Change directory:
# cd /var/log/apache2
Then do a directory listing (in long list format):
# ls -la
We should see a new file headers_log:
17. So let’s generate some traffic and then check this new log. From your own laptop or Mac,
use a browser of choice to navigate to ExtraIP1 which was presented with in the ILT portal
page. Hit the key-sequence Ctrl+F5 a few times in the browser to refresh more than once.
Switch back to the Apache1 session already open in PuTTY:
Check the headers_log for some new entries:
# tail headers_log
Client IP
Vserver IP
You should see the Client IP and Vserver IP entries with the respective IPs.
| 23 |
Exercise Summary
In this exercise we:
| 24 |
Configured Network Profiles to use different Subnet IPs for production and monitoring traffic.
We configured the Virtual Server to send a HTTP Header including the Virtual Server IP and
port number.
We configured the Service to send the IP of the client machine.
We configured the Apache web server to write a new log file to capture the new headers we
Exercise 3
Inbound NAT (INAT)/Listen Policies/ACLs
In this exercise we will configure Inbound NAT and see why it functionally differs from a virtual
We will also configure both Listen Policies at the virtual server level and ACLs at the global level
which offer similar but different functionality too offering alternate methods to achieve similar
Step by step guidance
Estimated time to complete this lab: 20 minutes.
Logon to NetScaler 1 via PuTTY. Minimize all windows on Win7Client and double-click
the PuTTY icon on the desktop. Select NS1 and click Open and enter credentials
Let’s add a new INAT entry to the NetScaler. The NetScaler will listen on IP and then NAT all traffic to the Apache web server at The
commands are:
> add inat inat_apache
From your Workstation, Windows Laptop or Mac use a browser to navigate to http://yourdashed-fqdn.mycitrixtraining.net (ExtraIP2). An example URL: http://75-126-148172.mycitrixtraining.net/
We can see that INAT offers a simple way to allow external access to an internal
resource. What if we try a different protocol, say HTTPS for instance?
Navigate to https://your-dashed-fqdn.mycitrixtraining.net/home.php (using ExtraIP2
again). You may receive a certificate error in the browser if the CyberTrust Intermediate
and Root certificates are not in the Trusted Root Certificate Authorities store on Windows
or the Keychain on Mac. Please accept the option to accept the certificate and continue
You should see a Citrix colors page.
| 25 |
Switching back to Win7Client VM and into PuTTY to NS1 again we can view details of
our INAT configuration using the show command.
> show inat
1) NAME: inat_apache
Public IP:
Private IP:
Tcpproxy: DISABLED
Proxy IP:
If you would like to see more statistics the nsconmsg utility can be used here. The
nsconmsg utility is used to examine binary based newnslog log files on the NetScaler
appliance. The newnslog files capture all kinds of counters and events happening on
the NetScaler and offer a view of the current status of many thousands of events
happening on the appliance.
Always run nsconmsg with the capital -K switch otherwise the newnslog.ppe.x files
could be overwritten!
| 26 |
The commands are:
> shell
# cd /var/nslog/newnslog
# nsconmsg –K newnslog.ppe.0 –g inat_tot -d current
You should get an output like below:
inat_tot_Hits 4639
Note: If you refresh the PHP info page a couple of times, then switch back to the PuTTY
session you should see these counters increment.
Let’s remove the INAT configuration so we can move forward to the next part of the
exercise. We will reuse the IP address.
Firstly exit the BSD shell to get back to the NSCLI and then remove the INAT
# exit
> rm inat inat_apache
INAT is pretty straightforward. It is pure IP network address translation and has no notion
of ports. How about if we would prefer to use a traditional Vserver instead? Even better,
one that listened on more than one port simultaneously?
We can achieve this using a Virtual Server with TCP protocol and the port number set to
asterisk *. This will listen on all ports so allows any TCP based protocols.
| 27 |
Let’s switch back to the UI on NetScaler 1. Now add a “wildcard” service, go to Traffic
Management > Load Balancing > Services and click Add and the settings:
Service Name: svc_apache_any
Protocol: ANY
Port: *
Click Create and Close.
| 28 |
Go to Traffic Management > Load Balancing > Virtual Servers node and click Add.
Use the following settings:
Name: lb_vs_apache_any
Protocol: ANY
IP Address:
Port: *
Services: svc_apache_any
Don’t click Create yet!
| 29 |
10. Click the Advanced tab and scroll down until you can see Listen Policy. Expand the
Enter the Priority as 100 and click the Configure button. In the Create Expression box
enter the following expression:
Click Create for the expression and then click Create for the virtual server and then
| 30 |
11. From your Workstation, Windows Laptop or Mac use a browser to navigate to:
http://your-dashed-fqdn.mycitrixtraining.net (ExtraIP2).
Then change the URL to https:// and example is https://your-dashedfqdn.mycitrixtraining.net/ home.php (using ExtraIP2 again) - you will get an SSL
certificate error – please continue on. This demonstrates the one virtual server is open
on both HTTP and HTTPS ports.
SSH access is open on our Apache boxes in the environment, so let’s try to SSH to the
virtual server using PuTTY. In the SoftLayer environment we only have NAT rules and
firewall open for HTTP and HTTPS to the IP address So one way to
verify connectivity is to move closer to the virtual server with our tests.
Open up PuTTY from the Desktop and enter the IP and click Open:
You should get an error:
| 31 |
12. Add the SSH TCP port to our Listen Policy and attempt Step 9 again. Switch to the
NetScaler UI and open up lb_vs_apache_any and select the Advanced tab scrolling
down to Listen Policy. Click Configure and modify the existing listen policy expression
to add in port 22 for SSH access:
Click Create and OK on the virtual server.
13. Time to test again from PuTTY. From PuTTY enter the IP protocol as
SSH and click Open:
Success this time!
14. Listen policies allow you to specify listening ports on a virtual server so it is a very
granular solution. What if you wanted to do the same at a global level, which would work
across multiple virtual servers? This where Access Control Lists (ACLs) come in.
First up, let’s remove the listen policy from the Vserver. Switch to the NetScaler UI and
open up lb_vs_apache_any, select the Advanced tab scrolling down to Listen
Policy. Click Configure and delete the existing listen policy priority and expression.
Click OK.
Now the virtual server is listening on every port again. We will approach this from a
different angle and apply Access Control Lists to allow or deny access to TCP ports
based on criteria that we will create.
Extended ACLs allow more options to identify source, destination IPs or ports so suit the
purpose of this exercise. We will apply some extended ACLs to allow TCP ports 80 and
443 to the Vserver at and add an implicit DENY for all other ports.
| 32 |
15. Let’s do this from the NSCLI, open up PuTTY SSH session to NS1. Enter the following
> add ns acl extended_acl_443 ALLOW -destIP = destPort = 443 -protocol TCP -priority 80
> add ns acl extended_acl_80 ALLOW -destIP = destPort = 80 -protocol TCP -priority 90
> add ns acl extended_acl_deny DENY -destIP = protocol TCP -priority 100
ACLs are not applied straight away until we issue the following command:
> apply ns acls
16. SSH traffic should be denied by the ACL list. Check the stats for the Deny ACL:
> stat ns acl extended_acl_deny
Open up PuTTY from the Desktop and enter the IP protocol as SSH and
click Open. Be patient as it may take a couple of minutes to timeout, however you
should not connect and should get the error ‘Network error: Connection timed out’:
17. Check the stats for the Deny ACL again:
> stat ns acl extended_acl_deny
You should see the stats increment for DENY.
18. Jump back to your laptop and test both http and https access to the Apache web page,
both should be successful. Check all the stats in one go from the NSCLI:
> stat ns acl
You should see some stats for ‘Allow ACL hits’ and ‘ACL hits’ increment.
| 33 |
19. Let’s add in an ACL for SSH to see if we can successfully connect.
> add ns acl extended_acl_22 ALLOW -destIP = destPort = 22 -protocol TCP -priority 95
> apply ns acls
Test PuTTY again over SSH to Working?
20. Save the NetScaler configuration:
> save config
Exercise Summary
INAT offers and easy way to allow access to internal resources using a public IP to internal IP
INAT is easy to configure and has no notion of ports.
Listen Policies on a Virtual Server offer a similar feature, however offers the possibility of listening
only on certain ports. Used with an ‘ANY-ANY’ virtual server one IP can be used to allow access to
multiple ports.
ACLs can be more or less granular offering plenty of scope in terms of protocols supported and the
configuration options in terms of ALLOW, DENY access. ACLs can match equal or not equal to a
value of choice. This affords great flexibility in allowing or denying traffic based on your criteria.
| 34 |
Exercise 4
Policy Based Routing (PBR)/Link Load Balancing
In this exercise we will use Policy Based Routing functionality of the NetScaler to route packets not
destined for the NetScaler based on a policy. There is an Apache Web Server in the environment,
having a Vyatta router in front as default gateway. Beyond that gateway working upstream there are
two different paths to the NetScaler. We will leverage two PBR rules to user different paths to the
backend servers dependent on the IP requested.
Expanding this exercise further we will leverage Link Load Balancing to load balance the two Vyatta
routers. Each of these could be viewed as the equivalent of an edge router connected to a WAN
link. As part of that configuration a transparent monitor is needed.
This exercise will be done using the NetScaler CLI.
Below is the topology showing the path for SSH and TCP traffic respectively from the NetScaler to
Apache1 web server.
| 35 |
Step by step guidance
Estimated time to complete this lab: 20 minutes.
| 36 |
Logon to NetScaler 1 via PuTTY. Double-click the PuTTY icon on the Win7Client
desktop. Select NS1 and click Open and enter the credentials nsroot/nsroot.
We would like the NetScaler to forward packets that are not destined for it. To do this
we need to check that Layer 3 mode (IP forwarding) is enabled:
> show ns mode | grep –i l3
L3 mode is enabled by default so you should get the output:
9) Layer 3 mode (ip forwarding) L3 ON
So we’ll add two PBRs. Both will redirect all traffic destined for our Apache web servers
on IPs to 12 to a next hop router that we specify. For SSH traffic we will
use the 192.168.15.x router, for HTTP traffic we will use the 192.168.17.x router.
> add ns pbr pbr_ssh ALLOW –nextHop –destIP = –destPort = 22 –protocol TCP –
priority 80
> add ns pbr pbr_http ALLOW –nextHop –destIP = –destPort = 80 –protocol TCP –
priority 90
Policy based routes are not committed until an ‘Apply’ is issued which commits the
configuration on the NetScaler:
> apply pbrs
Open up a new tab in Internet Explorer and enter the URL
Switch back to the NS1 session in PuTTY and check the PBR stats.
> stat pbr pbr_http
PBR: pbr_http
Rate (/s)
Hits for this PBR
Looking good. The HTTP PBR is getting hits!
| 37 |
Let’s check the logs on the Apache1 box, Click the left-hand corner icon in PuTTY and
select New Session:
Select Apache1 in the Saved Sessions section, click Load and select Open. When
prompted you can logon with the credentials root/Citrix123.
From the command line change to the Apache2 logs directory and get the tail-end of
the access_log file:
# cd /var/log/apache2
# tail access_log - - [16/Dec/2013:13:55:51 +0000] "GET /sugarcrm/
HTTP/1.1" 302 - - [16/Dec/2013:13:55:51 +0000] "GET
/sugarcrm/install.php HTTP/1.1" 200 2758 - - [16/Dec/2013:13:56:56 +0000] "POST
/sugarcrm/install.php HTTP/1.1" 200 10709
Note: All of the connections should issue from one of the SNIP addresses or which are used to connect to the
Switch back to the SSH connection to NetScaler 1 in PuTTY and issue the command:
> stat pbr pbr_ssh
PBR: pbr_ssh
Rate (/s)
Hits for this PBR
| 38 |
To progress with the next part of this exercise we will remove the PBR configuration:
> rm ns pbr pbr_ssh
> rm ns pbr pbr_http
> apply pbrs
In the next steps we move to use Link Load Balancing with Policy Based Routing. When
using a NetScaler to load balance routers or firewalls there needs to be a health monitor
that will check a host upstream of the router or firewall. The way to do this is to specify a
transparent monitor. This type of monitor is based on the HTTP protocol and the
transparent option.
Configuring the PBR we will specify the LLB Virtual Server as the next hop router.
Add transparent monitors:
> add lb monitor mon_trans_apache1 TCP –destip –destPort 80 –transparent YES
Add wildcard services for Vyatta routers 1 and 2:
> add service svc_router1_any any *
> add service svc_router2_any any *
Bind the respective transparent monitors to the services:
> bind service svc_router1_any –monitorName mon_trans_apache1
> bind service svc_router2_any –monitorName mon_trans_apache1
Show the status of one of the added services:
> show service svc_router1_any
Monitor Name: mon_trans_apache1
State: UP
Weight: 1
Passive: 0
Probes: 32
Failed [Total: 0 Current: 0]
Last response: Success - TCP syn+ack received.
Response Time: 0.0 millisec
Add a Link Load Balancing virtual server:
> add lb Vserver lb_vs_routers any
| 39 |
Bind the services to the new virtual server:
> bind lb vserver lb_vs_routers svc_router1_any
> bind lb vserver lb_vs_routers svc_router2_any
Add a default route that specifies the newly created LLB virtual server:
> add lb route lb_vs_routers
You should get an error message:
ERROR: LB method not supported for LLB/PBR
The Load Balancing method specified on the virtual server is not compatible with Link
Load Balancing and Policy Based Routing. The default method is ‘Least Connections’
so let’s change that to a compatible method, Round Robin:
> set lb vserver lb_vs_routers –lbmethod ROUNDROBIN
> add lb route lb_vs_routers
Success will be met with a ‘Done’!
We are going to add a policy based route that will route traffic destined for the Apache1
server on IPs from to
> add ns pbr pbr_routers ALLOW –nextHop lb_vs_routers –destIP = –destPort = 80 –protocol TCP
priority 100
Apply the new PBR rule:
> apply pbrs
| 40 |
You can test the new PBR rule by opening the following URL When the SugarCRM page loads, click Next.
Let’s check the stats for this new PBR rule. Switching back to PuTTY, we can issue the
> show pbr pbr_routers
Name: pbr_routers
Action: ALLOW
Hits: 1295
destIP =
Protocol: TCP
destPort = 80
Active Status: ENABLED
Applied Status: APPLIED
Priority: 100
NextHop: lb_vs_routers
There should be a hit counter for the number of hits this particular PBR rule has
received. We can go one further and check.
| 41 |
We will disable one of the routers to make sure Link Load Balancing is working. Then
we can test again from the client.
In XenCenter right-click on Vyatta-Router2 and select Shut Down.
Test Step 19 again. You should have no problems with connectivity to the SugarCRM
page and the PBR stats should increment for the number of hits.
You can check the status of the LLB Vserver from the NS1 CLI:
> show lb vserver lb_vs_routers
In XenCenter right-click on Vyatta-Router2 and select Start.
Save the NetScaler configuration:
> save config
Back out the PBR/LLB configs before the next exercise:
> rm pbr pbr_routers
> apply pbrs
> rm lb route
> save config
Exercise Summary
In this exercise we:
| 42 |
Created some policy based routes.
Created a Link Load Balancing Virtual Server.
Created a new PBR with LLB as next hop router.
Exercise 5
Route Health Injection
In this exercise we will use the Route Health Injection (RHI) feature of the NetScaler to inject routes
of Virtual Servers on the NetScaler into neighboring routers using the built-in ZebOS router
functionality on the NetScaler.
Both Static and Direct route advertisement is supported.
NOTE: A Virtual IP is needed from a previous exercise. The VIP is created in
Exercise 2.
Step by step guidance
Estimated time to complete this lab: 10 minutes.
Logon to the Win7Client VM. Open up the NetScaler CLI by opening up PuTTY from the
Desktop. Select NS1 to logon onto NetScaler 1 and click Open.
Credentials: nsroot/nsroot
For this feature we require Direct Route Advertisement (DRADV) mode to be enabled on
the NetScaler. Let’s check the modes that are enabled:
> show ns mode | grep –i dradv
Then enable the mode:
> enable ns mode DRADV
Let’s look to see if we have any VIPs created on the NetScaler:
> show ip –type VIP
We should see one VIP available.
Let’s enabled the Host Route feature on this VIP. We need to enter the router we are
going to advertise this VIP to:
> set ns ip –hostRoute ENABLED –hostRtGw
| 43 |
Let’s inspect the routing table on the NetScaler:
> show route
We should see the DIRECT|ADV type beside the network to indicate there
is an advertisement of the host route from the NetScaler.
| 44 |
Let’s inspect the ZebOS routing table by entering vtysh mode and entering the command
to show the routing table:
ns#show ip route
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external
type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, I - Intranet
* - candidate default
C is directly connected, lo0
C is directly connected, vlan0
K via, vlan0
C is directly connected, vlan0
O [110/20] via, vlan0,
C is directly connected, vlan0
O [110/20] via, vlan0,
O [110/30] via, vlan0,
[110/30] via, vlan0,
We can see is advertised as a kernel route.
| 45 |
Let’s look at the routing table on the downstream Vyatta router. Select the VM VyattaRouter2 in XenCenter and select the Console tab. Logon with the credentials
Check the routing table:
$ show ip route
You should see an entry like:
O>* [110/20] via, eth0
via, eth1
Return to the Win7Client VM and to the PuTTY session for Exit the
ZebOS shell, disable the LB Vserver in question and see what effect that has.
> disable lb vserver lb_vs_apache
> vtysh
ns#show ip route
The route should have disappeared from ZebOS which means the route should also be
gone from the Vyatta routing table too.
10. Save the configuration on the NetScaler:
> save config
Before proceeding with the next exercise re-enable the virtual server as we may need it
> enable lb vserver lb_vs_apache
Exercise Summary
In this exercise we configured Direct Route Advertisement mode, altered an existing Virtual IP to
point to the next hop router and advertised the VIP as a route on one of the internal Vyatta routers.
| 46 |
Exercise 6
Token based Load Balancing
In this exercise we will use a token string present in a HTTP request URL to be used as a pseudopersistent load balancing method. The example URL in the exercise is /token.php?token=123.
The token.php page will display a result based on the value of token passed to it. The NetScaler
will leverage the Policy expression engine to glean the value of the token and use that to set
Step by step guidance
Estimated time to complete this lab: 10 minutes.
From the Win7Client VM open up the Internet Explorer browser. Navigate to the URL for
Use the credentials nsroot/nsroot.
Navigate to Traffic Management > Load Balancing and open up the existing
lb_vs_apache virtual server.
Click the Method and Persistence tab and select the Token method from the Method
drop-down list. In the Rule field select Configure and enter the following text:
Before clicking Create we can evaluate how the expression is evaluated by the NetScaler
which will give some clues to how this feature works.
| 47 |
[Optional Step]
Click Evaluate and then from Sample drop-down select GET Request. In the HTTP
Request Data field you need to modify the existing GET request to one similar to what we
will test with. Change the text from:
GET / HTTP/1.1
GET /token.php?token=123
Click Evaluate.
You should get an output like the following that demonstrates that the token value ‘123’ will
be used as a token for the load balancing method. This means that whatever service bound
to the virtual server that is selected to serve the request, will be used for all subsequent
requests where the token is also 123.
| 48 |
Click Close on the Advanced Expression Evaluator (if it’s open), Create on the Expression
and OK on the virtual server.
In Internet Explorer create a new tab and enter the URL
Open up a different browser e.g. Google Chrome and enter the same URL with same
In Internet Explorer create a new tab and enter the URL with a different token
Same result or different remote address?
Open up Google Chrome and enter the same URL with the same second token
What’s the result this time? Token LB working?
Both ‘123’ and ‘321’ are valid tokens defined in the token.php web page. However if you
use a non-valid token such as ‘555’ then the NetScaler will still use this token as a basis for
load balancing due to the fact it matches the expression we created.
| 49 |
Check the LB Vserver stats which will give a breakdown of which service served the
> stat lb vserver lb_vs_apache
Challenge section - Encrypt the token in the URL
If you want to proceed from this point onwards, this is the *Optional* challenge part of the
exercise. The challenge is to encrypt the value of the token in the URL before it is passed
back to the client. The NetScaler is well positioned to achieve this as a TCP proxy and the
power of the policy engine can be leveraged here.
This challenge will involve some vi on Linux, some PHP and copying/pasting PHP code
while testing pages from Windows. Please only proceed further if you are confident with
these components.
| 50 |
Logon to Apache1 VM with credentials root/Citrix123
Change to the Apache2 docs directory:
# cd /var/www/localhost/htdocs
Create a new file in vi with the name test2.php:
# vi test2.php
Change to vi insert mode, press the ‘i’ key. Right-click and paste in the following PHP
<div align="center">
Home page -- displaying value
<form name="Form1" action="token2.php" method="post">
<div><select name="token"">
<option selected="token">Select Option</option>
<option value="1">Show my IP</option>
<option value="2">Show my IP - Base64 encoded</option>
<option value="3">Reload this URL with base64 appended</option>
<button type="submit">Save</button>
Exit insert mode by pressing ‘Esc’ key and type:wq to save and exit.
| 51 |
Create a new file in vi with the name token2.php:
# vi token2.php
Change to insert mode, press the ‘i’ key. Right-click and paste in:
if($var1 == 1)
echo $serverip;
if($var1 == 2)
echo $serverip;
echo "<br/>";
echo $encode;
if($var1 == 3)
header("Location: test2.php?token=$encode");
echo “Not a valid token”;
Exit insert mode by pressing ‘Esc’ key and type:wq to save and exit.
| 52 |
From the LandingVM or Win7Client open up Internet Explorer and enter the URL
On the page there are three options:
1. Show my IP
2. Show my IP and the Base64 encoded version of my IP
3. Reload this page with the Base64 encoded version of my IP appended
Test each to see the varying results.
If we are quite security conscious we might have a concern with a URL containing the
Server IP in Base64 encoding. Due to the fact that encoding is used rather than encryption,
this text can be decoded easily with a Base64 decoder. There are ones online e.g.
To go to the next level we can encode and then encrypt. The encryption can be applied on
the NetScaler and happen on the HTTP Response before the client sees it. This will be
achieved with the Rewrite feature and the magic of the Policy Engine!
Enable the Rewrite feature on the NetScaler:
> enable feature rewrite
Add a Rewrite action. This action looks for a HTTP Header called “Location” and targets
any text after the string “token=”:
> add rewrite action encrypt_token_act replace
Add a Rewrite policy. This policy looks for the HTTP Header “Location”, when there is a
string “test2.php” and it contains “token=”.
> add rewrite policy encrypt_Token_pol
token=\")" encrypt_token_act
Bind the Rewrite policy to the virtual server:
> bind lb vserver lb_vs_apache -policyName Encrypt_Token_pol -type
RESPONSE -priority 100
| 53 |
Test from the client again and select Option number 3. Click
Save. Observe the URL now. It was changed from the encoded value
MTkyLjE2OC4xOS4xMA== to an encrypted one.
For more complex web pages, further rewrite actions would be need to decrypt the token on
HTTP GET requests from the client. See http://support.citrix.com/article/CTX134795
The commands for this particular exercise.
> add rewrite action decrypt_token_act replace
> add rewrite policy decrypt_token_pol
token=\")" decrypt_token_act
Exercise Summary
Token based Load Balancing was configured and seen working in the environment.
[Optional] challenge exercise to create some PHP pages, encode the token in the URL and to
encrypt the token before returning to the client.
| 54 |
Exercise 7
Direct Server Return (DSR)
In this exercise we will configure Direct Server Return. In this topology the normal TCP Proxy model
for the NetScaler is changed in that all return traffic from the backend servers do not traverse the
In order to achieve this configuration a number of components need to be in place. Firstly MAC
Based Forwarding needs to be enabled on the NetScaler. This mode does not consult the routing
table for routing decisions but sends packets based on MAC addresses that are learned on
interfaces instead. Secondly, the service(s) bound to the virtual server need to be set to Use Source
IP (USIP) mode. Thirdly, a loopback adapter needs to be added to each backend server with the
same IP address as the Virtual IP (VIP) of the Load Balanced virtual server.
In this exercise for simplicity we will use one backend server.
NOTE: Remove any existing virtual server using the IP or change the IP to
something not is use, for example before completing this exercise.
Step by step guidance
Estimated time to complete this lab: 15 minutes.
In XenCenter click the Apache2 VM. Select the Networking tab and click Properties, note
the MAC address of interface 0. We will need this detail later in the exercise.
Now select the Console tab and logon with the credentials root/Citrix123
| 55 |
At the command line enter the following configuration lines to setup a loopback adapter on
the Apache2 web server:
# sysctl -w net.ipv4.conf.lo.arp_ignore=1
# sysctl -w net.ipv4.conf.lo.arp_announce=2
# sysctl -w net.ipv4.conf.all.arp_ignore=1
# sysctl -w net.ipv4.conf.all.arp_announce=2
# ifconfig lo:1 netmask -arp up
To make sure the loopback adapter was created, enter the command:
# ifconfig
You should get an output like this:
| 56 |
In this important step we will configure the Apache HTTPD daemon to listen on the new IP
addresses that we assigned to the newly created loopback adapter.
# cd /etc/apache2/vhosts.d/
# vi 00_default_vhost.conf
In VI type /Listen to find the first instance of the string ‘Listen’. The line looks like this:
# Listen
Listen 80
Take note that this is Linux so everything is case-sensitive. Here we will insert some text.
Type ‘i’ to enter insert mode and edit the lines to match the text below:
Exit insert mode by hitting the Esc key. Exit and save with the following command:
Restart the Apache2 daemon to pick up the new settings with this command:
# /etc/init.d/apache2 restart
Jump to Win7Client and the NetScaler CLI. Enabled MAC Based Forwarding mode on the
> enable ns mode MBF
Add a new Load Balanced virtual server. Set the Load Balancing method to SourceIPHash
and ensure the Redirection mode is set to MAC based:
> add lb vserver lb_vs_dsr ANY * -lbMethod
SourceIPHash -m MAC -sessionless enabled
Add a new service for Apache2 server and set the USIP mode:
> add service svc_apache2 ANY * -usip YES
| 57 |
Bind the newly created Apache2 service to the virtual server:
> bind lb vserver lb_vs_dsr svc_apache2
From the Win7Client click Start > All Programs > Wireshark. In Wireshark we are going
to capture some packets from the Windows 7 machine. We will use a Capture Filter to focus
on the traffic for the IP address range we are interested in only. Select the Capture menu
and then Options. Double-click the one Citrix interface listed:
Then under Capture Filter enter net This will only capture
packets from source or destined for IPs in the range -
Click OK and then Start on the Capture Options box. Test from a browser to the URL
| 58 |
Open up a Windows Command Prompt. Click Start > Run and enter cmd. Hit Enter.
From the command prompt, check the ARP table by typing the command arp -a:
In this ARP table we can see a MAC address for the virtual server, which is interface 1/1 on
the NetScaler 06-e0-89-e0-b0-f1 and the MAC address for the Apache2 server interface at
Switch back to Wireshark. Stop the capture by clicking the Stop button
the Capture menu and selecting Stop.
or by clicking
Enter the following into the Display filter bar to filter the traffic and only show HTTP GET
http.request.method eq GET
There should be a frame that in the info bar has GET /php/test.php HTTP/1.1:
Select this frame, right-click and select Follow TCP Stream. Select Close as we are not
interested in the HTML, we want to get into the innards and inspect the MAC addresses!
| 59 |
NOTE: Your Display Filter may not necessarily match the screenshot below in terms of
tcp.stream number.
Select the first frame in the list and then expand Ethernet II in the middle bar:
This is the SYN in the TCP handshake where the destination MAC is 06:e0:89:e0:b0:f1.
This is the MAC address for interface 1/1 on NS1. The source MAC is the Win7Client
In the next frame, we can see a SYN-ACK. Check Ethernet II for this frame:
Checking the MAC addresses we should see the source MAC. Check the Apache2 MAC
address we noted in Step 1 earlier. Does it match? It should.
What does this tell us? The NetScaler in a standard load balancing scenario operates as a
TCP Proxy, setting up a client-side connection and a new server-side connection. In that
scenario all the client-side conversations will be between the client and the virtual server.
The MAC addresses seen will be for the client interface and the NetScaler interface.
However as per above we see above Direct Server Return is working.
| 60 |
Following the MAC addresses we see the path Win7Client > NetScaler > Apache2 >
Win7Client. In a non-DSR LB scenario this would be Win7Client > NetScaler > Apache2 >
NetScaler > Win7Client.
Save the configuration:
> save config
To continue on to one of the previous exercises from this one, you will need to disable Mac
Based Forwarding, remove the Virtual Server and disable USIP on the service. The
commands are:
> disable ns mode MBF
> rm lb vserver lb_vs_dsr
> set service svc_apache2 –usip NO
> save config
Exercise Summary
An ANY protocol wildcard port load balanced virtual server was created. This vserver had the
redirection mode set to MAC, the lbmethod set to SOURCEIPHASH and the Sessionless option
A service was added, also ANY type with wildcard port. This needs Use Source IP (USIP) enabled.
To support the configuration, MAC Based Forwarding mode needs to be enabled globally on the
On the Apache Web Server a loopback adapter was created listening on the same IP as the Virtual
IP on the NetScaler. This loopback adapter is set to be non-ARPing.
With this configuration, the NetScaler is not consulting the routing table to assess the destination of
packets. It remembers the interfaces it saw particular MAC addresses on.
Please complete this survey
We value your feedback! Please take a moment to let us know about your training
experience by completing the brief Instructor-led Learning Lab Survey.
| 61 |
Appendix I Acronym Definitions
| 62 |
Access Control List
Command Line Interface
Domain Name System
Fully Qualified Domain Name
Graphical User Interface
High Availability
Hypertext Transport Protocol
Inbound NAT
Internet Protocol
Load Balancing
Network Address Translation
NetScaler IP
Open Shortest Path First
Policy Based Routing
Route Health Injection
Reverse NAT
Subnet IP
Secure Shell
Transport Control Protocol
User Datagram Protocol
User Interface
Virtual IP
Change Description
Updated By
Original version
Andrew Sandford
January 2014
About Citrix
Citrix (NASDAQ:CTXS) is a cloud company that enables mobile workstyles—empowering people to
work and collaborate from anywhere, securely accessing apps and data on any of the latest
devices, as easily as they would in their own office. Citrix solutions help IT and service providers
build clouds, leveraging virtualization and networking technologies to deliver high-performance,
elastic and cost-effective cloud services. With market-leading cloud solutions for mobility, desktop
virtualization, networking, cloud platforms, collaboration and data sharing, Citrix helps organizations
of all sizes achieve the speed and agility necessary to succeed in a mobile and dynamic world.
Citrix products are in use at more than 330,000 organizations and by over 100 million users
globally. Annual revenue in 2012 was $2.59 billion. Learn more at www.citrix.com.
| 63 |