FIREWALLS & NETWORK SECURITY with
Intrusion Detection and VPNs, 2nd ed.
11
Setting Up a Virtual
Private Network
By Whitman, Mattord, & Austin
© 2008 Course Technology
Learning Objectives
 Explain the components and essential operations
of virtual private networks (VPNs)
 Describe the different types of VPNs
 Create VPN setups, such as mesh or hub-andspoke configurations
 Choose the right tunneling protocol for your VPN
 Enable secure remote access for individual users
via a VPN
 Recommend best practices for effective
configuration and maintenance of VPNs
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 2
Introduction
 Organizations routinely join LANs to facilitate
secure point-to-point communications
 Private leased lines don’t scale well, utilize
complex technology, and are expensive
 VPNs function like private leased lines
– Encapsulate and encrypt data being transmitted
– Use authentication to ensure only approved
users gain access
 VPNs provide secure point-to-point
communications over public Internet
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 3
VPN Components and Operations
 VPNs can be set up with special hardware or
with firewall software that includes VPN
functionality
 Many firewalls have VPN systems built in
 Correctly set up VPN can be a critical
component in an organization’s perimeter
security configuration
 Goal of VPNs is to provide a cost-effective and
secure way to connect business locations to
one another and remote workers to office
networks
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 4
VPN Components
 VPNs consist of two types of components:
– Hardware devices
– Software that performs security-related activities
 VPN tunnels have two endpoints or terminators
 Endpoints:
– Hardware devices or software modules
– Encrypt data to secure information
– Authenticate to ensure host requesting data is an
approved user
– Encapsulate data to protect integrity of
information being sent
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 5
VPN Components (continued)
 VPN connection occurs within TCP/IP tunnel
 Tunnel: channel or pathway of networks used
by VPN that runs through the Internet from one
endpoint to another
 “Tunnel” can be misleading as it implies:
– There is a single cable joining endpoints
– Only approved VPN users can utilize that cable
 In reality, VPN “tunnel” is virtual
 Using the Internet keeps costs down and
simplifies setup of VPN but can also add
uncertainty to communications
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 6
VPN Components (continued)
 Endpoint devices can be one of the following:
– A server running a tunneling protocol
– A VPN appliance (a special hardware device
devoted to setting up VPN communications)
– A firewall/VPN combination
– A router-based VPN (routers that support IPSec
can be set up on perimeter of connected LANs)
 VPN scenario may also include:
– Certificate servers: manage certificates
– Client computers: run VPN client software,
allowing remote users LAN access over the VPN
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 7
Essential Activities of VPNs
 Information transferred via VPN travels over the
Internet and must be well protected
 Essential activities that protect data are:
– IP encapsulation
– Data payload encryption
– Encrypted authentication
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 8
IP Encapsulation
 Used to protect VPN data packets
 Process of enclosing one packet within another
packet that has different IP source and
destination information
 Hides source and destination information of
encapsulated packets
 IP addresses of encapsulated packets can be in
the private reserved blocks that are not usually
routable over the Internet
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 9
Data Payload Encryption
 VPNs can be configured to fully or partially
encrypt data portion of packets
 Encryption accomplished in one of two ways:
– Transport method: host encrypts traffic when it is
generated; data is encrypted, but not headers
– Tunnel method: traffic encrypted and decrypted
in transit; both header and data portions of
packets are encrypted
 Level of encryption varies
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 10
Encrypted Authentication
 Encryption domain: everything in the protected
network and behind the gateway
 Authentication essential; VPN communication
recipients must know sender is approved user
 Hosts authenticated by exchanging keys
 Two types of keys:
– Symmetric keys: keys are the same; hosts
exchange same secret key to verify identities
– Asymmetric keys: participants have private key
and public key; public keys exchanged; public
key used to encrypt; decrypt using private key
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 11
Benefits and Drawbacks of VPNs
 Benefits:
– Secure networking without costly leased lines
– Encryption/translation handled by dedicated
systems, reducing production machine workload
– Allows control of physical setup
 Drawbacks:
– Complex and, if configured improperly, can
create significant network vulnerabilities
– Uses unpredictable and often unreliable Internet
– Some vendor solutions have more documented
security issues than others
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 12
VPNs Extend Network Boundaries
 VPN connections that are “always on” extend
your network to locations out of your control
 Some suggestions for dealing with increased
risk presented by these connections:
– Use of two or more authentication tools to
identify remote users
– Integrate virus protection
– Use Network Access Control (NAC)
– Set usage limits
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 13
Types of VPNs
 In general, you can set up two types of VPN:
– Site-to-site: links two or more networks
– Client-to-site: makes a network accessible to
remote users who need dial-in access
 These two VPN types are not mutually exclusive
 Options for configuring VPNs:
– Hardware systems
– Software systems
– Hybrids
 VPNs need to be able to work with any number
of different operating systems or computer types
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 14
VPN Appliances
 Hardware device specially designed to
terminate VPNs and join multiple LANs
 Can permit connections between large numbers
of users or multiple networks
 Don’t provide other services such as file sharing
and printing
 Some examples include the SonicWALL series
and the Symantec Firewall/VPN appliance
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 15
Software VPN Systems
 Generally less expensive than hardware
systems
 Tend to scale better on fast-growing networks
 Some examples include F-Secure VPN+ and
Novell’s BorderManager VPN services
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 16
VPN Combinations of Hardware and
Software
 VPN systems may implement VPN appliance at
the central network and use client software at
remote end of each VPN connection
 Most VPN concentrator appliances are capable
of operating in one of two modes:
– Client mode: concentrator acts as software client,
enabling users to connect to other remote
networks via VPN
– Network extension mode: concentrator acts as
hardware device enabling secure site-to-site
VPN connection
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 17
Combination VPNs
 VPN system that is “mixed” uses hardware and
software from different vendors
 Challenge: get all pieces of the system to
communicate with one another successfully
 Solution: pick a standard security protocol that
is widely used and supported by all devices,
such as IPSec
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 18
VPN Setups
 With two participants in a VPN, configuration is
relatively straightforward in terms of:
– Expense
– Technical difficulty
– Time involved
 When three or more networks/individuals are
connected, several configuration options exist:
– Mesh
– Hub-and-spoke
– Hybrid
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 19
Mesh Configuration
 Each participant (network, router, or computer)
in the VPN has an approved relationship, called
a security association (SA), with every other
participant
 During VPN configuration, each participant must
be specifically identified to every other
participant using the VPN
 Before initiating connection, each VPN
terminator checks its routing table or SA table to
confirm the other participant has an SA with it
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 20
Mesh VPN
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 21
Hub-and-Spoke Configuration
 A single VPN router contains records of all SAs
in the VPN
 Any LANs or computers participating in VPN
need only connect to central server, not to any
other machines in VPN
 Easy to increase the size of VPN as more
branch offices or computers are added
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 22
Hub-and-Spoke VPN
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 23
Hybrid Configuration
 As organizations grow, mesh or hub-and-spoke
VPN designs commonly evolve into a mixture of
the two
 Mesh configurations tend to be more efficient;
central core linking most important network
branches should be mesh configuration; other
branch offices added as spokes connecting to
VPN router at central office
 Hybrid setup benefits from strengths of each
one—scalability of hub-and-spoke and speed of
mesh
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 24
Configurations and Extranet and
Intranet Access
 Each VPN endpoint represents extension of
corporate network to new location—an extranet
 Same security measures taken to protect
corporate network should be applied to VPN
endpoints (firewalls, anti-virus, etc.)
 VPNs can also be used to give parts of
organization access to other areas through
corporate intranet
 VPN users inside organization should have
usage limits, anti-virus, and firewall protection,
just as outside users should
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 25
Tunneling Protocols Used with VPNs
 In the past, firewalls providing establishment of
VPNs used proprietary protocols
 Such firewalls could only establish connections
with remote LANs using same firewall brand
 Today, widespread acceptance of IPSec
protocol with Internet Key Exchange (IKE)
system means proprietary protocols are used
far less often
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 26
IPSec/IKE
 IPSec provides two security methods:
– Authenticated Header (AH): authenticates
packets
– Encapsulating Security Payload (ESP): encrypts
data portion of packets
 IPSec can work in two different modes:
– Transport mode: provides secure
communications between hosts
– Tunnel mode: used to create secure links
between two private networks
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 27
IPSec/IKE (continued)
 IPSec/IKE VPN connection process:
– 1. Request to establish a connection sent
– 2. Remote host generates random number and
sends to machine that made original request
– 3. Original machine encrypts its pre-shared key
using random number and sends to remote host
– 4. Remote host decrypts key, compares it to its
own pre-shared key or keyring; if key matches,
remote host encrypts public key using pre-shared
key and sends to original machine
– 5. Original machine uses public key to establish
security association (SA) and VPN connection
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 28
PPTP
 Point-to-Point Tunneling Protocol (PPTP)
 Commonly used to connect to a network using a
dial-in modem connection
 Uses Microsoft Point-to-Point Encryption
(MPPE) to encrypt data
 Useful if support for older clients is needed
 Also useful because packets sent can pass
through firewalls that perform Network Address
Translation (NAT)
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 29
L2TP
 Layer 2 Tunneling Protocol (L2TP)
 Extension of Point-to-Point Protocol (PPP)
 Uses IPSec rather than MPPE to encrypt data
 Provides secure authenticated remote access
by separating connection initiation process from
encapsulated data forwarding process
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 30
PPP Over SSL/PPP Over SSH
 Point-to-Point Protocol (PPP) Over Secure
Sockets Layer (SSL) and Point-to-Point Protocol
(PPP) Over Secure Shell (SSH)
– UNIX-based methods for creating VPNs
– Combine existing tunnel system (PPP) with way
of encrypting data in transport (SSL or SSH)
 SSL: public key encryption system used to
provide secure communications over WWW
 SSH: UNIX secure shell; performs secure
authenticated logons and encrypted
communications; requires pre-shared key
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 31
VPN Protocols and Their Uses
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 32
Enabling Remote Access Connections
within VPNs
 To enable remote user to connect to VPN, user
must be issued VPN client software
 User’s computer should be equipped with a
firewall and anti-virus software
 Key may need to be obtained for remote user if
IPSec is used to make VPN connection
 Problems may be encountered finding phone
provider having dial-up numbers in all locations
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 33
Configuring the Server
 If firewall-based VPN is used, client computer
must be identified
 Check Point FireWall-1 calls the process
defining a network object
 Major operating systems incorporate their own
methods of providing secure remote access
 Linux uses IP Masquerade feature
 Windows XP and 2000 include New Connection
Wizard
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 34
Configuring Clients
 Involves installing and configuring VPN client
software or using New Connection Wizard
 FireWall-1 uses SecuRemote that enables
connections to hosts or networks via VPN
 Important issues to consider:
– Will client software work with all client platforms
– Is client workstation itself firewall protected
 Because each VPN connection is potential
opening for viruses and hackers, requirement
that remote hosts be protected with firewalls
should be part of organization’s VPN policy
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 35
VPN Best Practices
 Successful operation of VPN depends not only
on hardware and software components and
overall configuration
 Also depends on a number of best practices
 These include:
– Security policy rules specific to the VPN
– Integration of firewall packet filtering with VPN
traffic
– Auditing VPN to ensure acceptable performance
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 36
The Need for a VPN Policy
 Essential for identifying who can use the VPN
and for ensuring all users know what constitutes
proper use
 Can be a separate stand-alone policy or part of
a larger security policy
 Points to cover include but are not limited to:
–
–
–
–
–
Who is permitted to have VPN access
Whether authentication is to be used and how
Whether split tunneling is permitted
How long users can be connected in one session
Whether virus protection is included
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 37
Packet Filtering and VPNs
 Decision must be made early as to where data
encryption and decryption will be performed in
relation to packet filtering
 Encryption and decryption can occur either
inside or outside the packet-filtering perimeter
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 38
PPTP Filters
 PPTP commonly used when older clients need
to connect to a network through a VPN or when
a tunnel must pass through a firewall that
performs NAT
 For PPTP traffic to pass through a firewall,
packet-filtering rules must permit such
communications
 Incoming PPTP connections on TCP Port 1723
 PPTP packets use Generic Routing
Encapsulating (GRE) packets identified by
protocol identification number ID 47
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 39
L2TP and IPSec Packet-Filtering Rules
 L2TP uses IPSec to encrypt traffic as it passes
through the firewall
 Packet-filtering rules must be set up that cover
IPSec traffic
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 40
Auditing and Testing the VPN
 Each VPN computer client should be tested
 VPN should be checked to ensure component
reliability and acceptable file transfer rates
 If parts of network frequently fail, switch ISPs
 If ISP switch is needed, consider the following:
– How often does network go offline?
– Are there backup servers to keep customers
online if primary server goes down?
– Are there backup power supplies in case of a
power outage?
– How far is the network backbone?
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 41
Chapter Summary
 VPNs:
– Provide secure point-to-point communications
over the public Internet
– Used for e-commerce and telecommuting
– Can be set up with special hardware or with
firewall software that includes VPN functionality
– Are a critical component in an organization’s
perimeter security configuration
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 42
Chapter Summary (continued)
 VPN data travels over public networks and
needs to be well protected
 Essential data protection activities:
– IP encapsulation
– Data payload encryption
– Encrypted authentication
 Two different types of VPN:
– Site-to-site
– Client-to-site
 The two are not necessarily mutually exclusive
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 43
Chapter Summary (continued)
 VPN configurations:
– Mesh configuration: each participant has an
approved relationship with every other participant
– Hub-and-spoke arrangement: single, central VPN
router contains records of all associations; any
other participants connect only to central server
– Hybrid setup: mixture that often evolves from the
other configuration types as organization grows
 Widespread use of IPSec with Internet Key
Exchange (IKE) means proprietary protocols
used far less often
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 44
Chapter Summary (continued)
 IPSec provides two security methods:
– Authenticated Header (AH): authenticates
packets
– Encapsulating Security Payload (ESP): encrypts
the data portion of packets
 Both methods can be used together
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 45
Chapter Summary (continued)
 Point-to-Point Tunneling Protocol (PPTP) used
to connect to network using dial-in modem
 Layer 2 Tunneling Protocol (L2TP) extension of
protocol long used for dial-up connections on
the Internet, Point-to-Point Protocol (PPP)
 Point-to-Point Protocol (PPP) Over Secure
Sockets Layer (SSL) and Point-to-Point Protocol
(PPP) Over Secure Shell (SSH)
– UNIX-based methods for creating VPNs
– Combine existing tunnel system (PPP) with data
encryption in transport (SSL or SSH)
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 46
Chapter Summary (continued)
 To enable remote user to connect to a VPN,
issue that user VPN client software
 Make sure user’s computer has anti-virus
software and a firewall
 May need to obtain key for remote user if using
IPSec to make VPN connection
 VPN best practices include:
– Security policy rules specific to the VPN
– Integration of firewall packet filtering and VPN
traffic
– Auditing VPN to ensure acceptable performance
Firewalls & Network Security, 2nd ed. - Chapter 11
Slide 47