Microsoft Challenge Authentication Handshake
Protocol (MS-CHAP)
in Virtual Private Network (VPN)
CS265 Cryptography
Dr. Stamp
ChungShun Wei
002923077
Mar. 24, 2005
1. Introduction
After the telephone has been invented, one can just dial numbers on the telephone and
talk to another who may be living on a different continent from the caller. This may seem
to be an evolutionary invention 60 years ago. However, today, the Internet can perform
an even powerful communication channel between two end points worldwide. Not only
the Internet can deliver voice, but it can also transmit all formats of data, such as image,
text, and video. The Internet provides tremendous convenience for people to
communicate with each other around the world. The Internet also allows businessman to
join a conference meeting or to access company’s private networks without physically
being in the company office. The technology behind this private networking access is
called Virtual Private Network (VPN) which can authenticate remote users and encrypt
the data traveling through the channel. VPN technology utilizes the Internet to carry data
packets from the source point to the destination host. This seems to be highly insecure
when company employees may send confidential information through the public internet.
This paper will discuss and analyze Microsoft Challenge Handshake Authentication
Protocol (MS-CHAP) which is the authentication protocol that Microsoft Windows uses
to protect the data traveling between remote users and VPN servers.
2. Virtual Private Network (VPN)
Virtual Private Network (VPN) can be seen as a private network that is restricted to
outside access for security and confidentiality reasons. VPN is often being deployed by
corporations to establish company private networks and allow remote employees who are
not physically sitting in local company offices to access the company’s private network
and logically equal to be connecting to the company’s private network locally. VPN is
also used when two branches of the same company wishes to share information. They can
use VPN tunneling through Wide Area Network (WAN) and form a logically equivalent
private Local Area Network (LAN) connection between these two branches shown in
Figure 2.1.
If a remote employee/client wants to connect to a VPN. There are two methods for the
client to initiate the connection. The first method is through a modem on the client
machine. The client can dial a number to a Remote Access Server (VPN server) that will
generate a dedicated phone line connection from the client machine to the company’s
private network. Then, the data will be sent through this dedicated phone line. This is
usually called Remote Access Service. The disadvantage of this method is that the client
sometimes will require dialing a long-distance phone call to get to the VPN server. Also,
each client that is connected to the VPN server requires a direct phone line connection.
This method is too costly and resource-consuming. The second method is to use a current
existing internet service from Internet Service Provider (ISP) and connect the client’s
machine to the company’s VPN shown in Figure 2.2. The data are no long traversing
through a phone line; instead, they are routed through the Internet to the VPN server. The
advantage of this method is that clients can avoid costly long-distance phone calls. Also,
there is no dedicated phone line required for each client who connects to VPN [H02].
Figure 2.1 Virtual Private Network (VPN)
Connecting Two Company Branches
Figure 2.2 A VPN Connection Connecting a
Remote Client to a Private Intranet
VPN basically provides four properties:
1. Encapsulation
2. Authentication
3. Data encryption
4. Address and name server allocation
VPN authentication is the main focus in the article. Next section, we are going to discuss
in detail on two VPN authentication protocols—Password Authentication Protocol (PAP)
and Challenge Handshake Authentication Protocol (CHAP). They are both using
symmetric key system to authenticate remote clients. The user information including
passwords is usually stored on a database server—Remote Authentication Dial-in User
Service (RADIUS)—sitting locally with the VPN server [H02].
3. Authentication Protocol
3.1 Password Authentication Protocol (PAP)
After a connection is established between the client and the VPN server, the VPN server
can request the connection to use Password Authentication Protocol (PAP) for
authentication. Next, the client has to send his/her username and password in clear text to
the VPN server. This protocol is highly insecure to allow password traveling through
public internet without encryption. The reason that any client should ever use PAP is
when the client dials into a VPN server that only understand clear-text password or don’t
support any encrypted password protocol [HR]. Because of this drawback, we are going
to move our attention to a more secure authentication protocol—Challenge Handshake
Authentication Protocol (CHAP).
3.2 Challenge Handshake Authentication Protocol (CHAP)
Challenge Handshake Authentication protocol (CHAP) is considered more secure than
Password Authentication Protocol (PAP) because it encrypts the password as well as the
data. One of the requirements of CHAP is that both end points must have a shared secret
key before they can start transmitting any passwords or data. Actually, there are many
versions of CHAP existed today. Their fundamental methodology is similar to one
another, but some improvements on security can be discussed among them. CHAP is the
most basic Challenge Handshake Authentication Protocol which is not as popular as
Microsoft Challenge Handshake Authentication Protocol (MS-CHAP). MS-CHAP is
basically used in all Windows operating system. MS-CHAP currently has two versions.
MS-CHAP version 1 (MS-CHAPv1) has been found some serious security vulnerabilities.
Therefore, MS-CHAP version 2 (MS-CHAPv2) is the most popular authentication
protocol used in Windows operating system today [HR]. Since MS-CHAPv2 is the most
up-to-date CHAP protocol, we are going to analyze it and will introduce the
improvements on MS-CHAPv2 over MS-CHAPv1.
3.3 Microsoft Challenge Authentication Handshake Protocol, Version 2 (MSCHAPv2)
Since all Challenge Authentication Handshake Protocols are similar with some
improvements on security issues, we will focus our attention to MS-CHAPv2 and discuss
some differences and improvements between MS-CHAPv1 and MS-CHAPv2. Let’s first
look at the authentication procedure of MS-CHAPv2 [SB99].
1. Client requests a login challenge from the server.
2. The server generates a 16-byte random challenge and sends to the client. (In MSCHAPv1, an 8-byte challenge is generated instead of 16-byte challenge).
3. On receipt of the 16-byte random challenge from the server, the client generates
its own 16-byte random challenge called Peer Authenticator Challenge.
4. With the 16-byte random challenge from server, 16-byte Peer Authenticator
Challenge, and the client’s username, the client now concatenates these three
values and feed to a SHA-1 hash function. The first 8 byte of the output value
from the SHA-1 hash function will be picked out as the 8-byte challenge. A
graphic illustration is shown in Figure 3.1.
5. With the 8-byte challenge, the client continues using this challenge and Windows
NT hash function to create a 24-byte reply. This computation will also be
described later. (This part is identical to MS-CHAPv1 except that MS-CHAPv1
uses both LAN Manager and Windows NT hash functions to hash the client’s
password twice and concatenate two 24-byte hash values into a 48-byte reply.
Since LAN Manager hash function is considered much weaker than Windows NT
hash function. MS-CHAPv2 eliminates the step of hashing client’s password with
LAN Manager hash function).
6. The client then sends the 24-byte reply together with the Peer Authenticator
Challenge from Step 3.
7. By retrieving the client’s password stored in database based on the client’s
username, the server can decrypt the 24-byte reply and compare with the 8-byte
challenge. If they are matched, the client is authenticated.
8. The server then uses the 16-byte Peer Authenticator Challenge and the client’s
hashed password to create a 20-byte Authenticator Response.
9. After receiving the Authenticator Response, the client computes its own
Authenticator Response and compares with the Authenticator Response from the
server. If they are matched, the server is authenticated.
Random
challenge
(16 bytes)
Generated by
Server
Peer
Authenticator
Challenge
(16 bytes)
Generated by
Client
Client’s
username
concatenate
Hash
(SHA-1)
Challenge
(the first 8
bytes)
Figure 3.1 Generating 8-byte Challenge
Although we have seen improvements that MS-CHAPv2 has done over MS-CHAPv1,
there are still security and design loopholes that allow attackers to obtain client’s
password. Now, we will look closely at some of these potential security vulnerabilities in
MS-CHAPv2.
3.3.1 Generating 8-byte Challenge
First, let’s explore how the 8-byte challenge in step 4 was actually generated in both MSCHAPv1 and MS-CHAPv2. In MS-CHAPv1, this 8-byte challenge is randomly
generated by the server and sent to the client. Clearly, this is a security problem which
allows eavesdroppers to acquire this 8-byte challenge by simply listening to the channel
between the server and the client. In contrast, MS-CHAPv2 does not require the 8challenge to be transmitted from the server to the client. It utilized a server’s 16-byte
random challenge and a client’s 16-byte Peer Authenticator Challenge, and the client’s
username to generate the 8-byte challenge at the client site. This can prevent
eavesdroppers to obtain this 8-byte challenge. Although this may seem to be a
complicated computation and highly secure design, this step is just a waste in security
perspective. In step 6, the client is actually sending the 24-byte reply together with the
Peer Authenticator Challenge to the server. If the eavesdropper is listening on the channel
from step 2 to step 6, he/she can still find out the 8-byte challenge by computing it
himself/herself with all the information obtained from the channel. As a result, this
improvement of MS-CHAPv2 does not seem to be any securer than MS-CHAPv1 [SB99].
3.3.2 Generating 24-byte MS-CHAP reply
From the step 5 above, the client uses his/her password and the 8-byte challenge
generated from step 4 to create a 24-byte MS-CHAP reply. In this section, the detail
procedure of generating this 24-byte MS-CHAP reply will be examined. Figure 3.2 also
illustrates this process. First, the client feeds his/her password to Windows NT hash
function and will obtain a 16-byte output value. This 16-byte output value will then be
padded with five zero bytes to 21 bytes long. Next, the 21-byte string will be cut into
three 7-byte DES keys. In Figure 3.2, I called these three keys as DES key 1, DES key 2,
DES key 3. Each of these keys will be used to encrypt the 8-byte challenge. Finally, these
three encrypted blocks will be concatenated into a 24-byte MS-CHAP reply.
password
input
NT
hash
Output
(16 bytes)
16 bytes is
padded to 21
bytes with 0
21-byte output
split into 3 7-byte DES keys
DES key 1
(7 bytes)
DES key 2
(7 bytes)
DES key 3
(7 bytes)
(xx00000)
encrypt
encrypt
cncrypt
challenge
(8 bytes)
challenge
(8 bytes)
challenge
(8 bytes)
concatenate
MS-CHAP
reply
(24 bytes)
Figure 3.2 Generating 24-byte MS-CHAP reply
This 24-byte MS-CHAP reply is generated through a complicated algorithm. It is also
long enough to be secure. However, 192 bit long does not mean that the attacker must try
all 2192 possible DES key values to decrypt this MS-CHAP reply. Remember that the last
5 bytes of the 24-byte MS-CHAP reply are all zeros. That is, DES key 3 only has 2 bytes
for the attacker to guess. This only takes 216 tries to find out the real key value. Although
there are still 14 bytes left, it does not take 2112 tries to search the DES key. Because DES
key1 and DES key 2 are used separately to encrypt the challenge. Only 2 * 256 tries is
needed to obtain both DES key 1 and DES key 2. As a result, the total efforts that a
attacker requires to find out all three DES keys are (216 + 257). Besides using brute force
method, the attacker could easily speed up the search by dictionary key search. Since
human passwords are often related or derived from a word or phrase in dictionary. By
trying dictionary words as passwords may help decreasing the time required to break the
password. As a matter of fact, an experiment was conducted to break the password using
a single EFF DES Cracker machine in 9 days. In the article, Cryptanalysis of Microsoft's
PPTP Authentication Extensions (MS-CHAPv2), Schneier and Mudge suggested a
simple method to avoid this serious weakness. That is, to generate the MS-CHAP reply
by using SHA-1 hash function to hash both 16-byte output from Windows NT hash and
the 8-byte challenge [SB99]. This technique would be enough to prevent this kind of
attack.
4. Conclusion
VPN is the most popular methodology that corporations prefer to allow employees or
clients accessing company’s private network. It is extremely important that data traveling
between the employees and the VPN server is secure because confidential information
may be sent through this VPN channel. Because of the significance of transmitting
confidential data through VPN, Microsoft has developed its proprietary version of
Challenge Handshake Authentication Protocol (CHAP) called Microsoft Challenge
Handshake Authentication Protocol (MS-CHAP) which allows a VPN server to
authenticate its clients. MS-CHAP has two versions. Because version 1 was found to
have many security vulnerabilities, Microsoft released MS-CHAP version 2 with security
improvements over MS-CHAPv1. Although today most of the Windows operating
systems are using MS-CHAPv2 as the authentication protocol to talk to the Windows
VPN server, researchers still find vulnerabilities on MS-CHAPv2 and actually ran an
experiment to break it. Nevertheless, authentication phase is only a part of security
protection in VPN. With the encryption and other security technology that VPN provides,
VPN is still the most favorable method that companies used for their remote access.
5. References
[AZ97] Alexander, Zubair. Is RAS Safe. December 1997.
<http://www.windowsitpro.com/Article/ArticleID/135/135.html?Ad=1>.
[FM97] Ford, Merilee; Lew, H. Kim; Spanier, Steve; Stevenson, Tim; Inc Cisco Systems.
Internetworking Technologies Handbook. Cisco Press. March 1, 1997.
[HR] Haden, Rhys. Data Network Resource: Point to Point Protocol (PPP).
<http://www.rhyshaden.com/ppp.htm>.
[H02] Homebase.com. Virtual Private Networking. Technical References. Nov. 8, 2002
<http://endeavorquest.net:8880/knowbase/vpnsetup.htm>.
[SB98] Schneier, Bruce; Mudge, P. Cryptanalysis of Microsoft's Point-to-Point tunneling
protocol (PPTP). Technical report, Counterpane Systems, June 1998.
<http://citeseer.ist.psu.edu/article/schneier98cryptanalysis.html>.
[SB99] Schneier, Bruce; Mudge, P. Cryptanalysis of Microsoft's PPTP Authentication
Extensions (MS-CHAPv2). Technical report, Counterpane Systems, October 19,
1999. <http://www.schneier.com/paper-pptpv2.html>.
[SW96] Simpson W. RFC 1994: PPP Challenge Handshake Authentication Protocol
(CHAP). August 1996 <http://www.cse.ohio-state.edu/cgi-bin/rfc/rfc1994.html>.