Deployment Solution 6.5 PXE IP Communication Flowchart March 8, 20166 © 2008 Symantec Corporation. All rights reserved. ABOUT SYMANTEC Symantec was founded in 1982 by visionary computer scientists. In that spirit, as technology changed so did Symantec. We are focus ed on providing security, storage and systems management solutions to help businesses and consumers secure and manage their information. For more information, visit www.symantec.com. NOTICE Copyright © 2008 Symantec Corporation. All rights reserved. S ymantec, the Symantec Logo , and Altiris are trademarks or registered trademarks of S ymantec Corporation or its affiliates in the U.S. and other countries. Other names may be trademarks of their respectiv e owners. The product described in this document is distributed under licenses restricting its use, copying, distribution, and decompilation/reverse engineering. No part of this document may be reproduced in any form by any means without prior written aut horization of Symantec Corporation and its licensors, if any. THE DOCUMENTATION IS PROVIDED “AS IS” AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND W ARRANTIES, INCLUDING ANY IMPLIED W ARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID. SYMANTEC CORPORATION SHALL NOT BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES IN CONNECTION W ITH THE FURNISHING, PERFORMANCE, OR USE OF THIS DOCUMENTATION. THE INFORMATION CONTAINED IN THIS DOCUMENTATION IS SUBJECT TO CHANGE W ITHOUT NOTICE. The Licensed Software and Documentation are deemed to be commercial computer software as defined in FAR 12.212 and subject to restricted rights as defined in FAR Section 52.227 -19 "Commercial Computer Software - Restricted Rights" and DFARS 227.7202, “Rights in Commercial Computer Software or Commercial Computer Software Documentation”, as applicable, and any successor regulations. Any use, modification, repr oduction release, performance, display or disclosure of the Licensed Software and Documentation by the U.S. Government shall be solely in accordance with the terms of this Agreement. Symantec Corporation 20330 Stevens Creek Blvd. Cupertino, CA 95014 http://www.symantec.com www.symantec.com Step 1: DHCP request/confirmation and PXE boot menu There are two types of architecture that create different types of IP traffic. First we will show communication with PXE and DHCP are on different computers, and then we will show when they are on the same physical computer. DHCP Server PXE Server 5 2 3 1 4 Client Machine 1. The client computer sends out from address 0.0.0.0 port 68 over UDP to address 255.255.255.255 port 67. This is a broadcast packet that is attempting to discover a DHCP server. The packet contains the client MAC address, various configurations of what format and what items the client is expecting to receive as part of the DHCP request. The packet also contains in the conf iguration the option 60 with the string “PXEClient:Arch:00000:UNDI:002001” for a PXE boot as well. If no response is received right away, the client computer sends this same packet out again (sometimes this packet is sent 2–3 times) IP Source Address 0.0.0.0 Source Port Number 68 IP Destination Address 255.255.255.255 Destination Port Number 67 IP Protocol UDP Packet Type DHCP Discovery 2. The DHCP server sends out a packet from its IP address and port 67 over UDP to a broadcast address of 255.255.255.255 and port 68. The packet contains an available IP address that can be used by the client. It also contains the MAC address of the client computer that www.symantec.com Deployment Solution 6.5 PXE IP Communication Flowchart > 1 the new IP address has been reserved for. The DHCP server will only send this out once for each discovery request that it receives and then wait to hear back from the client. IP Source Address IP of DHCP Source Port Number 67 IP Destination Address 255.255.255.255 Destination Port Number 68 IP Protocol UDP Packet Type DHCP Offer Same screen as above. 3. The PXE server will hear the client DHCP discovery request, and then wait for the DHCP response. Once it sees both of these packets on the network it then will send out from its IP address on port 67 a UDP broadcast packet to 255.255.255.255 port 68. This packet is very similar to the DHCP response packet, except that it contains option 60 with the string “PXEClient”, and it has option 43 with the MTFTP server IP address as well as two ports to use to access the MTFTP server (the port to send from and the port to receive on). Option 43 also contains the PXE boot menu and boot prompt information. IP Source Address IP of PXE Source Port Number 67 IP Destination Address 255.255.255.255 Destination Port Number 68 IP Protocol UDP Packet Type DHCP Offer Same screen as above. 4. The client sends out another broadcast packet from 0.0.0.0 port 68 over UDP to address 255.255.255.255 port 67. This packet is a DHCP request instead of the earlier DHCP discovery packet. Basically this is a confirmation packet telling the DHCP server that it has received the offered IP address, and that it is going to be using that IP address. IP Source Address 0.0.0.0 Source Port Number 68 IP Destination Address 255.255.255.255 Destination Port Number 67 IP Protocol UDP Packet Type DHCP Request Same screen as above. 2 < Deployment Solution 6.5 PXE IP Communication Flowchart www.symantec.com 5. The DHCP server broadcasts another packet from port 67 to port 68 containing the MAC address of the client computer as well as the confirmed new IP address. This basically is an acknowledgement to the client computer letting it know that it has successfully reserved the IP address given for that client computer. The packet also can contain DNS server IP addresses, domain name, router, and subnet mask. IP Source Address IP of DHCP Source Port Number 67 IP Destination Address 255.255.255.255 Destination Port Number 68 IP Protocol UDP Packet Type DHCP Acknowledgement Second architecture type: DHCP and PXE reside on the same ph ysical computer. DHCP Server PXE Server 1 2 3 4 5 6 Client Machine 6. The client computer sends out from address 0.0.0.0 port 68 over UDP to address 255.255.255.255 port 67. This is a broadcast packet that is attempting to discover a DHCP server . The packet contains the client MAC address, various configurations of what format and what items the client is expecting to receive as part of the DHCP request. The packet also contains in the conf iguration the option 60 with the string “PXEClient:Arch:00000:UNDI:002001” for a PXE boot as well. If no response is received right away, the client computer sends this same packet out again (sometimes this packet is sent 2–3 times) www.symantec.com Deployment Solution 6.5 PXE IP Communication Flowchart > 3 IP Source Address 0.0.0.0 Source Port Number 68 IP Destination Address 255.255.255.255 Destination Port Number 67 IP Protocol UDP Packet Type DHCP Discovery 7. The DHCP server sends out a packet from its IP address and port 67 over UDP to a broadcast address of 255.255.255.255 and port 68. The packet contains an available IP address that can be used by the client. It also contains the MAC address of the client computer that the new IP address has been reserved for. This packet also contains option 60 with the string “PXEClient” to let the client know that this response is the PXE response as well as the DHCP response. The DHCP server will only send this out once for each discovery request that it receives and then wait to hear back from the cl ient. IP Source Address IP of PXE/DHCP Source Port Number 67 IP Destination Address 255.255.255.255 Destination Port Number 68 IP Protocol UDP Packet Type DHCP Offer Same as screen above. 8. The client sends out another broadcast packet from 0.0.0.0 port 68 over UDP to address 255.255.255.255 port 67. This packet is a DHCP request instead of the earlier DHCP discovery packet. Basically this is a confirmation packet telling the DHCP server that it has received the offered IP address, and that it is going to be using that IP address. This packet also contains the same option 60 as packet 1, but this option is ignored by the DHCP server at this time. IP Source Address 0.0.0.0 Source Port Number 68 IP Destination Address 255.255.255.255 Destination Port Number 67 IP Protocol UDP Packet Type DHCP Request Same as screen above. 9. The DHCP server broadcasts another packet from port 67 to port 68 containing the MAC address of the client computer as well as the confirmed new IP address. This basically is an acknowled gement to 4 < Deployment Solution 6.5 PXE IP Communication Flowchart www.symantec.com the client computer letting it know that it has successfully reserved the IP address given for that client computer. The packet also can contain DNS server IP addresses, domain name, router, and subnet mask. IP Source Address IP of PXE/DHCP Source Port Number 67 IP Destination Address 255.255.255.255 Destination Port Number 68 IP Protocol UDP Packet Type DHCP Acknowledgement Same as screen above. 10. The client computer sends out a unicast packet from its new IP address over port 68 with UDP to the address of the PXE/DHCP server on port 4011. The purpose of this packet is to request from the PXE server the PXE boot menu, along with al l of its corresponding information. This packet is nearly identical to the packet sent in step 3, except that this time the PXE server will see the packet and recognize that option 60 is in this packet (as this option was ignored by DHCP in step 3). IP Source Address New IP address of Client Source Port Number 68 IP Destination Address IP of PXE/DHCP Destination Port Number 4011 IP Protocol UDP Packet Type DHCP Request Same as screen above. 11. The PXE server will send a unicast UDP packet to the client computer from its IP address on port 67 to the IP address of the client computer on port 68. This packet is very similar to the DHCP acknowledgement packet in step 4, except that it is unicast and contains option 43 with the MTFTP server IP address as well as two ports to use to access the MTFTP server (the port to send from and the port to receive on). Option 43 also contains the PXE boot menu and boot prompt information. IP Source Address www.symantec.com IP of PXE/DHCP Source Port Number 67 IP Destination Address New IP address of Client Destination Port Number 68 IP Protocol UDP Packet Type DHCP Acknowledgement Deployment Solution 6.5 PXE IP Communication Flowchart > 5 At this point the client computer will do one of a few options. The client might be running the initial deployment boot option, the user might press [F8] and view the full PXE boot menu, there might be a job scheduled for that client, and it automatically chooses a boot option, or there might not be anything scheduled for the client computer and it automatically chooses local boot. Each of these options will be detailed below. Additional Information about DHCP options 60, 54 and 43 Much of the useful information that is passed between the PXE Server and client is put into these options. The above information only briefly explains these options and does not go into much detail of the format of these options. Option 60 is referred to in DHCP as “Vendor -Specific Information”. It is basically a string saying “PXEClient” in the packet that is sent from both a client computer that is booting off of its NIC, and it is also in the PXE server’s response. The string in that option might have more characters than just “PXEClient”, but it must have at least that string in it. If the PXE server and DHCP are on the same computer, this option will be set in the initial DHCP response. If those components are on separate computers the DHCP server response will not have that option, but the PXE server will. The option in the clients DHCP request lets the PXE server know that the client wants to PXE boot as well as get a DHCP server. The option in the PXE servers response l ets the client know who the PXE server is, and who to request more information from to continue the PXE boot. Option 54 is labeled as “Server Identifier”. This is an IP address that will be used by the client to request the start of the boot file download from the MTFTP server. Usually this address will be the same as the PXE server, but it can be different if the PXE server and MTFTP server are on different physical computers. The IP address for the MTFTP server in option 43 does not contain the direct IP address, but rather the multicast IP address. This option is needed so that the client can directly address the MTFTP server without sending a multicast/broadcast packet. Option 43 is only sent from the PXE server to the client. This option contains many values and is broken into various sub options. This option contains all of the data that the client needs to request any PXE boot option. The sub-options (and data contained) in option 43 are as follows: MTFTP server IP address along with ports to send from and to send to, the MTFTP timeout and delay times, the PXE boot control and boot servers, the PXE boot menu and PXE prompt. The PXE boot menu will have the default menu choice item at the top (this menu order is dynamically made for each client based on w hat jobs are assigned to the 6 < Deployment Solution 6.5 PXE IP Communication Flowchart www.symantec.com client). The boot menu also has in the third byte of the field the menu timeout. If that byte in the menu is 00, the top option will automatically be chosen immediately. If that byte is 03 then it will wait 3 seconds before choosing the top menu item (such is the default case of local boot when no jobs are assigned), and if that byte is FF, the menu will wait indefinitely (this is the default behavior of initial deployment). How the PXE server creates custom boot menus for each client In the above steps it was mentioned that the PXE server sends down to the client a personalized boot menu with the default option on the top of the menu and a timeout period. It might be questioned how the PXE server knows what to send to the clien t computer. W henever a PXE server service starts for the first time it makes a request to the Deployment Server engine for a list of known client computer’s MAC addresses. It also queries from the DS engine if any of these MAC addresses have a job assigned to them that would require a PXE boot into automation. The PXE server does not store any of this data in a file, but rather keeps a running index or database in RAM of all client computers MAC addresses, the state of the computer (production or automation), the node type, and the boot option ID (which PXE boot option is the default or should be first). The PXE server updates this index whenever there are any changes made to the PXE server (such as when the PXE Configuration Utility is closed), whenever a j ob is assigned or scheduled to a client computer, and it also updates itself every so often (around every 5 minutes or so). Step 2: PXE Boot menu option selection Local Boot If a user is at the client computer while it is booting up and he or she manually presses F8 and select local boot, or if her or she presses escape during the above PXE boot process, no further packets are sent from the client in regards to PXE, and the client computer continues the BIOS boot order (most likely the production hard drive next). If no user is at the client computer and the boot menu times out (determined by the boot menu in option 43) then the client computer will use the boot item that is first on the boot menu. If that option was local boot, no further communication will be made between the client and the server. The BIOS will just continue to the next boot option after the NIC card (usually the production hard drive). www.symantec.com Deployment Solution 6.5 PXE IP Communication Flowchart > 7 MTFTP Server Switch (or other layer 2 device) PXE Server 7 5 3 6 2 1 4 Client Machine Automation OS boot part 1(Downloading the .0 file) 1. Regardless of the PXE menu choice selected, and how it is selected (whether it was automatically selected because it was at the top of the list, or a user at the computer manually selected one of the options). Once the client knows which menu choice it is going to choose is sends a UDP datagram to the PXE server for more information about that specific menu choice so that it can start to download it. This is a UDP packet sent unicast from port 4011 to the PXE server’s port 4011. This packet does not contain any DHCP options (such as 60) because the DHCP process is over at this point. The packet is simply a UDP datagram of 548 bytes that contains the request for boot control from the PXE server in an Altiris proprietary format. IP Source Address IP of Client Source Port Number 4011 IP Destination Address IP of PXE Destination Port Number 4011 IP Protocol UDP Packet Type Datagram 2. In response the PXE server sends down a unicast UDP packet from port 67 to the IP of the client computer on port 4011. This packet is almost the same as the previous DHCP acknowledgement except that instead of having no boot file name, the boot file name is in the packet as the file on the MTFTP server to download (the .0 file name). This response also has options 60 and 43, but the data in them is already on the client. The only reason for those other options is so that the client knows that this DHCP response is a direct resu lt of its request for the file name of the .0 file for the selected boot menu item. 8 < Deployment Solution 6.5 PXE IP Communication Flowchart www.symantec.com IP Source Address IP of PXE Source Port Number 67 IP Destination Address IP of Client Destination Port Number 4011 IP Protocol UDP Packet Type DHCP Acknowledgement 3. The client computer now knows exactly what file to ask for from the MTFTP server. It also knows the direct IP address of the MTFTP server (this is from option 54 of the PXE/DHCP response). The client computer sends a unicast datagram UDP packet from port 1758 to the direct IP address of the MTFTP server IP ad dress on port 1759 (ports 1758 and 1759 are the default ports used for MTFTP requests and responses, but these ports can be configured in the PXE configuration tool. For the rest of this tutorial these defaults will be used). The data in this packet is in an Altiris proprietary format, but mainly contains the .0 boot file name, and a request for download. IP Source Address IP of Client Source Port Number 1758 IP Destination Address IP of MTFTP Destination Port Number 1759 IP Protocol UDP Packet Type Datagram 4. The client computer sends another packet right away to the MTFTP multicast address. This is an IGMP membership report packet. There are no source or destination ports in this packet, and in fact it is not received by any other computers on the network. The purpose of this packet is to let the switches (and other level 2 devices) in this network know that this client computer is part of the multicast group for the multicast address that it has sent out. W heneve r multicast packets of the reported address are sent from this point on, they will be sent to this client computer. IP Source Address IP of Client Source Port Number none IP Destination Address 224.1.x.x Destination Port Number none IP Protocol IGMP Packet Type IGMP Report 5. The MTFTP server will next start sending the first .0 boot file. The first packet will be sent twice. Once to the direct IP address of the client computer, and the other to the multicast address established by the MTFTP server. These packets will be UDP datagrams going from port 1759 to the client port of 1758. www.symantec.com Deployment Solution 6.5 PXE IP Communication Flowchart > 9 IP Source Address IP of MTFTP Source Port Number 1759 IP Destination Address 224.1.x.x Destination Port Number 1758 IP Protocol UDP Packet Type Datagram IP Source Address IP of MTFTP Source Port Number 1759 IP Destination Address IP of Client Destination Port Number 1758 IP Protocol UDP Packet Type Datagram and 6. The client computer after receiving the packet will respond with a small unicast UDP datagram packet from its IP address on port 1758 to the direct IP address of the MTFTP server on port 1759. This packet is just a confirmation that it recei ved the last packet from the MTFTP server. It also lets the MTFTP server know which of the first 2 packets it received (either the multicast one of the directed one). IP Source Address IP of Client Source Port Number 1758 IP Destination Address IP of MTFTP Destination Port Number 1759 IP Protocol UDP Packet Type Datagram 7. The MTFTP server will send the next UDP datagram packet o f the .0 boot file. This packet will be either a multicast packet or a unicast packet depending on the response it received from the client in its previous communication. Once this format is defined it will continue to send the UDP datagram packets in this format until the entire .0 file is sent down. IP Source Address IP of MTFTP Source Port Number 1759 IP Destination Address 224.1.x.x Destination Port Number 1758 IP Protocol UDP Packet Type Datagram IP Source Address IP of MTFTP Source Port Number 1759 IP Destination Address IP of Client Destination Port Number 1758 IP Protocol UDP Packet Type Datagram or Steps 6 and 7 are repeated alternatively until the MTFTP server has sent down its last packet containing the end of the .0 file, and the clien t has sent its final acknowledgement. 10 < Deployment Solution 6.5 PXE IP Communication Flowchart www.symantec.com Contents and purpose of the .0 file All automation OS environments have a .0 file. This file was called in Deployment Solution 6.1 managed.0 or newcomp.0 (for Managed PC and Initial Deployment respectively). In 6.5 the se files will be named after what order they appear by default in the PXE boot menu. The first PXE boot file will have a file named MenuOption128.0 and the second one will have a boot file named MenuOption129.0 … etc. Once this file has been downloaded com pleted from the MTFTP server the client will load the file into memory and start to execute the file’s code. This file is a bootstrap program that tells the client computers how much memory to allocate for the rest of the pre -boot operating system, and what other files will be included in the pre-boot operating system. The bootstrap program also starts the requests for the other boot files, loads those into memory, and then transfers control over to those other programs (generally those are the actual autom ation environment). Usually this file is somewhere between 15–25 KB (depending on what the rest of the boot OS is) in size, but never larger than 32 KB. Loading the rest of the operating system At this point the .0 file has control of the rest of the PXE b oot process. Each of the environments has a different .0 file and thus a different method for continuing the rest of the automation environment. For example DOS re-establishes a new multicast session with a new multicast IP and then starts downloading a .1 file. Linux stops using multicast and goes to TFTP and starts downloading the pxelinux.cfg file. Windows PE also stops using multicast and uses TFTP to start downloading the NTLDR file. There could also be a custom OS that behaved in a completely differen t manner. Normally after the boot loader program (the .0 file) has finished setting up memory and loaded the rest (or at least some of the other) of the OS files into memory it transfers control to some other program to actually run the automation environment. www.symantec.com Deployment Solution 6.5 PXE IP Communication Flowchart > 11