NIC Replacement Guide
Isilon
X410
Replace a Network Interface Card (NIC)
May, 2016
l
l
l
l
l
l
l
l
l
Replacing a NIC.......................................................................................................... 2
Gather logs................................................................................................................ 2
Download a Field Replacement Unit (FRU) package.................................................... 2
Install the NIC replacement........................................................................................ 3
Install the FRU package and run scripts...................................................................... 7
Gather logs.............................................................................................................. 10
Returning a failed part to Isilon................................................................................ 10
Update the install database..................................................................................... 10
Where to go for support............................................................................................11
NIC Replacement Guide
Replacing a NIC
You can replace a failed network interface card (NIC) in the field.
CAUTION
Perform this procedure on only one node at a time. Performing maintenance on multiple
nodes in parallel may lower the protection level of the cluster, put data at risk, and lead
to the interruption of client workflows.
Working with clusters in SmartLock compliance mode
Clusters running in SmartLock compliance mode require a sudo prefix to run root
commands.
If a cluster is running in SmartLock compliance mode, root access is disabled on the
cluster. Because of this, you can run some commands only through the sudo program.
Prefixing a command with sudo enables you to run commands that require root access.
For example, if you do not have root access, the following command fails:
isi drivefirmware status
However, if you are on the sudoers list, the following command succeeds:
sudo isi drivefirmware status
Compliance mode commands that require changes beyond the sudo prefix are noted in
the procedure steps.
For more information on the sudo program and compliance mode commands, see the
OneFS CLI Administration Guide.
Gather logs
Before you begin any maintenance on a cluster, gather cluster logs.
You must collect cluster logs before all maintenance procedures. Cluster logs provide
snapshots of the cluster, which you can review to make sure that maintenance is
successful.
Procedure
1. Open a secure shell (SSH) connection to any node in the cluster and log in.
2. Gather cluster logs by running the following command:
isi_gather_info
Download a Field Replacement Unit (FRU) package
Before you replace a component in a configure-to-order (CTO) node, obtain a Field
Replacement Unit (FRU) package from the EMC FTP site. The FRU package updates the
CTO and as-built information on the node, then forwards the updated information to
Isilon Technical Support.
2
X410 NIC Replacement Guide
Replace a Network Interface Card (NIC)
Procedure
1. Download the latest FRU package from ftp://ftp.emc.com/outgoing/Fru_Package/.
2. Note the name of the FRU package. You will use the name for other commands.
Package names follow this convention:
IsiFru_Package_ <date-time-stamp> .tgz
For example: IsiFru_Package_201507072125.tgz
3. Place the FRU package on the cluster through a network drop, or by asking someone
at the cluster site to place the package for you. If neither of these options is available
to you, contact Isilon Technical Support for assistance.
Install the NIC replacement
Remove the failed Network Interface Card (NIC) and install the replacement hardware.
Power down the node
Power down the node before performing maintenance.
Procedure
1. Connect to an available node in the cluster with a serial cable or network drop.
2. Determine the IP address of the node you are powering down by typing the command:
isi status -q
3. From the node that you connected to, open a secure shell (SSH) connection to the
node that is to be shut down by typing the command:
ssh <node_ip_address>
4. Power down the node by typing the following command:
shutdown -p now
If the node does not respond to the shutdown command, press the Power button on
the node three times, and then wait five minutes. If the node still does not shut down,
you are at risk for losing data. Do not proceed. Contact EMC Isilon Technical support
for assistance.
CAUTION
A forced power down should be attempted only if a node is unresponsive. Forcing the
power down of a healthy node can result in data loss.
5. Verify that the node is powered down by typing the following command:
isi status -q
Confirm that the node has a status of D--R (Down, Read Only). See node 3 in the
following example.
ID |IP Address
|DASR| In
Out Total| Used / Size
|
Used / Size
---+---------------+----+-----+-----+-----+------------------+1|10.53.217.201 | OK | 48M|
0| 48M|
19G/ 6.2T(< 1%)|(No
Install the NIC replacement
3
NIC Replacement Guide
SSDs)
2|10.53.217.202
SSDs)
3|10.53.217.203
n/a/n/a( n/a)
| OK |
46M|
0|
46M|
23G/ 6.2T(< 1%)|(No
|D--R|
n/a|
n/a|
n/a|
n/a/
n/a( n/a)|
Slide the node out of the rack
Slide the node away from the rack to access the contents of the node.
Procedure
1. Label the InfiniBand, ethernet, and power cables connected to the back of the node to
ensure that they are reconnected correctly.
2. Disconnect all cables from the back of the node.
Note
If there are transceivers connected to the end of your IB or ethernet cables, make sure
to remove them with the cables. If you are using fiber ethernet cables, you will need to
disconnect the cable from the transceiver, then remove the transceiver from the node.
3. Remove the node front panel.
4. Remove the retaining screws that secure the node to the rack cabinet.
5. Slide the node from the rack cabinet to fully extend the slide rails and provide clear
access to the node. Do not remove the node from the slide rails.
DANGER
Slide the node out from the rack slowly. Do not extend the rails completely until you
confirm that the node is latched and safely secured to the rails.
Remove the node top panel
You remove the top panel to gain access to the contents of the node.
WARNING
Properly ground yourself to prevent electrostatic discharge from damaging the node. For
example, attach an ESD strap to your wrist and the node chassis.
Procedure
1. Loosen the captive screw that secures the node top panel.
2. Slide the top panel toward the rear of the node, and then lift the top panel to access
the node interior.
Remove the cross bracket and air baffle
To gain full access to the internal components of the node, you must remove the cross
bracket and air baffle.
The cross bracket is secured to the chasses with a thumb screw. The back of the air baffle
is hooked to the cross bracket and the front is hooked to the front end of the chassis. You
can remove both items simultaneously.
4
X410 NIC Replacement Guide
Replace a Network Interface Card (NIC)
Procedure
1. Locate the cross bracket and air baffle within the node.
GHE-000020
2. Loosen the thumb screw on the right side of the cross bracket.
3. Lift the end of the cross bracket that contains the thumb screw until you are able to
unhook the front end of the air baffle from the chassis.
4. Continue to lift the free end of the cross bracket until you can unhook it from the other
side of the chassis.
WARNING
As you lift the cross bracket and air baffle away from the node, ensure that you do
not pinch or disconnect cables that run through the cutouts in the air baffle.
Remove the failed NIC
After the node is open, locate the failed NIC. Remove the failed card, then install the new
card.
Procedure
1. Locate the NIC in the fourth card slot from the left side of the node.
2. If necessary, disconnect any cables from the NIC.
3. Remove the mounting screw securing the NIC to the node.
4. Remove the failed NIC from the node.
Remove the failed NIC
5
NIC Replacement Guide
GHE_000030
Install the new NIC
After the original NIC is removed, you can install the new NIC.
Procedure
1. Insert the new NIC in the empty card slot from which you removed the original NIC.
Check to ensure that the card is fully seated in the motherboard connector.
2. Secure the card to the back panel of the node with the mounting screw.
3. Reconnect any cables that you removed from the card.
Install the cross bracket and air baffle
You can replace the cross bracket and air baffle simultaneously.
Procedure
1. With the cross bracket hooked to the back end of the air baffle, hook the side of the
cross bracket that does not have a thumb screw to the bracket slots on the left side of
the node.
2. Lower the cross bracket into the node until you can hook the front end of the air baffle
to the front of the node chassis.
WARNING
As you slide the air baffle into place, position cables to run through the cutouts in the
baffle. Do not pinch cables underneath the baffle.
3. Secure the right side of the bracket to the node with the thumb screw.
WARNING
The cross bracket sits directly above the PCIe cards and DIMMs. Use caution when
installing the cross bracket so that no components are dislodged or damaged. Ensure
that all PCIe cards are completely upright, perpendicular to the board.
6
X410 NIC Replacement Guide
Replace a Network Interface Card (NIC)
Install the node top panel
You must secure the top panel onto the node.
Procedure
1. Place the top panel on the node so that the front edge of the top panel is about one
inch behind the drive bays, and then slide the top panel forward into place.
WARNING
The chassis intrusion switch can be damaged if the top panel is slid too far back on
the node.
2. Tighten the captive top panel screw to secure the top panel to the node.
Return the node to the rack
Return the node to the rack after all work is complete.
Procedure
1. Slide the node back into the rack cabinet.
WARNING
Slide the node slowly so you do not slam the node into the rack and damage the
node.
2. Reconnect the ethernet, InfiniBand, and power cables to the back of the node.
3. Secure the node to the rack cabinet.
4. Replace the node front panel.
Power up the node
Power up the node by pressing the power button on the back panel.
Procedure
1. Power up the node by pressing the power button on the back panel of the node. It is
located in the upper left corner of the back panel, to the right of the power supplies.
Install the FRU package and run scripts
Update the configure-to-order (CTO) and as-built information on the node by installing a
FRU package.
Note
If your cluster is running in SmartLock compliance mode, you will need to enter the
provided compliance mode commands to run the FRU scripts.
Install the node top panel
7
NIC Replacement Guide
Install the FRU package on the node
Unpack and install the FRU package on the node.
Procedure
1. Place the FRU package on the node.
2. Unpack the FRU package by running the following command:
tar -zxvf IsiFru_Package_<date-time-stamp>.tgz
3. Type cd to change to the directory containing the FRU tar.
4. Install the package. Depending on your version of OneFS, run one of the following
commands:
OneFS 8.0 or later
isi upgrade patches install IsiFru_Package_<date-timestamp>_.tar
Earlier than OneFS 8.0
isi pkg install IsiFru_Package_<date-time-stamp>.tar
As the package installs, the following message appears:
Preparing to install the package...
Checking the package for installation...
Installing the package
Committing the installation...
Package is committed.
Run the update script
After the FRU package is installed on the node, run the update script.
Procedure
1. Move to the FRU package location by running the following command:
cd /var/crash/cto/fruPackages/IsiFru_Package_<date-time-stamp>
2. Perform the update script by running the following command:
./isi_fru_update_cluster
The system displays confirmation of the following items:
l
CTO capability
l
Current node hardware configuration
Run the ABR script
Run the As Built Record (ABR) script to report the updated hardware to Isilon Technical
Support.
Procedure
1. Verify installation of the updated hardware by running the following command:
./isi_cto_update --abr
8
X410 NIC Replacement Guide
Replace a Network Interface Card (NIC)
The update is verified and a series of status messages confirm the node configuration,
and if an FTP connection is available, an updated ABR is sent to Isilon Technical
Support.
2. If an external connection is not available, manually collect and deliver to Isilon
Technical Support the updated ABR.
3. If the cluster is running in SmartLock compliance mode, verify installation of the
updated hardware by running the following command:
sudo /usr/bin/isi_hwtools/isi_cto_update --abr --filepath .
Note
You must include the period at the end of the command.
Sending an ABR to Isilon with no connectivity
If no external connectivity is available, the As Built Record on a Configure to Order (CTO)
node cannot be automatically delivered to Isilon Technical Support.
If external connectivity is available, the ABR is automatically generated and delivered to
Isilon Technical Support. If there is no external connectivity available, you must generate
and copy the ABR from the node, and then send the ABR to Isilon Technical Support
through an alternate connection.
Generate an ABR
You can manually send an As Built Record (ABR) by copying an XML file from the node
and emailing the file to Isilon Technical Support. You need network access to the node, or
you can request that the customer provide the file to you.
Procedure
1. Generate an ABR by running the following command:
isi_make_abr
The command generates a temporary file named asbuilt_ <serial-number>_<datetime-stamp> .xml.
2. Identify the full name of the ABR file by running the following command:
isi_inventory_tool --display --itemType asbuilt | grep
asbuiltFileName=
The system output contains information about the ABR file.
3. Place the ABR file where you can copy it by running the following command:
isi_inventory_tool --display --itemType asbuilt > /ifs/
asbuilt_ <serial-number>_<date-time-stamp> .xml
4. Copy the generated asbuilt_ <serial-number>_<date-time-stamp> .xml file.
5. If an FTP connection is not available, contact Isilon Technical Support for an alternate
delivery method.
Run the ABR script
9
NIC Replacement Guide
Remove the FRU package from the node
After all scripts are run, remove the FRU package from the node.
Procedure
1. Change out of the FRU package directory by running the following command:
cd /
2. Delete the FRU package from the node. Depending on your version of OneFS, run one
of the following commands:
OneFS 8.0 or later
isi upgrade patches uninstall IsiFru_Package_ <date-timestamp>
Earlier than OneFS 8.0
isi pkg delete IsiFru_Package_ <date-time-stamp>
Gather logs
After you complete maintenance on a cluster, gather cluster logs.
You must collect cluster logs after all maintenance. Cluster logs provide snapshots of the
cluster that you can review to make sure that maintenance is successful.
Procedure
1. Gather cluster logs by typing the command:
isi_gather_info
Returning a failed part to Isilon
Return the failed part to Isilon Technical Support.
Procedure
1. Contact Isilon Technical Support to notify them that you are returning a failed part.
2. Package the failed part in the packaging materials provided with the replacement
part.
3. Attach the return label that was included with the replacement part.
4. For the RMA number, write the support case number provided by Isilon Technical
Support.
5. Ship the failed part to the address specified on the return label.
Update the install database
After all work is complete, update the install database.
Procedure
1. Browse to the EMC Product Registration and Install base Maintenance service portal,
at: http://emc.force.com/createPSCcase.
10
X410 NIC Replacement Guide
Replace a Network Interface Card (NIC)
2. Select the Product Registration and Install Base Maintenance
option.
3. To open the form, select the IB Status Change option.
4. Complete the form with the applicable information.
5. To submit the form, click Submit.
Where to go for support
Contact EMC Isilon Technical Support for any questions about EMC Isilon products.
Online Support
Live Chat
Create a Service Request
Telephone
Support
United States: 1-800-SVC-4EMC (800-782-4362)
Canada: 800-543-4782
Worldwide: +1-508-497-7901
For local phone numbers for a specific country, see EMC Customer
Support Centers.
Help with Online
Support
For questions specific to EMC Online Support registration or access,
email support@emc.com.
Isilon Info Hubs
For the list of Isilon info hubs, see the Isilon Info Hubs page on the
EMC Isilon Community Network. Isilon info hubs organize Isilon
documentation, videos, blogs, and user-contributed content into
topic areas, making it easy to find content about subjects that
interest you.
Support for IsilonSD Edge
If you are running a free version of IsilonSD Edge, community support is available through
the EMC Isilon Community Network. However, if you have purchased one or more licenses
of IsilonSD Edge, you can contact EMC Isilon Technical Support for assistance, provided
you have a valid support contract for the product.
Where to go for support
11
NIC Replacement Guide
Copyright © 2016 EMC Corporation. All rights reserved. Published in the USA.
Published May, 2016
EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without
notice.
The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with
respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a
particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software
license.
EMC², EMC, and the EMC logo are registered trademarks or trademarks of EMC Corporation in the United States and other countries.
All other trademarks used herein are the property of their respective owners.
For the most up-to-date regulatory document for your product line, go to EMC Online Support (https://support.emc.com).
12
X410 NIC Replacement Guide