EMC ISILON CUSTOMER TROUBLESHOOTING GUIDE HOW TO ADD A DRIVE TO A NODE Abstract This guide helps you to troubleshoot problems with replacing a drive in a node. December 28, 2015 1 - EMC Isilon Customer Troubleshooting Guide: How to Add a Drive to a Node We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback. Contents and overview Note Follow all of these steps, in order, until you reach a resolution. 1. Follow these steps. Page 3 Before you begin 2. Perform troubleshooting steps in order. Page 4 Start troubleshooting 3. Appendixes Page 5 Add the drive Appendix A If you need further assistance Appendix B How to use this flowchart Appendix C Example isi devices output Appendix D Example isi job status output with FlexProtect job running Appendix E Example isi job status output with FlexProtect job completed Appendix F Example isi devices output showing a PREPARING or a HEALTHY drive Appendix G Example output from the isi devices -a add -d :<bay#> command 2 - EMC Isilon Customer Troubleshooting Guide: How to Add a Drive to a Node We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback. Before you begin CAUTION! If the node, subnet, or pool that you are working on goes down during the course of troubleshooting and you do not have any other way to connect to the cluster, you could experience data unavailability. Therefore, make sure that you have more than one way to connect to the cluster before you start this troubleshooting process. The best method is to have a serial cable available. This way, if you are unable to connect through the network, you will still be able to connect to the cluster physically. For specific requirements and instructions for making a physical connection to the cluster, see article 16744 on the EMC Online Support site. Before you begin troubleshooting, confirm that you can connect through either another subnet or pool, or that you have physical access to the cluster. Configure logging through SSH We recommend that you configure screen logging to log all session input and output during your troubleshooting session. This log file can be shared with EMC Isilon Technical Support, if you require assistance at any point during troubleshooting. Note: The screen session capability does not work in OneFS 7.1.0.6 and 7.1.1.2. If you are running either of these versions, you can configure logging by using your local SSH client's logging feature. 1. Open an SSH connection to the cluster and log in by using the root account. Note: If the cluster is in compliance mode, use the compadmin account to log in. All compadmin commands must be preceded by the sudo prefix. 2. Change the directory to /ifs/data/Isilon_Support by running the following command: cd /ifs/data/Isilon_Support 3. Run the following command to capture all input and output from the session : screen -L This will create a file named screenlog.0 that will be appended to during your session. 4. Perform troubleshooting. 3 - EMC Isilon Customer Troubleshooting Guide: How to Add a Drive to a Node We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback. Start troubleshooting Introduction Start troubleshooting here. If you need help to understand the flowchart conventions used in this guide, see Appendix B: How to use this flowchart. Note Start Self encrypting drives (SEDs) can stay in the PREPARING state for an hour or longer. This behavior is normal when formatting a drive. Do not interrupt the PREPARING state because doing so locks the drive. If you have not done so already, log in to the cluster and configure screen logging through SSH, as described on page 3. Determine the drive health by running the following command: isi devices See __________ Appendix C for example output. Does the output indicate that any of the drive bays are in SMARTFAIL, REPLACE, or EMPTY status? SMARTFAIL REPLACE or EMPTY Go to Page 5 Go to Page 6 4 - EMC Isilon Customer Troubleshooting Guide: How to Add a Drive to a Node We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback. Add the drive You could have arrived here from: _________________________ Page 4 – Start troubleshooting _____________________ Page 6 – Add the drive (2) Page 5 Verify whether the FlexProtect job is running by running the following command : isi job status In the Running and queued jobs section, find the FlexProtect job. See __________ Appendix D for example output. Yes Allow the FlexProtect job to finish. After the job is successfully completed, the drive bay status changes to REPLACE. See __________ Appendix E for example output of a successfully completed FlexProtect job and the subsequent REPLACE status. Is the FlexProtect job running? No Note the page number that you are currently on. Upload log files and contact Isilon Technical Support, as instructed in Appendix A. After the FlexProtect job is successfully completed, return to page 4 to continue troubleshooting. Return to Page 4 5 - EMC Isilon Customer Troubleshooting Guide: How to Add a Drive to a Node We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback. Add the drive (2) You could have arrived here from: Page 4 – Start troubleshooting Page 6 Look for drives that are in a down or soft_fail status by running the following command: sysctl efs.gmp.group See the example output at the bottom of this page. Yes Does the output indicate that any devices are in a down or soft_failed status? Return to Page 5 No Go to Page 7 Example sysctl efs.gmp.group output Cluster-1# sysctl efs.gmp.group efs.gmp.group: <2,110>: { 1:0-33, 2:0-16,18-33, 3:0-14,16-33,36, down: 2:17, soft_failed: 2:17 } 6 - EMC Isilon Customer Troubleshooting Guide: How to Add a Drive to a Node We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback. Add the drive (3) You could have arrived here from: Page 6 – Add the drive (2) Page 7 Replace the drive by consulting the drive replacement guide for the particular type of node. See this search string for a list of the available drive replacement documents. To add the new drive, run the following command on the node that contains the new drive, where <bay#> is the number of the bay that the drive is added to: isi devices -a add -d :<bay#> See __________ Appendix G for example output to determine whether the drive is successfully added . Is the drive successfully added? Yes No Note the page number that you are currently on. Upload log files and contact Isilon Technical Support, as instructed in Appendix A. 7 - EMC Isilon Customer Troubleshooting Guide: How to Add a Drive to a Node We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback. Go to Page 8 Add the drive (4) You could have arrived here from: _____________________ Page 7 – Add the drive (3) ______________________ Page 10 – Add the drive (6) Note Self encrypting drives (SEDs) can stay in the PREPARING state for an hour or longer. This is normal behavior when formatting a drive. Do not interrupt the PREPARING state because this will lock the drive. Page 8 Verify that the drive is either in the process of formatting, or is ready to use, by running the following command: isi devices In the output, look for the bay number and the status of PREPARING or HEALTHY. See__________ Appendix F for example output. Does the output indicate that the drive status is PREPARING or HEALTHY? No Yes A drive status of PREPARING or HEALTHY indicates that the drive replacement is successful. Return the non-working drive to EMC Isilon Technical Support by using the provided return label that is located under the shipping label of the new drive. End troubleshooting 8 - EMC Isilon Customer Troubleshooting Guide: How to Add a Drive to a Node We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback. Go to Page 9 Add the drive (5) You could have arrived here from: Page 8 – Add the drive (4) Page 9 Does the output indicate that the drive status is EMPTY or REPLACE? No Note the page number that you are currently on. Upload log files and contact Isilon Technical Support, as instructed in Appendix A. No Unable to continue troubleshooting. Isilon Technical Support suggests scheduling an outage window to complete the drive installation. When you are able to continue troubleshooting, return to _______ page 10 of this guide. Yes Determine whether your workflow allows you to power cycle the node now, without affecting production. Are you able to power cycle the node to finish troubleshooting, without affecting production? Yes Go to Page 10 9 - EMC Isilon Customer Troubleshooting Guide: How to Add a Drive to a Node We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback. Add the drive (6) You could have arrived here from: Page 9 – Add the drive (5) Page 10 Power cycle the node. After the node reboots, try to add the drive again by running the following command on the affected node, where <bay#> is the number of the bay that the drive was added to: isi devices -a add -d :<bay#> See __________ Appendix G for example output that indicates the drive replacement is successful. Did the drive add operation succeed? Yes No Note the page number that you are currently on. Upload log files and contact Isilon Technical Support, as instructed in Appendix A. 10 - EMC Isilon Customer Troubleshooting Guide: How to Add a Drive to a Node We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback. Return to Page 8 Appendix A: If you need further assistance Contact EMC Isilon Technical Support If you need to contact Isilon Technical Support during troubleshooting, reference the page or step that you need help with. This information and the log file will help Isilon Technical Support staff resolve your case more quickly. Upload node log files and the screen log file to EMC Isilon Technical Support 1. When troubleshooting is complete, type exit to end your screen session. 2. Gather and upload the node log set and include the SSH screen log file by using the command appropriate for your method of uploading files. If you are not sure which method to use, use FTP. ESRS: isi_gather_info --esrs --local-only -f /ifs/data/Isilon_Support/screenlog.0 FTP: isi_gather_info --ftp --local-only -f /ifs/data/Isilon_Support/screenlog.0 HTTP: isi_gather_info --http --local-only -f /ifs/data/Isilon_Support/screenlog.0 SMTP: isi_gather_info --email --local-only -f /ifs/data/Isilon_Support/screenlog.0 SupportIQ: Copy and paste the following command. Note: When you copy and paste the command into the command-line interface, it will appear on multiple lines (exactly as it appears on the page), but when you press Enter, the command will run as it should. isi_gather_info --local-only -f /ifs/data/Isilon_Support/screenlog.0 --noupload \ --symlink /var/crash/SupportIQ/upload/ftp 3. If you receive a message that the upload was unsuccessful , refer to ___________ article 16759 on the EMC Online Support site for directions on how to upload files over FTP . 11 - EMC Isilon Customer Troubleshooting Guide: How to Add a Drive to a Node We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback. Appendix B: How to use this flowchart Introduction Describes what the section helps you to accomplish. You could have arrived here from: Page # - Page title Page # Note Provides context and additional information. Sometimes a note is linked to a process step with a colored dot. Directional arrows indicate the path through the process flow. Yes No Decision diamond Process step with command: Process step command xyz CAUTION! Caution boxes warn that a particular step needs to be performed with great care, to prevent serious consequences. Go to Page # Optional process step End point Document Shape Calls out supporting documentation for a process step. When possible, these shapes contain links to the reference document. Sometimes linked to a process step with a colored dot. 12 - EMC Isilon Customer Troubleshooting Guide: How to Add a Drive to a Node We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback. Appendix C: Example isi devices output You could have arrived here from: Page 4 – Start troubleshooting Example isi devices output Cluster-1# isi devices Node 1, [ATTN], [SED Node] Bay 1 Lnum 34 Bay 2 Lnum 33 Bay 3 Lnum 17 Bay 4 Lnum 16 Bay 5 Lnum 32 Bay 6 Lnum 15 Bay 7 Lnum 14 Bay 8 Lnum 13 Bay 9 Lnum 31 Bay 10 Lnum 12 Bay 11 Lnum 11 Bay 12 Lnum 10 Bay 13 Lnum 30 Bay 14 Lnum 29 Bay 15 Lnum 28 Bay 16 Lnum 9 Bay 17 Lnum 27 Bay 18 Lnum N/A Bay 19 Lnum 26 Bay 20 Lnum 8 Bay 21 Lnum 25 Bay 22 Lnum 24 Bay 23 Lnum 7 Bay 24 Lnum 6 Bay 25 Lnum 5 Bay 26 Lnum 4 Bay 27 Lnum 3 Bay 28 Lnum 23 Bay 29 Lnum 2 Bay 30 Lnum 22 Bay 31 Lnum 1 Bay 32 Lnum 21 Bay 33 Lnum 0 Bay 34 Lnum 20 Bay 35 Lnum 19 Bay 36 Lnum 18 [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [EMPTY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] SN:Z298KBBF0000921364ED SN:Z298H54B00009410BT4C SN:Z298KARS0000C306EMY3 SN:Z298JZCA000094042SHE SN:Z298JXYC0000C3060CU0 SN:Z298KBJX0000C250BUY4 SN:Z298K0PR00009301VEAY SN:Z298JY3J0000C3063VXX SN:Z298JY81000093078AQ8 SN:Z298JYL90000C2490VXP SN:Z298KB7Z0000C305AF8J SN:Z298JYB800009252UFP1 SN:Z298KB2A0000C2510H0Q SN:Z298JYFY00009301VE79 SN:Z298JYB300009304H7KS SN:Z298KAWP0000S114Z726 SN:Z298KBCF0000C2510J1K SN:N/A SN:Z298K0AX0000C3159JV0 SN:Z298KBFS0000C2516P72 SN:Z298K06E00009252U1EU SN:Z298KB3R00009302X2V7 SN:Z298JYFR000094042SHP SN:Z298K97700009328BY46 SN:Z298KBDM0000C31532VC SN:Z298JYE400009313D75E SN:Z298JY4G0000C2516K68 SN:Z298KB250000C315CUYS SN:Z298JYGH00009314VE8N SN:Z298KBD50000C24982KW SN:Z298K0RE0000C3060DP0 SN:Z298JXZH0000C251CN9W SN:Z298KBL60000C314FPMQ SN:Z298KB1T0000C250GLZG SN:Z298JYDA00009315GEZ1 SN:Z298JYRH00009412TWLC 13 - EMC Isilon Customer Troubleshooting Guide: How to Add a Drive to a Node We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback. /dev/da1 /dev/da2 /dev/da18 /dev/da19 /dev/da3 /dev/da20 /dev/da21 /dev/da22 /dev/da4 /dev/da23 /dev/da24 /dev/da25 /dev/da5 /dev/da6 /dev/da7 /dev/da26 /dev/da8 N/A /dev/da9 /dev/da27 /dev/da10 /dev/da11 /dev/da28 /dev/da29 /dev/da30 /dev/da31 /dev/da32 /dev/da12 /dev/da33 /dev/da13 /dev/da34 /dev/da14 /dev/da35 /dev/da15 /dev/da16 /dev/da17 Appendix D: Example isi job status output with FlexProtect job running You could have arrived here from: Page 5 – Add the drive Example isi job status output with FlexProtect job running In this example, the drive in bay 36 was SmartFailed. The output shows that the FlexProtect job is running. See the bold text in the output: Cluster-1# isi job status The job engine can temporarily only run FlexProtect and FlexProtectLin jobs. Coordinator: 1 Connected: True Disconnected Nodes: Down or Read-Only Nodes: False Statistics Ready: True Cluster Is Degraded: True Run Jobs When Degraded: False Running and queued jobs: ID Type State Impact Pri Phase Running Time ---------------------------------------------------------562 FlexProtect Running Medium 1 2/6 3s ---------------------------------------------------------Total: 1 Recent finished jobs: ID Type State Time ----------------------------------------------------552 WormQueue Succeeded 2015-12-06T02:00:05 553 ShadowStoreProtect Succeeded 2015-12-06T04:00:13 554 ShadowStoreProtect Succeeded 2015-12-06T20:00:11 555 WormQueue Succeeded 2015-12-07T02:00:02 556 ShadowStoreProtect Succeeded 2015-12-07T04:00:10 557 ShadowStoreProtect Succeeded 2015-12-07T20:00:08 558 WormQueue Succeeded 2015-12-08T02:00:30 559 ShadowStoreProtect Succeeded 2015-12-08T04:00:07 560 ShadowStoreProtect Succeeded 2015-12-08T20:00:05 561 WormQueue Succeeded 2015-12-09T02:00:27 ----------------------------------------------------Total: 10 Continued on the ________ next page. 14 - EMC Isilon Customer Troubleshooting Guide: How to Add a Drive to a Node We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback. Appendix D: Example isi job status output with FlexProtect job running (2) You could have arrived here from: Appendix D: Example isi job status with FlexProtect job running To confirm that the bay status is SMARTFAIL, run isi devices. See the bold text in the output: Cluster-1# isi devices Node 1, [ATTN], [SED Node] Bay 1 Lnum 34 [HEALTHY] Bay 2 Lnum 33 [HEALTHY] Bay 3 Lnum 17 [HEALTHY] (truncated) Bay 14 Lnum 29 [HEALTHY] Bay 15 Lnum 28 [HEALTHY] Bay 16 Lnum 9 [HEALTHY] Bay 17 Lnum 27 [HEALTHY] Bay 18 Lnum N/A [EMPTY] Bay 19 Lnum 26 [HEALTHY] Bay 20 Lnum 8 [HEALTHY] (truncated) Bay 35 Lnum 19 [HEALTHY] Bay 36 Lnum 18 [SMARTFAIL] SN:Z298KBBF0000921364ED SN:Z298H54B00009410BT4C SN:Z298KARS0000C306EMY3 /dev/da1 /dev/da2 /dev/da18 SN:Z298JYFY00009301VE79 SN:Z298JYB300009304H7KS SN:Z298KAWP0000S114Z726 SN:Z298KBCF0000C2510J1K SN:N/A SN:Z298K0AX0000C3159JV0 SN:Z298KBFS0000C2516P72 /dev/da6 /dev/da7 /dev/da26 /dev/da8 N/A /dev/da9 /dev/da27 SN:Z298JYDA00009315GEZ1 SN:Z298JYRH00009412TWLC /dev/da16 /dev/da17 15 - EMC Isilon Customer Troubleshooting Guide: How to Add a Drive to a Node We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback. Appendix E: Example isi job status output with FlexProtect job completed You could have arrived here from: Page 5 – Add the drive Example isi job status output with FlexProtect job completed To confirm the FlexProtect job is running, run isi devices. See the bolded line in the output: Cluster-1# isi job status The job engine is running. No running or queued jobs. Recent finished jobs: ID Type State Time ----------------------------------------------------553 ShadowStoreProtect Succeeded 2015-12-06T04:00:13 554 ShadowStoreProtect Succeeded 2015-12-06T20:00:11 555 WormQueue Succeeded 2015-12-07T02:00:02 556 ShadowStoreProtect Succeeded 2015-12-07T04:00:10 557 ShadowStoreProtect Succeeded 2015-12-07T20:00:08 558 WormQueue Succeeded 2015-12-08T02:00:30 559 ShadowStoreProtect Succeeded 2015-12-08T04:00:07 560 ShadowStoreProtect Succeeded 2015-12-08T20:00:05 561 WormQueue Succeeded 2015-12-09T02:00:27 562 FlexProtect Succeeded 2015-12-09T02:07:15 ----------------------------------------------------Total: 10 To confirm the drive is ready to replace, run isi devices. See the bolded line in the output: Cluster-1# isi devices Node 1, [ATTN], [SED Node] Bay 1 Lnum 34 Bay 2 Lnum 33 Bay 3 Lnum 17 (truncated) Bay 14 Lnum 29 Bay 15 Lnum 28 Bay 16 Lnum 9 Bay 17 Lnum 27 Bay 18 Lnum N/A Bay 19 Lnum 26 Bay 20 Lnum 8 (truncated) Bay 35 Lnum 19 Bay 36 Lnum 18 [HEALTHY] [HEALTHY] [HEALTHY] SN:Z298KBBF0000921364ED SN:Z298H54B00009410BT4C SN:Z298KARS0000C306EMY3 /dev/da1 /dev/da2 /dev/da18 [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [EMPTY] [HEALTHY] [HEALTHY] SN:Z298JYFY00009301VE79 SN:Z298JYB300009304H7KS SN:Z298KAWP0000S114Z726 SN:Z298KBCF0000C2510J1K SN:N/A SN:Z298K0AX0000C3159JV0 SN:Z298KBFS0000C2516P72 /dev/da6 /dev/da7 /dev/da26 /dev/da8 N/A /dev/da9 /dev/da27 [HEALTHY] [REPLACE] SN:Z298JYDA00009315GEZ1 SN:Z298JYRH00009412TWLC /dev/da16 /dev/da17 16 - EMC Isilon Customer Troubleshooting Guide: How to Add a Drive to a Node We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback. Appendix F: Example isi devices output showing a PREPARING or a HEALTHY drive You could have arrived here from: Page 8 – Add the drive (4) Example isi devices output showing a PREPARING drive To confirm the drive is formatting, run isi devices. See the bold text in the output: Cluster-1# isi devices Node 1, [ATTN], [SED Node] Bay 1 Lnum 34 Bay 2 Lnum 33 Bay 3 Lnum 17 (truncated) Bay 14 Lnum 29 Bay 15 Lnum 28 Bay 16 Lnum 9 Bay 17 Lnum 27 Bay 18 Lnum N/A Bay 19 Lnum 26 Bay 20 Lnum 8 (truncated) Bay 35 Lnum 19 Bay 36 Lnum 18 [HEALTHY] [HEALTHY] [HEALTHY] SN:Z298KBBF0000921364ED SN:Z298H54B00009410BT4C SN:Z298KARS0000C306EMY3 /dev/da1 /dev/da2 /dev/da18 [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [EMPTY] [HEALTHY] [HEALTHY] SN:Z298JYFY00009301VE79 SN:Z298JYB300009304H7KS SN:Z298KAWP0000S114Z726 SN:Z298KBCF0000C2510J1K SN:N/A SN:Z298K0AX0000C3159JV0 SN:Z298KBFS0000C2516P72 /dev/da6 /dev/da7 /dev/da26 /dev/da8 N/A /dev/da9 /dev/da27 [HEALTHY] [PREPARING] SN:Z298JYDA00009315GEZ1 SN:Z298JYRH00009412TWLC /dev/da16 /dev/da17 Example isi devices output showing a HEALTHY drive To confirm the drive is ready to replace, run isi devices. See the bold text in the output: Cluster-1# isi devices Node 1, [ATTN], [SED Node] Bay 1 Lnum 34 Bay 2 Lnum 33 Bay 3 Lnum 17 (truncated) Bay 14 Lnum 29 Bay 15 Lnum 28 Bay 16 Lnum 9 Bay 17 Lnum 27 Bay 18 Lnum N/A Bay 19 Lnum 26 Bay 20 Lnum 8 (truncated) Bay 35 Lnum 19 Bay 36 Lnum 18 [HEALTHY] [HEALTHY] [HEALTHY] SN:Z298KBBF0000921364ED SN:Z298H54B00009410BT4C SN:Z298KARS0000C306EMY3 /dev/da1 /dev/da2 /dev/da18 [HEALTHY] [HEALTHY] [HEALTHY] [HEALTHY] [EMPTY] [HEALTHY] [HEALTHY] SN:Z298JYFY00009301VE79 SN:Z298JYB300009304H7KS SN:Z298KAWP0000S114Z726 SN:Z298KBCF0000C2510J1K SN:N/A SN:Z298K0AX0000C3159JV0 SN:Z298KBFS0000C2516P72 /dev/da6 /dev/da7 /dev/da26 /dev/da8 N/A /dev/da9 /dev/da27 [HEALTHY] [HEALTHY] SN:Z298JYDA00009315GEZ1 SN:Z298JYRH00009412TWLC /dev/da16 /dev/da17 17 - EMC Isilon Customer Troubleshooting Guide: How to Add a Drive to a Node We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback. Appendix G: Example output from the isi devices -a add -d :<bay#> command You could have arrived here from: _____________________ Page 7 – Add the drive (3) ______________________ Page 10 – Add the drive (6) Example output from the isi devices -a add -d :<bay#> command Example output when reading a reinserted, or stopfailed, drive: Cluster-1# isi devices -a add -d :24 Initiating add on bay 24 . . . . . . . . . . . !! The add operation succeeded. A healthy drive was found in bay 24. Example output when adding a blank drive: Cluster-1# isi devices -a add -d :36 Initiating add on bay 36 . . . . . . . . . . . !! The add operation failed. An unrecognized drive was found in bay !! 36. To format this drive and add it to the file system, run 'isi !! devices -a format' on this drive and follow the instructions !! provided. Cluster-1# isi devices -a format -d :36 !! !! !! !! This drive does not appear to be pre-formatted with OneFS. Formatting this drive will destroy any existing data and add it to OneFS. Do you wish to continue formatting this drive? (yes, [cancel]) >>> yes 'format' drive action initiated. This process could take a few minutes. To monitor the progress of this action, run 'isi devices'. When the drive appears green, the 'format' and purposing actions have completed successfully. Example output when the drive add operation fails, or when there is no drive detected: Cluster-1# isi devices -a add -d :18 Initiating add on bay 18 . . . . . . . . . . . !! The add operation failed. No drive was found in bay 18. Verify that !! a functional drive is physically present in the bay and try again. 18 - EMC Isilon Customer Troubleshooting Guide: How to Add a Drive to a Node We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback. Copyright © 2015 EMC Corporation. All rights reserved. Published in USA. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. EMC², EMC, and the EMC logo are registered trademarks or trademarks of EMC Corporation in the United States and other countries. All other trademarks used herein are the property of their respective owners. For the most up-to-date regulatory document for your product line, go to EMC Online Support (https://support.emc.com). 19 - EMC Isilon Customer Troubleshooting Guide: How to Add a Drive to a Node We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback.