The FibreVault or fibre channel RAID enclosure is designed to remain powered on and hot-repairable; that is, you can replace some components without powering off the storage system or the host(s) to which it is attached. Each FibreVault or fibre channel RAID enclosure compartment should contain a component (drive, power supply, and so on) or filler panel to ensure EMI compliance and proper air flow over the components.
While the FibreVault or fibre channel RAID enclosure is powered on, you can remove a disk module, redundant link control card (LCC), or filler, but you should replace it as soon as possible with a replacement component or a filler. You can also remove the fan assembly while an enclosure is powered on, but you must replace it within two minutes to keep the disk drive modules powered on. If you do not replace the fan module within 2 minutes after removing it from an operating enclosure, the enclosure automatically powers off the drives. Do not remove a faulty component until you have a replacement component available. Contact your Silicon Graphics service or sales representative to order additional parts.
This chapter includes the following main sections:
Use only Silicon Graphics FC-AL disk modules. Always confirm the type of fibre drive (RAID or non-RAID) before you install it. Each drive module has an identifying sticker with its part number.
Table 4-1 provides descriptions and part numbers for the fibre drives.
Table 4-1. Fibre Channel Disk Drive Descriptions
Fibre Drive Type | Drive Capacity | Drive Speed | Sector Sizing | Part Number |
|---|---|---|---|---|
Non-RAID | 9.1 GB | 7200 RPM | 512-byte sectors | 9470140 |
RAID | 8.8 GB | 7200 RPM | 520-byte sectors | 9470192 |
Non-RAID | 9.1 GB | 10,000 RPM | 512-byte sectors | 9470282 |
RAID | 8.8 GB | 10,000 RPM | 520-byte sectors | 9470281 |
Non-RAID | 18 GB | 7200 RPM | 512-byte sectors | 9470255 |
RAID | 17.8 GB | 7200 RPM | 520-byte sectors | 9470257 |
 | Note: Always consult with your service provider before replacing a disk with another that has a different capacity or speed rating (different part number). |
Be aware of the following information when adding or replacing disks:
RAID and non-RAID fibre drives are not compatible and may not be mixed in the same enclosure.
A lower capacity RAID drive must not be used as a replacement disk or hot spare in an array of higher capacity disks. For example, you must never install an 8.8 GB RAID drive as a hot spare or replacement in an enclosure of 17.8 GB disks.
Installing a lower speed (RPM) drive in an array of faster RPM disks causes the entire array bandwidth (read and write speed) to slow to the lower speed capacity.
 | Caution: The disk modules have no ESD shielding or protective case. Do not stack them on top of each other before installing them. Do not drop, jar, or handle the drives roughly. Use standard ESD precautions as listed in the instructions and always leave disks in an anti-static container until ready for installation. |
Do not replace more than one disk module at a time unless the enclosure is powered off.
Special rules apply when you replace a fibre channel RAID disk module in either of the following groups:
The first three disk modules in the first fibre channel RAID enclosure in a chain, or in the only enclosure in the array, contain licensed internal code (LIC). These disk modules, with disk IDs 00, 01, and 02, are known as database disk modules.
If all three are removed at the same time when the array is powered on, contact with the array is lost. If you remove them with the array powered off, label these as you remove them, because they must be reinstalled in their original positions.
If the array uses write caching, the disk modules that the array uses for its cache vault are 00 through 08.
If a cache vault disk module fails, the array dumps its write cache image to the remaining modules in the vault. Then it writes all dirty (modified) pages to disk and disables write caching. Write caching remains disabled until a replacement disk module is inserted and the array rebuilds the LUN with the replacement module in it. The Write Cache State field in the Cache section of the SP Information window (see the section on SP cache information in Chapter 4 of the Origin FibreVault and Fibre Channel RAID Administrator's Guide ) tells whether array write caching is enabled or disabled.
Before adding, removing, or replacing disk modules, note the following:
Fill slots starting with the lowest address (0) and add disks until you reach the highest applicable address (left to right).
In a fibre channel RAID enclosure, drives 0, 1, and 2 contain licensed internal code. They should be replaced one at a time.
If you want to replace all the disk drive modules in a RAID LUN, you must have your service provider update the licensed internal code and PROM code. This procedure requires a service call.
All fibre channel RAID disk enclosures and their expansion enclosures attached to an FC-AL should have at least five disk modules installed.
If you must configure a non-RAID FibreVault with only one drive, it must be either the last enclosure on the loop or the only enclosure on the loop.
The non-RAID FibreVault enclosure must always have a disk module installed in either slot 0 or 1. Access to enclosure-specific environmental information depends on at least one of these slots being populated with a disk.
A FibreVault or fibre channel RAID enclosure cannot remain connected to an FC-AL if it has no drives installed.
Note that a RAID 3 LUN can use only one hot spare in case of disk failure. If a second hot spare is available and a second disk module in the RAID 3 LUN fails, the LUN does not use the second hot spare.
In a fibre channel RAID enclosure, removing the wrong disk module can introduce an additional fault that shuts down the LUN containing the failed drive. Before removing a disk module, check the module's amber check or fault light. If it is illuminated, or if both LEDs are off, the host is bypassing it, indicating a failure.
To check further on the status of a disk module:
Use the fccli GETSTATUS; command for non-RAID (FibreVault) disks.
For a RAID disk module, use the command ssmcli -d device getdisk -state or the GUI.
For full details on these commands, see the Origin FibreVault and Fibre Channel RAID Administrator's Guide .
To add or replace a disk module, follow these steps:
Be sure you understand the rules and limits for replacing fibre channel RAID and non-RAID disks. See “General Guidelines for Adding or Removing Disks” and “Confirming a Failed Disk Module”.
Have ready the proper replacement disk module.
Back up any information on the disk(s) you are replacing as needed.
Use one of the following methods to take the disk offline and prepare for installation:
Tell the host that you are removing a (non-RAID) disk module and take it off line using the fccli REMOVE option. The Origin FibreVault and Fibre Channel RAID Administrator's Guide has details for using the line commands.
Shutdown the host and fibre channel enclosure systems.
If a failed disk is part of a RAID LUN, you might need to unbind the LUN before the replacement under certain circumstances. See the Origin FibreVault and Fibre Channel RAID Administrator's Guide .
Open the FibreVault or fibre channel RAID enclosure's front door by inserting the key in the door's latch and turning the key to the unlocked icon position.
Remove the key. Push the door's latch, and pull the door down until it is fully open.
If you accidently pull the door down with so much force that it becomes unhinged, snap it back on the hinges.
Attach the clip of the ESD wristband to the FibreVault or fibre channel RAID enclosure chassis and put the wristband around your wrist with the metal button against your skin.
Caution: The enclosure has no mechanism to prevent you from removing a disk module while the drive is spinning. Do not remove a module unless its green active light is off and its amber check light is on (or the entire array may shutdown). Locate the disk or filler module to be removed from the target slot. This module's yellow LED is lit (or both LEDs are off) to show that the host is bypassing it. The green activity LEDs of the other disk modules in the loop blink at a regular interval as part of a background loop verification process.
Note: Do not remove a disk module or filler unless you have its replacement ready to install. Do not remove more than one disk module at a time. Grasp the disk module's handle so that your thumb is on its latch at the top of the handle.
Push the latch and slowly pull the module from the slot, supporting it with your other hand, see Figure 4-1. The disk module's active and check LEDs turn off. If power is on and if the disk you removed was bound in a RAID LUN, wait for the active LEDs of the other disk modules in the loop to resume blinking at a regular interval, which indicates that the driver has completed any necessary loop discovery.
Grasp the handle of the new disk module and align the module with the guides in the empty slot. The module's thumb latch should be at the top.
Slowly push the disk module into the slot until it clicks into place. The disk module's active LED flashes and then remains on (if power is on).
If power is on and you have replaced a RAID disk module, the disk module's activity LED blinks; it may also flash rapidly.
When the replacement RAID disk module functions normally as part of the LUN, its green LED blinks at the same interval as the rest of the disk modules on the loop. At this point, you can replace another RAID disk module if necessary.
Note: If the fault LED stays on after you insert the disk module, remove it and reinsert it. If it remains on, use another replacement disk module. If you are replacing a failed non-RAID disk module, its amber LED remains lit until you use the fccli INSERT command as explained in Step 14.
Remove and store the ESD wristband when all replacement procedures are complete.
When you have replaced or added the disk module, use one of the following procedures to bring the disk online.
Power the host and the fibre channel enclosure back on if they were powered off during the installation procedure.
Use the fccli INSERT option in the command line interface to inform the host (that owns the loop) of its presence, see the Origin FibreVault and Fibre Channel RAID Administrator's Guide . The fault LED remains lit until the drive is online.
Push the fibre enclosure's front door up until it latches into place. Note that the maximum number of times a disk module can be inserted and removed is 500. Exceeding this limit may damage the SCA connector on the disk module or the connector on the enclosure midplane.
The drives read their FC-AL physical address only at power-on or when the drive is reset. As a result, each enclosure address is set when installed in the Fibre rack while the power is off; do not change the address while the enclosure's power is on.
Set the enclosure address for each FibreVault enclosure by using a pencil or ballpoint pen to move the down or up switch.
The LED lights up for the number you set. FibreVault enclosures are numbered 0-10; use only those numbers. Figure 4-2 shows the address switch. When you are finished setting the FibreVault address, push the enclosure's front door closed until it latches into place. If desired, lock the enclosure's front door.
Valid address ranges for fibre channel RAID (see Figure 4-3) are 0 through 109 (decimal). This equals 0 through 6D hexadecimal (0 on both switches through 6 on the left switch and D on the right switch). Each SP must have a unique address set using the switches.
The fibre channel RAID enclosure address the loop sees is determined and set on the system processor (SP) board (see Figure 4-4 for the location of each SP). Each SP must have a unique fibre channel front-end address. Table 4-2 shows the maximum disk numbering for FibreVault (fibre channel RAID is lower).
Table 4-2. FibreVault or Fibre Channel RAID Disk Slot Numbering
Fibre Enclosure ID | Disk Slot Number |
|---|---|
0 | 0-9 |
1 | 10-19 |
2 | 20-29 |
3 | 30-39 |
... | ... |
9 | 90-99 |
10 |
The disk slots in a FibreVault or fibre channel RAID enclosure are numbered 0-9. Disk modules are always numbered based on the ID of the FibreVault or fibre channel RAID enclosure.
If you are replacing the fan assembly in an operating storage system, you have a total of two minutes in which to complete the process. Do not remove a fan assembly unless you have its replacement ready to install in the enclosure.
| Note: An operating drive assembly automatically shuts down after 2 minutes if the fan assembly is not replaced, or if more than one fan fails. If the disk modules do shut down under these circumstances, the green disk module LEDs and the green (active) SP LEDs remain illuminated. Even though the disk modules and SP(s) are shut down, they are not faulted. The disk modules and SP(s) automatically power back on when a functional fan assembly is installed. |
Follow these steps:
Have the replacement fan assembly at hand, unwrapped and ready to insert.
Locate the enclosure with the faulty fan assembly; the fan assembly check LED is on.
Grasp the fan assembly's handles (squeeze latches) on one side and pull them inward towards the middle of the fan assembly until you feel them loosen.
While keeping a thumb on each of the already loose latches, grasp the set of squeeze latches on the opposite side of the assembly with your fingers and pull them in toward the middle of the assembly and outward, see Figure 4-5.
Caution: Do not leave the fan assembly out of the fibre enclosure for more than two minutes if it is running. Once the fan assembly is disconnected from the power supply, its check LED turns off. The enclosure's front panel check LED and the cooling check LED on each power supply start flashing.
Orient the new fan assembly. Although the fan assembly can attach to the enclosure with either side up, for a consistent image with multiple FibreVault or fibre channel RAID enclosures, install it with the status light in the upper right corner, as shown in Figure 4-6.
Squeezing the fan assembly latches toward each other, align and push the assembly forward into the chassis. The fan assembly snaps onto the power supply.
The power supply modules are located behind the fan unit in the Origin FibreVault or fibre channel RAID enclosure, as was shown in Figure 4-5. This procedure's steps are only fully applicable when the vault has dual (redundant) power supplies. If there is only one supply installed, the FibreVault or fibre channel RAID enclosure shuts down if the supply fails.
This section contains the following subsections:
Do not remove a power supply or filler unless you have its replacement ready to install.
Follow these steps:
Have the new (replacement or additional) power supply at hand, unwrapped and ready to insert. Have space available to receive the fan unit and the power supply you are replacing.
Make sure the lever on the new power supply is in the unlock position, as shown in Figure 4-7.
If you are replacing the power supply in an operating storage system, you have a total of two minutes in which to complete the process. This is the maximum amount of time allowed for the fan assembly to be out before an automatic shutdown of the enclosure is executed.
If two minutes is inadequate time for you to make the exchange, consider shutting down the enclosure first. See “Shutting Down the Fibre Channel Rack and Disk Enclosures” in Chapter 3.
Identify the FibreVault or fibre channel RAID enclosure containing the power supply with its check light on. The light is visible through the fan assembly.
Attach the clip of the ESD wristband to the enclosure chassis and put the wristband around your wrist with the metal button against your skin.
Grasp the fan assembly by the handles on either side and pull it out to expose the power supply (bay), as shown in Figure 4-5.
Caution: Do not leave the fan assembly out of the enclosure for more than two minutes or the vault goes through an automatic thermal shutdown. Turn off the faulty power supply by pressing the on/off switch. The power supply's active and check LEDs turn off.
Unplug the AC line cord from the faulty power supply (see Figure 4-8).
Remove the faulty power supply (see Figure 4-9):
For the top power supply, put your right palm against the right side of the power supply handle; with your thumb, push the power supply's latch up and then right to the unlock position.
For the bottom power supply, put your left palm against the left side of the power supply handle; with your thumb, push the power supply's latch down and then left to the unlock position.
Pulling the handle, slide the filler or power supply out of the bay, supporting it with your other hand, as shown in Figure 4-10.
To insert a new power supply into a FibreVault, follow these steps:
Make sure the lever on the new power supply is in the unlock position, as was shown in Figure 4-7.
Orient the new power supply, as shown in Figure 4-9, and slide it into the bay.
Move the lever to the lock position, as shown in Figure 4-9.
Plug the AC line cord into the new power supply, as shown in Figure 4-8.
Turn on the power supply by pressing the on/off switch.
The power supply's active LED turns on.
Reattach the fan assembly by squeezing the latches toward each other and pushing the assembly into the chassis.
Do not leave the fan assembly out of the fibre enclosure for more than two minutes if you want the enclosure to keep running.
You can install the fan assembly in either orientation. However, for a consistent image with multiple FibreVault or fibre channel RAID enclosures, install it with the status light in the upper right corner as shown in Figure 4-6.
To remove, replace, or add a power supply to the fibre channel RAID enclosure, you must first remove the disk fan module (see “Replacing a Fan Assembly”). As noted in the fan replacement instructions, if you leave the fan assembly out of the fibre enclosure for more than two minutes, the disks shut down automatically. In the case of the fibre channel RAID enclosure, the system processor board(s) shut down also. Unless you are experienced at power supply removal or replacement, it is likely that the procedure will require more than two minutes to complete.
Handle the power supply with care and use all standard ESD precautions. Use the following steps if you are removing and replacing an existing power supply:
After removing the fan module, turn off the supply's power by pressing the rocker switch to the 0 position.
Unplug and remove the supply's AC line connector (see Figure 4-11).
Unlatch the top power supply by moving the handle up and to the far right.
Unlatch the bottom power supply by moving the handle down and to the left.
Pull the supply straight out using the handle and both hands to steady it (see Figure 4-12).
After confirming the replacement supply's latch is in the open position, install the new supply by aligning it and sliding it straight in.
Note: The orientation of the bottom supply is inverted (upside down) from that of the top. Push the supply into place until you see the latching lever move toward the latched position (see Figure 4-13).
Push the latch all the way into the locked position.
Plug the AC connector into the supply and route the cord along the power supply, then under the LCC position so that the cord exits the chassis without blocking the installation of the fan module.
Set the supply's rocker switch to the On position.
Install the fan module, see “Replacing a Fan Assembly”.
This section explains removing and replacing an LCC or filler panel.
| Note: Do not remove an LCC or filler unless you have its replacement ready to install. |
Follow these steps to remove an existing LCC or filler panel and install a new LCC:
Have ready the new or replacement LCC.
Caution: When handling an LCC, be careful not to touch any of its electronic circuits or components. Remove the LCC from its packaging only when ready to install it. Locate the FibreVault or fibre channel RAID enclosure containing the LCC with its check LED illuminated.
Attach the clip of the ESD wristband to the fibre enclosure chassis and put the wristband around your wrist with the metal button against your skin.
Unplug the cable(s) (as applicable) connected to the faulty LCC, then pull up on the latch of the LCC or filler panel.
Grasp the LCC and slowly pull it out of the chassis, as shown in Figure 4-14. The LCC's active and check LEDs turn off if the enclosure is powered on.
Wait ten seconds to allow the fibre channel system FLARE code time to recognize that the LCC was removed.
Align the new LCC with the guides in the empty bay.
Slide the LCC into the slot, as shown in Figure 4-15.
Push down on the LCC's latch. The LCC's active LED illuminates if the enclosure is powered on.
Attach the LCC cables.
Remove and store the ESD wristband.
