Mass storage for the Silicon Graphics fibre channel option is provided by fibre channel arbitrated loop (FC-AL) disk drive modules. FC-AL disk modules have different interface cards from standard SCSI-2 disk modules, with 40-pin (instead of 80-pin) connectors. The disk modules run the SCSI-3 protocol and support Class 3 (connectionless, or datagram) service.
Up to ten disk modules can be contained in each fibre enclosure. Up to 11 Origin FibreVault enclosures can be contained in a fibre channel rack, for a maximum of 110 (non-RAID) fibre drives. A single fibre channel RAID loop supports a lesser number of drives. Technically, a maximum of five fibre channel RAID enclosures can fit in the rack. To provide room for expansion disk enclosures and other options, the recommended maximum is three enclosures in the rack.
Although each fibre channel option card can technically address up to 126 disk drives, there is no support for extending the fibre cable to another rack or cabinet. Therefore, the maximum amount of drives supported by a single fibre channel connection (loop) on a fibre board is equal to the maximum number (110) of drives installed in a single rack.
The FC-AL disk modules and the server chassis SCSI disk modules are not interchangeable and cannot be substituted for each other. Likewise, RAID and non-RAID fibre disk modules use different sectoring sizes and are not interchangeable.
This chapter includes the following main sections:
The non-RAID FibreVaults come in deskside (tower) and rackmount versions. Figure 3-1 shows the rear components on a rackmount FibreVault.
Figure 3-2 shows the deskside FibreVault enclosure with its locking wheels and anti-tip “legs.” You can lock the front wheels by pushing the locking lever on each front wheel down. Lift the locks up to allow the system to roll.
 | Caution: The deskside tower's anti-tip legs should always stay attached to the enclosure. The legs stabilize the tower unit and prevent it from tipping over. |
Figure 3-3 shows a rear view of the rackmount FibreVault components (fan module removed).
Figure 3-4 shows the FibreVault enclosure front panel.
This section explains more about the FibreVault enclosure's components:
The link controller card (LCC), disk cooling fan assembly, and enclosure power supply described in the next three sections are generic components used in both the FibreVault and fibre channel RAID expansion enclosures. The only difference between a FibreVault and a fibre channel RAID expansion enclosure is the sector size in the disk modules.
The listed components perform the same function in both types of enclosures and are completely interchangeable. The fibre channel RAID expansion enclosure is exactly the same mechanically as the FibreVault enclosure.
 | Note: Never exchange disk modules between a fibre channel RAID expansion enclosure and a FibreVault. |
The link controller card (LCC) controls data flow to and from the disk modules in FibreVault. The LCC also controls data flow between the FibreVault enclosure and the outside world (including additional linked enclosures) and monitors the enclosure environment.
The FibreVault enclosure comes standard with one LCC. An optional second LCC is required for high-availability or FailSafe operations. Note that special high-availability support software is also required. The LCC is visible at the back of the enclosure, as was shown in Figure 3-3. Figure 3-5 shows features of the LCC front panel.
Each LCC supports and controls one FC-AL. You can configure a FibreVault enclosure with one or two LCCs to provide one or two FC-ALs.
The LCCs in an enclosure are connected to host bus adapters in a server or to LCCs on other enclosures. The LCC receives and terminates the incoming FC-AL signal from either the server's host bus adapter or from another enclosure. It passes the input signal to the disk drives in the enclosure, and then drives the output signal through the cabling to the next fibre enclosure in the loop. The FC-AL cabling is not explicitly configured as a loop (that is, a long return from the last fibre enclosure to the server), but instead as a full–duplex point–to–point connection, with the last fibre enclosure in the chain closing the loop on its LCC.
The fibre channel RAID enclosure receives data from the host through the system processor (SP) port and passes the loop through the expansion (EXP) port on its LCC to any fibre channel RAID expansion enclosure(s). The FibreVault LCC's primary port (PRI) receives data from a host. Its expansion port (EXP) is used for sending data to the next in a series of linked FibreVaults.
The LCC status LEDs function as follows:
active (green): The LCC is operating.
check (amber): If the amber LED stays on, a fault occurred either in the LCC or an FC-AL connection.
LCC's have two physical methods of communicating status information or receiving control information from the host (or RAID processor) and target enclosure.
A serial channel is duplexed over fiber cable in the RAID configuration. This permits communication with additional RAID enclosures with or without drives being present even when the loop is not being used.
In non-RAID (FibreVault) configurations, the host communicates through the disk interface using the SFF 8067 protocol defined in the fibre channel standards.
The protocols that the LCC uses enable the host to poll fibre enclosure status and send commands that control the port bypass circuits on both LCCs, the enclosure, and the disk–module check lights. LCCs do not communicate with or control each other.
Figure 3-6 shows features of the rear fan assembly used on the enclosures.
The fan assembly, which contains three fans, cools the entire FibreVault or fibre channel RAID expansion enclosure. Fibre channel RAID enclosures use an additional (SP) fan assembly that is located on the front of the fibre channel RAID enclosure.
| Caution: To prevent thermal shutdown of the disk drive system, never remove the fan assembly for more than two minutes while the enclosure is powered on. |
The fan assembly connects directly to both power supplies, either of which can power it. The fans operate at 9 volts during normal operation to minimize acoustic noise. To compensate for decreased air flow when a fan fails, the voltage increases to 12 volts, resulting in higher acoustic noise.
If the amber check LED is lit, a fan assembly fault has occurred. You can replace the fan assembly while the enclosure is powered on as long as the process is completed in less than two minutes.
If the fan assembly is removed for more than two minutes, the FC-AL interconnect system stays powered on, but the disk modules are automatically powered off. The disk modules power on again when a functioning drive fan assembly is installed in the system. “Replacing a Fan Assembly” in Chapter 4 contains complete instructions for replacing a fan assembly.
Each fibre enclosure comes standard with one power supply and accommodates an optional second power supply. Figure 3-7 shows an example power supply.
The power supplies are located behind the fan assembly.
Note that the amber cooling check LED on the supply lights whenever the fan module is removed.
Each power supply is an autoranging, power–factor–corrected, multioutput offline converter. Each supply supports a fully configured (10-disk) fibre enclosure and shares current with the other supply if it is present.
The power supply status LEDs function as follows:
Active (green): The power supply is operating.
Power supply check (amber): If the LED stays on, a power supply fault has occurred.
Cooling check (amber): If the LED blinks, a cooling fault has occurred.
If a cooling check fault continues for more than two minutes, the drive modules are powered off automatically.
The drive and LCC voltage lines have individual soft–start switches with short-circuit current–limit capability to protect the disk drives and link control cards if you install them while the vault is powered on.
Each power supply has its own on/off switch and its own power cord for connection to separate power sources, which is particularly useful for high-availability systems. Voltage is supplied independently to each disk module. One shorted-out disk module does not affect the other disk modules in the enclosure. If a disk module is added while the enclosure is operating, the voltage is automatically increased gradually to accommodate the new disk module.
The power supplies or their fillers are inverted with respect to each other in the chassis. The chassis is designed to prevent incorrect insertion of the power supplies. In a highly available configuration, you can add or remove the power supply while the enclosure is powered on.
The FibreVault midplane features:
40-pin SCA-style unitized disk drive connector with blind mate and hot plug features
Dual-port disk interface
Slot-dependent disk addressing for each FibreVault enclosure in a rack
Isolated DC power for each disk
3.3-volt support (not currently provided by power supply)
Five disk LRCs in each direction
The fibre channel RAID enclosure shares a number of functionally similar components with the FibreVault enclosures. These components include the link controller card (LCC) and power supply.
| Caution: Although the function and appearance of the LCCs and power supplies are similar between FibreVault and fibre channel RAID enclosures, they are not interchangeable. |
These components are described in the section “FibreVault Components”.
The disk fan module is interchangeable with any of the enclosures.
This section concentrates on components that are unique to the fibre channel RAID enclosure.
Figure 3-9 shows the fibre channel RAID deskside tower system. It holds one fibre channel RAID disk enclosure see Figure 3-8, and one RAID expansion FibreVault enclosure, (see Figure 3-4). Each RAID tower unit requires a fibre channel connection. The fibre channel RAID and FibreVault RAID expansion enclosures mount vertically in the deskside tower.
The combined maximum capacity of the RAID deskside tower is 20 fibre channel RAID disks. Figure 3-9 shows the fibre channel RAID deskside unit.
Each fibre channel RAID enclosure has one or (optionally) two storage processor (SP) board assemblies. Figure 3-10 shows the fibre channel RAID enclosure's standard and optional storage processors (SPs). Figure 3-11 shows an example of the storage processor board (SP) and its components.
The storage processor's fan module is located at the front of the fibre channel RAID enclosure. It is a completely different module from that used in the upper rear section of the enclosure to cool the disk assembly.
Figure 3-12 illustrates the fibre channel RAID storage processor (SP) fan module.
The fibre channel rack accommodates up to 11 FibreVault (or five fibre channel RAID) enclosures. It has two power distribution units with two separate power inlets so that optional dual power supplies on the fibre enclosures can be connected to separate power sources.
Enclosures within the rack equipped with optional (external) MIAs can be cabled to a host up to 300 meters away. Up to 11 FibreVault enclosures within the rack can be daisy-chained to one port (loop) on a host's FC XIO board. Fibre channel RAID and fibre channel RAID expansion enclosures can also be daisy-chained within the rack.
A fibre channel rack fully loaded with 11 FibreVault enclosures, each with ten disk modules, weighs about 1100 lbs (about 500 kg).
Figure 3-13 diagrams a typical FC storage subsystem.
Hosts are connected to storage subsystems by full-duplex cabling, which can be copper or (optionally) optical. With full duplex cabling, each port provides both its input and output on a single connector. The last storage box in a chain automatically provides a loopback function (similar to connecting a SCSI terminator) which completes the loop. The LCC automatically shunts the open port if no signals from an additional connected enclosure are present.
If you need higher availability (especially with RAID caching), you may use the standby power supply (SPS) option in the rack or deskside RAID systems. Figure 3-14 shows features of the SPS panel. Table 3-1 explains the SPS LEDs.
LED Type and Color | Function or Meaning |
|---|---|
SPS internal fault (amber) | Lights when there is an internal SPS fault. Have the SPS replaced as soon as possible. |
Replace battery (amber) | The SPS battery cannot support the expected load. The RAID SP flushes all cache data to disk and disables caching. Have the SPS replaced as soon as possible. |
On battery (amber) | AC power has failed and the SPS is supplying AC power to the DPE. The DPE writes all cache data to disk and the SP event log records the incident. Restore AC power as soon as possible. |
Power on (green) | When this LED is glowing steadily, the SPS is operating normally. Flashing means the SPS battery is being charged. |
Note that the RAID enclosure's storage processor (SP) board cannot use write caching unless a fully charged SPS is connected. If the SP does not detect a functional SPS, it automatically disables write caching, reenabling it as soon as a functional SPS is detected. The SPS is always installed or replaced by a trained and approved field service technician. Figure 3-15 shows two SPS units connected to a RAID enclosure in a rack.
The FibreVault enclosure and its components are equipped with LEDs that indicate status. These LEDs are visible from the front and the back of the enclosures.
Table 3-2 summarizes LEDs visible from the front of the rack.
Table 3-2. LEDs Visible From the Front of the Fibre Enclosure
LED | Quantity | Color | Meaning |
|---|---|---|---|
FibreVault active | 1 | Green | On when FibreVault is powered on. |
FibreVault check | 1 | Amber | On when any fault condition exists. Flashing when a FibreVault cooling fault exists. When this fault persists for more than about two minutes, the FibreVault powers off the disk drives. |
Drive active | 1 per disk module slot | Green | Off when the disk module slot is empty or contains a filler. Flashing (mostly off) when the disk drive is powered on but is not spinning. Flashing (mostly on) when the disk drive is spinning. |
Drive check | 1 per disk module slot | Amber | On when the disk module is faulty or a port bypass is activated. |
Enclosure address | 12 | Green | One is lit to indicate the enclosure address (see “Setting an Enclosure Address” for details). |
| Note: If a FibreVault enclosure check LED is illuminated, inspect all other check LEDs to determine the faulty component. Have any faulty component replaced as soon as possible. If a component fails in an enclosure that is not highly available, it may be inoperable until the component is replaced. If a component fails in a highly available FibreVault enclosure, the enclosure's high availability is compromised. |
Table 3-3 summarizes enclosure LEDs visible with the rear door of the rack open.
Table 3-3. LEDs Visible From the Rear of the Fibre Enclosure
LED | Quantity | Color | Meaning |
|---|---|---|---|
LCC active | 1 per LCC | Green | On when the LCC is powered on. |
LCC check | 1 per LCC | Amber | On when either the LCC or an FC-AL connection is faulty. |
Power supply active | 1 per power supply | Green | On when the power supply is operating. |
Power supply check | 1 per power supply | Amber | On when the power supply is faulty. |
Cooling check | 1 per power supply | Amber | Flashing either when multiple fans in the fan assembly are faulty or the assembly is removed. The enclosure powers off the disk drives when the fault persists for more than about two minutes. |
Fan assembly check | 1 per fan assembly | Yellow | Remains on when one fan in the fan assembly is faulty; flashes when two fans are faulty. |
Chapter 4, “Fibre Channel Component Replacement,” contains complete instructions on replacing components.
To power off the Origin fibre channel rack:
Use the host system's administrative CLI or GUI to take all disk arrays or LUNs offline.
Open the rear door of the rack and push each of the PDU breaker switches down and into the Off position.
Unplug both of the rack's power cables (see Figure 3-16) to cut off all power to the rack.
To power off an individual fibre enclosure (while the rack is still powered on):
Use the host system's administrative CLI or GUI to take the target enclosure and its disks or (LUNs) offline.
Pull the disk fan module out of the back of the enclosure (see Figure 3-17).
Push the breaker switch on each power supply in the enclosure to 0 “Off”.
Shut down the main AC lines to the enclosure by unplugging the power cord(s) that connect it to the rack's PDUs.
