This chapter describes the function and physical components of the 2U CMPX module, which can add optional processors, local memory (DIMMs) and four PCI/PCI–X card slots to your system. Specifically, this chapter includes the following information:
The CMPX module includes the following features:
One or two optional 64-bit Intel processors.
The L1 controller manages and monitors the functions of the CMPX module, such as system temperature. The system includes an L1 controller display system that processes error messages.
The module can have one or two power supplies. The second power supply is redundant to assure that your CMPX module will always have power.
The L1 port connector connects the module to an L2 controller.
The NUMAlink ports connect the CMPX module to base compute modules, CPU expansion modules and/or other CMPX modules.
The console port is an RS-232 DB-9 serial connector used to connect a system console, which is used monitor and manage your system.
Four 64-bit PCI and PCI–X card slots on two busses can contain 33-MHz and 66-MHz PCI cards, and 66-MHz, 100-MHz, or 133-MHz PCI–X cards.
Figure 5-1 shows a front panel and side view of the CMPX module.
The CMPX module architecture includes the components discussed in the following subsections
The IP57 node board consists of the following components:
Twelve dual-inline memory module (DIMM) slots enable installation of DIMMs to provide up to 24 GB of memory.
The SHub ASIC enables communication between memory and I/O devices.
The serial ID EEPROM contains component information.
Two voltage regulators convert input voltages to those required by components.
Each module optionally has up to 24 GB of local memory, which includes main memory and directory memory for cache coherence.
Local memory is provided by DIMMs, which contain double data rate synchronous dynamic random-access memory (DDR SDRAM chips), installed in groups of 4, 8, or 12 DIMMs.
All the DIMMs must be present in a group for proper functionality. See Figure 5-2 for group locations in the memory slots.
 | Note: The four DIMMs that compose a group must be the same size; however, each group may differ in total memory size (capacity). |
The SHub ASIC enables communication among the processors, memory, network, and I/O devices. It controls all activity within the node board (for example, error correction and cache coherency). The SHub ASIC also supports page migration.
The SHub ASIC consists of the following:
One central crossbar (XN) provides connectivity between the SHub ASIC interfaces.
One processor interface (PI) communicates directly with one or two processors.
One memory/directory interface (MD) controls all memory access.
Two network interfaces (NI) interface between the crossbar unit and the NUMAlink interconnect.
One I/O interface (II) allows I/O devices to read and write memory (direct memory access [DMA] operations) and allows the processors within the system to control the I/O devices (PIO operations).
One local block (LB) services processor I/O (PIO) requests local to the SHub ASIC.
The interface board contains the following components:
L1 controller logic.
Power supply interface.
NUMAlink connectors.
Switching regulators.
Connectors to the IP57 node board and the PCI riser card.
The PCI riser card provides the following:
PIC ASIC.
Connectors that connect the PCI riser card to the interface board.
Four PCI/PCI–X card slots (64 bit, 3.3 V).
The CMPX module can contain one or two power supplies; the second power supply is optional and is required only when you want redundant power. The power supply can input 110/220 VAC and output 500 W (12 VDC, 5 VDC, and -12 VDC).
Both power supplies are hot–swappable. They are located at the front right side of the module. The primary power supply is the left supply and the redundant power supply is the right supply.
This section describes the external components of the CMPX module, which are located in the front and rear panels.
This section describes the front panel controls and indicators of the CMPX module, as shown in Figure 5-3.
The front panel of the CMPX module has the following items:
L1 controller display. A liquid crystal display (LCD) displays status and error messages that the L1 controller generates.
Note: See the SGI L1 and L2 Controller Software User's Guide (007-3938-00x) for more information on the L1 controller. Power button with LED. Press this button to power on the internal components. Alternatively, you can power on the internal components at a system console. The LED illuminates green when the internal components are on.
Reset button. Press this button to reset the internal processors and ASICs. The reset will cause a memory loss. (To perform a reset without losing memory, see the NMI button information that follows.)
NMI button. Press the NMI (non-maskable interrupt) button to reset the internal processors and ASICs without losing memory. Register data and memory are stored in a /var/adm/crash file.
Service-required LED. This LED illuminates yellow to indicate that an item has failed or is not operating properly, but the CMPX module is still operating.
Failure LED. This LED illuminates red to indicate that a failure has occurred and that the CMPX module is down.
This section describes the rear panel connectors, PCI/PCI–X slots, and LEDs of the CMPX module, as shown in Figure 5-4.
The rear panel of the CMPX module has the following items:
Power connector. This connector connects the CMPX module to an AC power outlet.
Console port. This DB–9 serial port (console and diagnostic port) enables you to connect a system console to the L1 controller on the CMPX module.
L1 port (USB type B). This Universal Serial Bus (USB) type B connector connects the CMPX module's L1 controller to an optional L2 controller.
Two NUMAlink-4 connectors. The NUMAlink connectors attach the CMPX module to one of the following modules: base compute module, expansion compute module, or NUMAlink-4 module (router). This connection is made with a NUMAlink cable at 3.2 GB/s in each direction.
NUMAlink LED. The NUMAlink connector has two LEDs. These LEDs are located to the right of the NUMAlink connector. One LED lights yellow to indicate that the CMPX module and the module to which it is connected are powered on. The other LED lights green when the link between the CMPX module and the module to which it is connected is established.
PCI/PCI–X slots (bus 1, slot 1; bus 1, slot 2; bus 2, slot 1; bus 2, slot 2). These slots are labeled from bottom to top PCI 1, PCI 2, PCI 3, and PCI 4. Two of these slots are on one bus, and two slots are on another. These 64-bit slots can contain 33-MHz and 66-MHz PCI cards, or 66-MHz, 100-MHz, or 133-MHz PCI–X cards. (For an updated list of supported cards, see SGI Supportfolio at http://support.sgi.com.)
Note: If you run PCI and PCI–X cards on the same bus at the same time, the PCI–X card will run on PCI mode. And if you run cards of different speeds on the same bus, the highest speed card will run at the speed of the slower card. For example, if a card is running at 100 MHz in one slot of a bus, and a card is running at 33 MHz in the second slot of the same bus, both cards will run at 33 MHz.
This section lists the internal CMPX module configuration options such as the number of DIMMs that can be installed in the CMPX module to increase its local memory.
This section also lists external CMPX module configuration options that can enhance the performance of the server system. For example, the server system configuration can include a CMPX module together with a base compute module, and a 2U SGI TP900 storage system to expand storage.
PCI and PCI–X cards and memory (DIMMs) are the configurable internal components of the CMPX module.
As a customer, you can configure PCI and PCI-X cards and memory. See Chapter 7, “Installing and Removing Customer-replaceable Units” for information about installing and removing these items to reconfigure your CMPX module.
 | Warning: To prevent personal injury or damage to your system, only trained SGI system support engineers (SSEs) can service or configure internal components of the CMPX module that are not specifically listed as serviceable and configurable by customers. |
The CMPX module can be configured together with the following items to expand the function of your server system:
Every system must include a base compute module. The base compute module can have 1 or 2 processors, up to 24 GB of local memory, an optional DVD-ROM, one or (optionally) two system disk drives, and two PCI/PCI-X and one PCI slots. The fourth slot includes a base I/O card that adds various connectors to your system.
The system expansion compute module, which is interconnected to the base compute module via a NUMAlink-4 cable, adds processors and memory.
The TP900 storage module provides additional storage to the system. See SGI Total Performance 900 Storage System User's Guide (007-4428-00x), for information about this module. The server system supports other storage modules. For information, see “Storage Expansion” in Chapter 2.
The optional NUMAlink-4 router module connects eight to 16 Altix 350 modules and allows for maximum expansion of the system.
The server system can be configured with a CMPX module in many different ways to satisfy your computing needs. This section shows two sample configurations.
Figure 5-5 shows a server system on a table top that includes the following items:
A CMPX module adds optional processors, additional local memory and four PCI/PCI–X card slots to your system.
A 2U base compute module has 1 or 2 processors, up to 24 GB of local memory, and two PCI/PCI–X and one PCI card slots. (Because the fourth lowermost slot comes with a factory-installed 66-MHz base I/O card, the slot directly above it, which is on the same bus as the slot in which the I/O card is installed, can only accommodate a PCI card that runs at a speed of 66 MHz or slower.)
Figure 5-6 shows a server system rackmounted in a 17U rack that includes the following items (from top to bottom in the rack):
Each CMPX module adds optional processors, additional local memory and four PCI/PCI–X card slots.
A system expansion compute module adds processors, and local memory.
A 2U base compute module adds 1 or 2 processors, local memory, and two PCI/PCI–X and one PCI card slots. (Because the fourth lowermost slot comes with a factory-installed 66-MHz base I/O card, the slot right above it, which is on the same bus as the slot in which the Base I/O PCI card is installed, can only accommodate a PCI card that runs at a speed of 66 MHz or slower.) Any faster card installed in this slot will run at the reduced 66 MHz speed.
Table 5-1 lists the bandwidth characteristics of the CMPX module.
Table 5-1. Bandwidth Characteristics of the CMPX Module
Characteristic | Peak Bandwidth | Sustainable Bandwidth |
|---|---|---|
NUMAlink channel | 6.4 GB/s full duplex | ~2.8 GB/s each direction |
Main memory | Up to 10.8 GB/s | Up to 10.8 GB/s |
Front Side Bus (FSB) | 6.4 GB/s | ~6.4 GB/s |
Table 5-2 lists the specifications for the CMPX module.
Table 5-2. CMPX Module Specifications
Characteristic | Specification |
|---|---|
Height | 3.44 in. (8.74 cm) |
Width | 17.06 in. (43.33 cm) |
Depth | 27 in. (68.58 cm) (with bezel) |
Weight | 50 lb (22.7 kg) maximum configuration[a] |
Heat Dissipation | 1467 BTUs |
Input power | 120 - 240 VAC |
[a] Weight will vary depending on whether the module has zero, one, or two optional processors, one or two power supplies, and on the number of DIMMs installed. | |