This chapter explains the basics of how to operate your new system in the following sections:
Before operating your system, familiarize yourself with the safety information in the following sections:
 | Caution: Observe all ESD precautions. Failure to do so can result in damage to the equipment. |
Wear a grounding wrist strap when you handle any ESD-sensitive device to eliminate possible ESD damage to equipment. Connect the wrist strap cord directly to earth ground.
 | Warning: Before operating or servicing any part of this product, read the “Safety Information” in Appendix B. |
| Warning: Keep fingers and conductive tools away from high-voltage areas. Failure to follow these precautions will result in serious injury or death. The high-voltage areas of the system are indicated with high-voltage warning labels. |
| Caution: Power off the system only after the system software has been shut down in an orderly manner. If you power off the system before you halt the operating system, data may be corrupted. |
All Altix UV 1000 system individual rack units (IRUs) use an embedded chassis management controller (CMC). The CMC communicates with both the blade-level board management controllers (BMCs) and the system management node (SMN), which runs the SGI Management Center software. In concert with the SGI Management Center software, they are generically known as the system control network.
The SGI Management Center System Administrator's Guide (P/N 007-5642-00x) provides information on using the GUI to administer your Altix UV 1000 system.
The Altix UV 1000 system control network provides control and monitoring functionality for each compute blade, power supply, and fan assembly in each IRU enclosure in the system.
The SGI Management Center is an application that provides control over multiple IRUs, and communication to other UV systems. Remote administration requires that the SMN be connected by an Ethernet connection to a private or public Local Area Network (LAN).
The CMC network provides the following functionality:
Powering the entire system on and off.
Powering individual IRUs on and off.
Power on/off individual blades in an IRU.
Monitoring the environmental state of the system.
Partitioning the system.
Enter controller commands to monitor or change particular system functions within a particular IRU. See the SGI UV CMC Controller Software User's Guide (P/N 007-5636-00x) for a complete list of command line interface (CLI) commands.
Provides access to the system OS console allowing you to run diagnostics and boot the system.
Access to the UV system controller network is accomplished by the following connection methods:
A LAN connection to the system management node (running the SGI Management Center software application). This can also be done using an optional VGA-connected console, see Figure 1-1.
A direct ethernet connection to the SBK port (see Figure 1-2) on a CMC (also see the note below).
A serial connection to the “Console” port on the CMC (see note below).
Note: In systems with less than four racks, a connection to any CMC is supported. In systems with more than four racks, use a CMC that is also used to interconnect the four rack building block groups - this CMC will have a connection to the SBK connector.
The ethernet connection is the preferred method of accessing the system console.
Administrators can perform one of the following options for connectivity:
If the SMN is plugged into the customer LAN, connect to the SMN (SSH w/ X11 Forwarding) and start the SGI Management Center remotely.
An in-rack system console can be directly connected to the system management node via VGA and PS2.You can then log into the SMN and perform system administration either through CLI commands or via the SGI Management Center interface. Note that the CMC erthernet port, (labeled SBK) requires connecting to a network with a DHCP server if the SMN node is not used. The CMC is factory set to DHCP mode and thus has no fixed IP address and cannot be accessed until an IP address is established.
A serial connection is used to communicate directly with the CMC. This connection is typically used for service purposes or for system controller and system console access in small systems where an ethernet connection or in-rack system console is not used or available.
The two primary ways to communicate with and administer the UV 1000 system are through SGI Management Center interface or the UV command line interface (CLI).
The UV command line interface is accessible by logging into either a system maintenance node (SMN) or chassis management controller (CMC).
Log in as root, (default password is root) when logging into the CMC.
Login as sysco, when logging into the SMN.
Once a connection to the SMN or CMC is established, various system control commands can be entered. See “Powering On and Off from the Command Line Interface” for specific examples of using the CLI commands.
The following CLI command options are available specifically for the SMN:
-h|--help This help message.
hh|--help This help message + CLI help message.
-q|--quiet No diagnostic message.
-s|--system Select UV system. If only one system is present, this one is selected.
Otherwise, this option is mandatory.
-S|--show depth Show nodes at depth >= 1 using optional supplied pattern.
Default pattern=*
-t|--target One target in one of the two following formats:
a. rack[/slot[/blade]]
b. r{1..}[{s|i}{1..2}[{b|n}{0..15}]]
| Note: This format is NOT for uvcli only. |
Examples: r1i02 = rack 1, slot 2
r2i1b4 = rack 2, slot 1, blade 4
Select the target from the CLI command itself, or, if not available, using the -t option.
The following are examples of uvcli commands:
uvcli --help This help.
uvcli -- leds --help Help on leds command.
uvcli leds r1i1b4 Show leds on BMC located at rack 1, slot1, blade 4.
uvcli -t 1/1 leds Show leds on all BMCs in rack 1, slot 1.
uvcli -- leds -v r1i1 Same as previous command but more verbose.
uvcli -S 1 Show all system serial numbers.
uvcli -S 1 '*/part*' Show all system partitions.
The following list of available CLI commands are specifically for the SMN:
auth authenticate SSN/APPWT change
bios perform bios actions
bmc access BMC shell
cmc access CMC shell
config show system configuration
console access system consoles
help list available commands
hel access hardware error logs
hwcfg access hardware configuration variable
leds display system LED values
log display system controller logs
power access power control/status
Type '<cmd> --help' for help on individual commands.
The SGI Management Center interface is a server monitoring and management system. The SGI Management Center provides status metrics on operational aspects for each node in a system. The interface can also be customized to meet the specific needs of individual systems.
The SGI Management Center System Administrator's Guide (P/N 007-5642-00x) provides information on using the interface to monitor and maintain your Altix UV 1000 system. Also, see Chapter 2 in this guide for additional reference information on the SGI Management Center interface.
This section explains how to power on and power off individual rack units, or your entire Altix UV 1000 system, as follows:
Using a system controller connection, you can power on and power off individual blades, IRUs or the entire system.
If you are using an SGI Management Center interface, you can monitor and manage your server from a remote location. For details, see the documentation for the power management tool you are using in concert with the SGI Management Center.
The Embedded Support Partner (ESP) program enables you and your SGI system support engineer (SSE) to monitor your server remotely and resolve issues before they become problems. For details on this program, see “Using Embedded Support Partner (ESP) ”.
To prepare to power on your system, follow these steps:
Check to ensure that the power connector on the cable between the rack's power distribution units (PDUs) and the wall power-plug receptacles are securely plugged in.
For each individual IRU that you want to power on, make sure that the power cables are plugged into all the IRU power supplies correctly, see the example in Figure 1-3. Setting the circuit breakers on the PDUs to the “On” position will apply power to the IRUs and will start the CMCs in the IRUs. Note that the CMC in each IRU stays powered on as long as there is power coming into the unit. Turn off the PDU breaker switch on each of the PDUs that supply voltage to the IRUs power supplies if you want to remove all power from the unit.
If you plan to power on a server that includes optional mass storage enclosures, make sure that the power switch on the rear of each PSU/cooling module (one or two per enclosure) is in the 1 (on) position.
Make sure that all PDU circuit breaker switches (see the examples in the following three figures) are turned on to provide power to the server when the system is powered on.
Figure 1-4 shows an example of a single-phase 2-plug PDU that can be used with the Altix UV 1000 system. This is the PDU that is used to distribute power to the IRUs when the system is configured with single-phase power.
Figure 1-5 shows an example of an eight-plug single-phase PDU that can be used in the Altix UV 1000 rack system. This unit is used to support auxiliary equipment in the rack.
Figure 1-6 shows examples of the three-phase PDUs that can be used in the SGI Altix UV 1000 system. These PDUs are used to distribute power to the IRUs when the system is configured with three-phase power
Commands issued from the SGI Management Center interface are typically sent to all enclosures and blades in the system (up to a maximum 4096 compute cores) depending on set parameters. SGI Management Center services are started and stopped from scripts that exist in
/etc/init.d
SGI Management Center, is commonly installed in /opt/sgi/sgimc, and is controlled by one of these services—this allows you to manage SGI Management Center services using standard Linux tools such as chkconfig and service.
If your SGI Management Center interface is not already running, or you are bringing it up for the first time, use the following steps:
Power on the server running the SGI Management Center interface.
Open an ssh or other terminal session command line console to the SMN using a remote workstation or local VGA terminal.
Use the information in the section “Preparing to Power On” to ensure that all system components are supplied with power and ready for bring up.
Log in to the SMN as root (the default password is sgisgi).
On the command line, enter mgrclient and press Enter.
The SGI Management Center Login dialog box is displayed.Enter a user name (root by default) and password (root by default) and click OK.
The SGI Management Center interface is displayed.The power on (green button) and power off (red button) are located in the middle of the SGI Management Center GUI's Tool Bar - icons which provide quick access to common tasks and features.
See the SGI Management Center System Administrator's Guide for more information.
The Altix UV 1000 command line interface is accessible by logging into either the system management node (SMN) as root or the CMC as root.
Instructions issued at the command line interface of a local console prompt typically only affect the local partition or a part of the system. Depending on the directory level you are logged in at, you may power up an entire partition (SSI), a single rack, or a single IRU enclosure. In CLI command console mode, you can obtain only limited information about the overall system configuration. An SMN has information about the IRUs in its SSI. Each IRU has information about its internal blades, and also (if other IRUs are attached via NUMAlink to the IRU) information about those IRUs.
Login to the SMN as root, via a terminal window similar to the following:
The default password for logging in to the SMN as root is sgisgi.
# ssh -X root@uv-system-smn
root@system-smn:~/hw>
Once a connection to the SMN is established, the SMN prompt is presented and various system control commands can be entered.
To see a list of available commands enter the following:
root@uv-system-smn:~/hw>ls /sysco/bin
Change the working directory to sysco, similar to the following:
root@uv-system-smn:~/hw>cd /sysco
In the following example the system is powered on without monitoring the progress or status of the power-on process.When a power command is issued, it checks to see if the individaul rack units (IRUs) are powered on; if not on, the power command powers up the IRUs and then the blades in the IRU are powered on.
Enter the power on command, similar to the following:
sysco@uv-system-smn:~/hw>power on
The system will take time to fully power up (depending on size and options).
The following example command options can be used with either the SMN or CMC CLI:
usage: power [-vcow] on|up [TARGET]...turns power on
-v, --verbose verbose output
-c, --clear clear EFI variables (system and partition targets only)
-o, --override override partition check
-w, --watch watch boot progress
To monitor the power-on sequence during boot, see the section “Monitoring Power On”, the -uvpower option must be included with the command to power on.
Because every Altix UV 1000 system comes with a system management node there should be few reasons for powering on the system from a CMC. Use the following information if you have a need to power on from a CMC rather than the SMN CLI or the SGI Management Center GUI. If the SMN is not available you can still boot the system directly by using the CMC, see “Booting Directly From a CMC”.
If a system management node (SMN) is not available, it is possible to power on and administer your system directly from the CMC. When available, the optional SMN should always be the primary interface to the system.
The console type and how these console types are connected to the Altix UV 1000 systems is determined by what console option is chosen. Establish either a serial connection and/or network/Ethernet connection to the CMC.
The console type and how these console types are connected to the Altix UV 1000 servers is determined by what console option is chosen. If you have an Altix UV 1000 server and wish to use a serially-connected “dumb terminal”, you can connect the terminal via a serial cable to the (DB-9) RS-232-style console port connector on the CMC. The terminal should be set to the following functional modes:
Baud rate of 115,200
8 data bits
One stop bit, no parity
No hardware flow control (RTS/CTS)
Note that a serial console is generally connected to the first (bottom) IRU in any single rack configuration.
If you have an Altix UV 1000 system and wish to use a serially-connected "dumb terminal", you can connect the terminal via a serial cable to the (DB-9) RS-232-style console port connector on the CMC board of the IRU.
The terminal should be set to the operational modes described in the previous subsection.
Note that a serial console is generally connected to the CMC on the first (bottom) IRU in any single rack configuration.
On the system management node (SMN) port, the CMC is configured to request an IP address via dynamic host configuration protocol (DHCP).
If your system does not have an SMN, the CMC address cannot be directly obtained by DHCP and will have to be assigned, see the following subsections for more information.
For IP address configuration, there are two options: DHCP or static IP. The following subsections provide information on the setup and use of both.
Network (LAN RJ-45) connections to the Altix UV 1000 CMC are always made via the SBK port.
For DHCP, you must determine the IP address that the CMC has been assigned; for a static IP, you must also configure the CMC to use the desired static IP address.
To use the serial port connection, you must attach and properly configure an RS-232 cable to the CMC's "CONSOLE" port. Configure the serial port as described in “Serial Console Hardware Requirements”.
When the serial port session is established, the console will show a CMC login, and the user can login to the CMC as user "root" with password "root".
To obtain and use a DHCP generated IP address, plug the CMC's external network port (SBK) into a network that provides IP addresses via DHCP, the CMC can then acquire an IP address.
To determine the IP address assigned to the CMC, you must first establish a connection to the CMC serial port (as indicated in the section “Serial Console Hardware Requirements”), and run the command "ifconfig eth0". This will report the IP address that the CMC is configured to use.
Running the CMC with DHCP is not recommended as the preferred option for Altix UV 1000 systems. The nature of DHCP makes it difficult to determine the IP address of the CMC, and it is possible for that IP address to change over time, depending on the DHCP configuration usage. The exception would be a configuration where the system administrator is using DHCP to assign a "permanent" IP address to the CMC.
To switch from a static IP back to DHCP, the configuration file /etc/sysconfig/ifcfg-eth0 on the CMC must be modified (see additional instructions in the “Using a Static IP Address” section). The file must contain the following line to enable use of DHCP:
BOOTPROTO=dhcp |
To configure the CMC to use a static IP address, the user/administrator must edit the configuration file /etc/sysconfig/ifcfg-eth0 on the CMC. The user can use the "vi" command
(i.e. "vi /etc/sysconfig/ifcfg-eth0") to modify the file.
The configuration file should be modified to contain these lines:
BOOTPROTO=static IPADDR=<IP address to use> NETMASK=<netmask> GATEWAY=<network gateway IP address> HOSTNAME=<hostname to use> |
Note that the "GATEWAY" and "HOSTNAME" lines are optional.
After modifying the file, save and write it using the vi command ":w!", and then exit vi using ":q". Then reboot the CMC (using the "reboot" command); after it reboots, it will be configured with the specified IP address.
You can use a network connection to power on your UV system as described in the following steps:
Establish a Network/Ethernet connection (as detailed in the previous subsections). CMCs have their rack and “U” position set at the factory. The CMC will have an IP address, similar to the following:
SBK 172.17.<rack>.<slot>
You can use the IP address of the CMC to login, as follows:
ssh root@<IP-ADDRESS>
Typically, the default password for the CMC set out of the SGI factory is root. The default password for logging in as sysco on the SMN is sgisgi.
The following example shows the CMC prompt:
SGI Chassis Manager Controller, Firmware Rev. 0.x.xx
CMC:r1i1c>
This refers to rack 1, IRU 1, CMC.
Power up your Altix UV system using the power on command, as follows:
CMC:r1i1c> power on
The system will take time to fully power up (depending on size and options). Larger systems take longer to fully power on. Information on booting Linux from the shell prompt is included at the end of the subsection (“Monitoring Power On”).
Typically, the default password for the CMC set out of the SGI factory is root.
Use the following steps to establish a network connection from the SMN to the CMC and power on the system using the CMC prompt and the command line interface:
Establish a network connection to the CMC by using the ssh command from the SMN to connect to the CMC, similar to the following example:
Note: This is only valid if your PC or workstation that is connected to the CMC (via the network connection) has its /etc/hosts file setup to include the CMCs.
ssh root@hostname-cmc
The following example shows the CMC prompt:
SGI Chassis Manager Controller, Firmware Rev. x.x.xx
CMC:r1i1c>
This refers to rack 1, IRU 1, CMC.
Power up your Altix UV system using the power-on command, as follows:
CMC:r1i1c> power on
Note that the larger a system is, the more time it will take to power up completely. Information on booting Linux from the shell prompt is included at the end of the subsection (“Monitoring Power On”).
Open a separate window on your PC or workstation and establish another connection to the SMN or CMC and use the uvcon command to open a system console and monitor the system boot process. Use the following steps:
CMC:r1i1c> uvcon
uvcon: attempting connection to localhost... uvcon: connection to SMN/CMC (localhost) established. uvcon: requesting baseio console access at r001i01b00... uvcon: tty mode enabled, use 'CTRL-]' 'q' to exit uvcon: console access established uvcon: CMC <--> BASEIO connection active ************************************************ ******* START OF CACHED CONSOLE OUTPUT ******* ************************************************ ******** [20100512.143541] BMC r001i01b10: Cold Reset via NL broadcast reset ******** [20100512.143541] BMC r001i01b07: Cold Reset via NL broadcast reset ******** [20100512.143540] BMC r001i01b08: Cold Reset via NL broadcast reset ******** [20100512.143540] BMC r001i01b12: Cold Reset via NL broadcast reset ******** [20100512.143541] BMC r001i01b14: Cold Reset via NL broadcast reset ******** [20100512.143541] BMC r001i01b04: Cold Reset via NL.... |
| Note: Use CTRL-] q to exit the console. |
Depending upon the size of your system, it can take 5 to 10 minutes for the Altix UV system to boot to the EFI shell. When the shell> prompt appears, enter fs0:, as follows:
shell> fs0:
At the fs0: prompt, enter the Linux boot loader information, as follows:
fs0:\> \efi\SuSE\elilo
The ELILO Linux Boot loader is called and various SGI configuration scripts are run and the SUSE Linux Enterprise Server 11 Service Pack x installation program appears.
To power down the Altix UV system, use the power off command, as follows:
CMC:r1i1c> power off ==== r001i01c (PRI) ==== |
You can also use the power status command, to check the power status of your system
CMC:r1i1c> power status ==== r001i01c (PRI) ==== |
on: 0, off: 32, unknown: 0, disabled: 0
The following command options can be used with the power off|down command:
usage: power [-vo] off|down [TARGET]...turns power off -v, --verbose verbose output -o, --override override partition check |
The following are examples of command options related to power status of the system IRUs. These commands and arguments can be used with either the SMN or CMC CLI.
usage: power [-vchow] reset [TARGET]...toggle reset
-v, --verbose verbose output
-c, --clear clear EFI variables (system and partition targets only)
-h, --hold hold reset high
-o, --override override partition check
-w, --watch watch boot progress
usage: power [-v] ioreset [TARGET]...toggle I/O reset
-v, --verbose verbose output
usage: power [-vhow] cycle [TARGET]...cycle power off on
-v, --verbose verbose output
-h, --hold hold reset high
-o, --override override partition check
-w, --watch watch boot progress
usage: power [-v10ud] [status] [TARGET]...show power status
-v, --verbose verbose output
-1, --on show only blades with on status
-0, --off show only blades with off status
-u, --unknown show only blades with unknown status
-d, --disabled show only blades with disabled status
usage: power [-ov] nmi|debug [TARGET]...issue NMI
-o, --override override partition check
-v, --verbose verbose output
usage: power [-v] margin [high|low|norm|<value>] [TARGET]...power margin control
high|low|norm|<value> margin state
-v, --verbose verbose output
usage: power --help
--help display this help and exit
|
Embedded Support Partner (ESP) automatically detects system conditions that indicate potential future problems and then notifies the appropriate personnel. This enables you and SGI system support engineers (SSEs) to proactively support systems and resolve issues before they develop into actual failures.
ESP enables users to monitor one or more systems at a site from a local or remote connection. ESP can perform the following functions:
Monitor the system configuration, events, performance, and availability.
Notify SSEs when specific events occur.
Generate reports.
ESP also supports the following:
Remote support and on-site troubleshooting.
System group management, which enables you to manage an entire group of systems from a single system.
For additional information on this and other available monitoring services, see the section “SGI Electronic Support ” in Chapter 6.
You can monitor and interact with your Altix UV 1000 server from the following sources:
Using the SGI 1U rackmount console option you can connect directly to the system management node (SMN) for basic monitoring and administration of the Altix system. See “1U Console Option” in Chapter 2 for more information; SLES 11 or later is required.
A PC or workstation on the local area network can connect to the SMN's external ethernet port and set up remote console sessions or display GUI objects from the SGI Management Center interface.
A serial console display can be plugged into the CMC at the rear of IRU 001. You can also monitor IRU information and system operational status from other IRUs that are connected to IRU 001.
These console connections enable you to view the status and error messages generated by the chassis management controllers in your Altix UV 1000 rack. For example, you can monitor error messages that warn of power or temperature values that are out of tolerance. See the section “1U Console Option” in Chapter 2, for additional information.
Besides adding a network-connected system console or basic VGA monitor, you can add or replace the following hardware items on your Altix UV 1000 series server:
Peripheral component interface (PCIe) cards into the optional PCIe expansion chassis.
PCIe cards into the blade-mounted PCIe riser card.
Disk drives in your dual disk drive riser card equipped compute blade.
The PCIe based I/O sub-systems, are industry standard for connecting peripherals, storage, and graphics to a processor blade. The following are the primary configurable I/O system interfaces for the Altix UV 1000 series systems:
The optional two-slot internal PCIe riser card is a compute blade-installed riser card that supports one x8 and one x16 PCIe Gen2 card.
The optional external PCIe riser card is a compute blade-installed riser card that supports two x16 PCI express Gen2 ports. These ports can be used to connect to an optional I/O expansion chassis that supports multiple PCIe cards. Each x16 connector on the riser card can support one I/O expansion chassis.
Important: PCIe cards installed in a two-slot internal PCIe riser card are not hot swappable or hot pluggable. The compute blade using the PCIe riser must be powered down and removed from the system before installation or removal of a PCIe card. Also see “Installing Cards in the 1U PCIe Expansion Chassis” in Chapter 5 for more information.
Not all blades or PCIe cards may be available with your system configuration. Check with your SGI sales or service representative for availability. See Chapter 5, “PCIe and Disk Add or Replace Procedures” for detailed instructions on installing or removing PCIe cards or Altix UV 1000 system disk drives.