This chapter describes how to install software on the SGI management node (SMN) and the Altix UV system itself in case of root disk failure on either platform. It covers these topics:
Figure 1-1 shows one rack of an SGI Altix UV system.
The system management node (SMN) is either located in the top 1U slot between the individual rack units (IRUs) or at the top of the rack. This section covers installation instructions for the SMN and covers these topics:
To install SLES 11 software images on the system management node perform the following steps:
Turn on, reset, or reboot the SMN. The power on button is on the right of the SMN, as shown in Figure 1-2.
Insert the SLES 11 Service Pack 1 DVD in the DVD drive on the left of the SMN as shown in Figure 1-2.
Once installation of software on the system management node is complete, remove the DVD from the DVD drive.
After the reboot completes, you will eventually see the YaST2 - firstboot@Linux Welcome screen, as shown in Figure 1-3. Select the Next button to continue.
Note: The YaST2 Installation Tool has a main menu with sub-menus. You will be redirected back to the main menu, at various times, as you follow the steps in this procedure.
You will be prompted by YaST2 firstboot installer to enter your system details including the root password, network configuration, time zone, and so on.From the Hostname and Domain Name screen, as shown in Figure 1-4, enter the hostname and domain name of your system in the appropriate fields. Make sure that Change Hostname via DHCP is not selected (no x should appear in the box). Note that the hostname is saved to /etc/hosts in step 10, below. Click the Next button to continue.
Note: The mostly used keys are Tab and Shift + Tab to move forward and backward in modules, the arrow keys to move up and down or left and right in lists, the shortcuts (press Alt + highlighted letter) and Enter to execute the selected action or activate a menu item.
You can use Ctrl L to refresh the YaST screen as necessary.
The Network Configuration II screen appears, as shown in Figure 1-5. Select Change and a small window pops up that lets you choose Network Interfaces... or Reset to Defaults. Choose Network Interfaces.
From the Network Settings screen, as shown in Figure 1-6, configure the first card under Name to establish the connection to your SGI Altix UV system. To do this, highlight the first card and select Edit.
Note: In SLES11, this screen is also where we will come back to in order to set up things like the default route and DNS. You can see all of those menu choices just to the right of Overview in Figure 1-6.
The Network Card Setup screen appears, as shown in Figure 1-7. SGI suggests using static IP addresses and not DHCP for admin nodes. Select Statically assigned IP Address. Once selected, you can enter the IP Address, Subnet Mask, and Hostname.
Note: You must use a fully qualified hostname (host + domain), such as, mysystem-admin.domainname .mycompany.com.
These are the settings for your SMN house/public network interface. You will enter the default route, if needed, in a different step. Select Next to continue.
At this point, you are back at the Network Settings screen as shown in Figure 1-8. At this time, select Hostname/DNS. In this screen, you should enter your house/public network hostname and fully qualified domain names. In addition, any name servers for your house/public network should be supplied. Please select (ensure an x is in the box) for Write hostname to /etc/hosts. Do not select OK yet.
Select Routing shown in Figure 1-9 and enter your house/public network default router information there. Now you can select OK.
You are now back at the Network Configuration II screen, Click Next.
In the Clock and Time Zone screen, you can enter the appropriate details. Select Next to continue.
In the Password for the System Administrator "root"' screen, enter the password you wish to use. Select Next to continue.
In the User Authentication Method screen, most customers will want to stick with the default (Local). Select Next to continue.
In the New Local User screen, you can just select Next (and say Yes to the Empty User Login warning). Select Next to continue.
In Installation Completed, select Finish.
After you have completed the YaST first boot installation instructions, login into the SMN. You can use YaST2 to confirm or correct any configuration settings.
Note: The SGI System Management Node Software 1.1 Release Notes are available on SGI SupportFolio https://support.sgi.com/login.
Using YaST2 or manually the rpm command, from the SGI-System-Management-Node-Software-1.1 ISO, install the following:rpm -i noarch/sgi-smn-release-1.1-sgi701r3.sles11.noarch.rpm rpm -i x86_64/monit-5.0.3-1sgi701r1.sles11.x86_64.rpm rpm -i x86_64/monit-sgi-rules-5.0.3-1sgi701r1.sles11.x86_64.rpm rpm -i x86_64/sysco-uv-libs-1.1-20100520.1219sgi701r1.sles11.x86_64.rpm rpm -i x86_64/sgi-gather-smn-uv-1.1-sgi701r1.sles11.x86_64.rpm rpm -i x86_64/sysco-uv-1.1-20100520.1219sgi701r1.sles11.x86_64.rpm
| Note: Documentation for SUSE Linux Enterprise Server 11 SP1 is located on your system in the /docu/en directory. |
For instructions for installing SLES 11 SP1 and SGI Foundation Software 2.3, see Procedure 1-2.
This section describes how to install Red Hat Enterprise Linux 6 on the system management node.
Installation instructions for Red Hat Enterprise Linux 6 (RHEL 6) are contained in the Red Hat Enterprise Linux 6 Installation Guide available at http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/pdf/Installation_Guide/Red_Hat_Enterprise_Linux-6-Installation_Guide-en-US.pdf
These instructions assume that you have a VGA display or that you are able to remotely display X11 graphics. If you do not have a VGA display, you should connect from your workstation to the target server with the following command:
% ssh -X root@target-server |
To install RHEL 6 software images on the system management node, perform the following steps:
Insert the product media and enter the following command to mount it:
% mount /dev/cdrom/mnt
Run the following command and follow the examples provided:
% /mnt/create-yum-config-file
Additional installation instructions for installing SGI Foundation Software (SFS) 2.3 and RHEL 6 are provided in the SFS 2.3 release notes file.
By default RHEL 6 uses NetworkManager and all the directions here assume it is off. Perform the following commands, before proceeding:
# chkconfig NetworkManager off # service NetworkManager stop
Add the IPADDR, NETMASK, and NETWORK values appropriate for the network interface to the /etc/sysconfig/network-scripts/ifcfg-eth0 file similar to the following example:
IPADDR=128.162.244.88 NETMASK=255.255.255.0 NETWORK=128.162.244.0
Create the /etc/sysconfig/network file similar to the following example:
[root@localhost ~]# cat /etc/sysconfig/network NETWORKING=yes HOSTNAME=my-system-admin GATEWAY=128.162.244.1
Create the /etc/resolv.conf file similar to the following example:
[root@localhost ~]# cat /etc/resolv.conf search domain-name.mycompany.com nameserver 137.38.224.40 nameserver 137.38.31.248 nameserver 137.38.225.5
Add the IP address of the house network interface and the name(s) of the admin node to /etc/hosts file similar to the following example:
# echo "128.162.244.88 my-system-admin.domain-name.mycompany.com my-system-admin" >> /etc/hosts
Set the admin node hostname, as follows:
# hostname my-system-admin
Force the invalidation of the host cache of nscd with the nscd(8) command on the hosts file, as follows:
# nscd -i hosts
Restart the following services (in this order), as follows:
# /etc/init.d/network restart # /etc/init.d/rpcbind start # /etc/init.d/nfslock start
Set the local timezone. The timezone is set with /etc/localtime, a timezone definition file. The timezone defined in /etc/localtime can be determined, as follows:
# strings /etc/localtime | tail -1 CST6CDT,M3.2.0,M11.1.0
Link the appropriate timezone file from directory /usr/share/zoneinfo to /etc/localtime. For example, set timezone to Pacific Time / Los Angeles, as follows:
# /bin/cp -l /usr/share/zoneinfo/PST8PDT /etc/localtime.$$ # /bin/mv /etc/localtime.$$ /etc/localtime
Confirm the timezone, as follows:
# strings /etc/localtime | tail -1 PST8PDT,M3.2.0,M11.1.0
Set network time configuration. By default, the configuration in /etc/ntp.conf directs requests to public servers of the pool.ntp.org project. Use public servers from the http://www.pool.ntp.org/en/ project:
server 0.rhel.pool.ntp.org server 1.rhel.pool.ntp.org server 2.rhel.pool.ntp.org
You may need to modify this ntp configuration file to point at a time server on your network. Please do not remove any entries that serve the cluster networks. For example, to direct requests to, for example, my.corp.mycompany.com, comment/delete the pool entries and insert the local entry, as follows:
# Use public servers from the pool.ntp.org project. # Please consider joining the pool (http://www.pool.ntp.org/join.html). #server 0.rhel.pool.ntp.org #server 1.rhel.pool.ntp.org #server 2.rhel.pool.ntp.org server my.corp.mycompany.com
Restart the ntp server, as follows:
# /etc/init.d/ntpd restart
Make sure you have registered with the Red Hat Network (RHN). If you have not yet registered, run the following command:
% /usr/bin/rhn_register
From the SGI-System-Management-Node-Software-1.1 ISO, install the following:
rpm -i monit-sgi-rules-5.0.3-1sgi702rp1.rhel6.x86_64.rpm rpm -i sgi-gather-smn-uv-1.0-sgi702rp9.rhel6.x86_64.rpm rpm -i sgi-smn-docs-1-sgi702rp8.rhel6.noarch.rpm rpm -i sgi-smn-release-1.1-sgi702rp50.rhel6.noarch.rpm rpm -i sgi-uv-bmc-1.2.3_4-20100929.noarch.rpm rpm -i sgi-uv-cmc-1.2.4_3-20100929.noarch.rpm rpm -i sgi-uv-iobmc-1.2.1_3-20100929.noarch.rpm rpm -i sysco-uv-1.0-20101115.1736sgi702rp1.rhel6.x86_64.rpm rpm -i sysco-uv-libs-1.0-20101115.1736sgi702rp1.rhel6.x86_64.rpm rpm -i sysco_uv-tools-1.3-20100922.i386.rpm
For information on how to install SGI Management Center (SMC) software, see the SGI Management Center Installation and Configuration guide available at http://docs.sgi.com. In particular, see Chapter 1, “Installing SGI Management Center” and “Install Management Center Payload” section in Chapter 4, “Creating Payloads and Images”.
This section describes how to do a clean install on a SGI Altix UV server.
Currently, to reinstall your software you need a USB-DVD and connect that to the BASEIO (see Figure 1-1) and boot the Novell rescue CD.
Use a USB-DVD drive to connect to the BASEIO and boot using the Novell rescue CD. You need to get BIOS booted to efi shell and boot from the DVD. Use a command similar to the following:
fs1:efi\boot\bootx64 -i initrd linux console=ttyS0,115200n8 ssh=1 |
| Note: The fs number may be different. Booting an Altix UV systems takes some time. It may take awhile for the screen to change. |
When partitioning the system disk, you need to create a /boot/efi partition instead of a /boot partition.
For information on installing software on your Altix UV sever, see “Installing SLES 11 SP1, SGI Foundation 2.3, and SGI Performance Suite 1.1 Software Together” or “Installing RHEL 6 on an Altix UV System”.
The following set of instructions is based on an install using the physical product media. For other installation methods, see the product release notes.
| Note: On large configurations, systems with 2048 processors, for example, can take up to 50 minutes to boot SLES. |
You can speed up the software install and booting of your system by initially turning off NUMAlink from the CMC, as follows:
cmc> hwcfg NL5_ENABLE=no cmc> power reset |
Once the system boots, you can re-enable NUMAlink from the CMC, as follows:
cmc> hwcfg NL5_ENABLE=yes cmc> power reset |
For an overview of SLES11 SP1 installation, see the SUSE Linux Enterprise Server 11 Installation Quick Start (installquick.pdf). This document provides a quick overview of how to run through the default installation of SLES walking you through a number of installation screens. For detailed installation instructions, see the SUSE Linux Enterprise Server 11 Deployment Guide (deployment.pdf)
| Note: Documentation for SUSE Linux Enterprise Server 11 SP1 is located on your system in the /docu/en directory. |
| Note: If you are only installing SLES11 SP1 and SGI Foundation 2.3 software, just skip the steps about installing SGI Performance Suite software. |
To install SUSE Linux Enterprise Server 11 (SLES11) SP1 from the DVD media and SGI Foundation Software 2.3 and SGI Performance Suite 1.1 software as an add-on products, perform the following steps:
Insert the SLES 11 SP1 Installation DVD in the DVD drive and reboot your system.
Follow the steps in the SUSE Linux Enterprise Server 11 Installation Quick Start. When you get to the Installation Mode screen, as shown in Figure 1-10, click the button next to Include Add-On Products from Separate Media and then click Next to continue.
From the Media Type screen, shown in Figure 1-11, click the button to the left of CD.
The Insert the add-on product CD dialog box appears. Insert the SGI-Foundation-x86_64 CD into the drive and click the Continue button and then Next to proceed.
From the Add-On Product Installation screen, as shown in Figure 1-12, click on SGI-Foundation-x86_64 1-6 cd:///.Directory and click the Add button and then Next to proceed
Repeat steps 3 through 6 to add a software bundle from the SGI Performance Suite 1.1 product as an add-on product. Eject the SGI Foundation 2.3 CD and replace it with an SGI Performance Suite CD.
The following CDs ship with the SGI Performance Suite 1.1 release:
SGI® Accelerate™ CD
SGI® MPI CD
SGI® REACT® CD
SGI® UPC CD
For more information on these products, see the SGI Performance Suite 1.1 Start Here.
From the Add-On Product Installation screen as shown in Figure 1-12, click on SGI-ProPack-x86_64 6-4 cd:///Directory click the Add button and then Next to proceed.
Follow the SLES11 SP1 instructions to complete the installation.
This section describes how to install Red Hat Enterprise Linux 6 on an Altix UV 100 or Altix UV 1000 system.
| Note: On large configurations, systems with 2048 processors, for example, can take up to 75 minutes to boot RHEL 6. |
You can speed up the software install and booting of your system by intially turning off NUMAlink from the CMC, as follows:
cmc> hwcfg NL5_ENABLE=no cmc> power reset |
Once the system boots, you can re-enable NUMAlink from the CMC, as follows:
cmc> hwcfg NL5_ENABLE=yes cmc> power reset |
To install RHEL 6 software images on the an Altix UV 100 or Altix UV 1000 system, perform the following steps:
These instructions assume an external USB DVD ROM drive, the BIOS must be in legacy mode. Make sure the BIOS is current, that is, later than the 1.32 release.
At the EFI shell, use the map command to get a list of recognized devices/filesystems, for example:
Shell> map Device mapping table fs0 :Removable HardDisk - Alias hd25a0b blk0 PciRoot(0x0)/Pci(0x3,0x0)/Pci(0x0,0x0)/Scsi(0x0,0x0)/HD(1,MBR,0x000D9FB8,0x22,0xF329C) fs1 :Removable HardDisk - Alias hd25b0b blk1 PciRoot(0x0)/Pci(0x3,0x0)/Pci(0x0,0x0)/Scsi(0x1,0x0)/HD(1,GPT,74D7CC85-8A6C-4F87-8865-E58002CACAD7,0x800,0x64000) fs2 :Removable CDRom - Alias cd67a0b blk2 | |___ CD/DVD media PciRoot(0x0)/Pci(0x1D,0x7)/USB(0x0,0x0)/CDROM(0x1,0x232,0x2C0)
Select the DVD media, in this example fs2:
Shell> fs2:
For a sanity check, list its contents, as follows:
fs2:\> ls Directory of: fs2:\ 09/22/10 06:03p 2,048 EFI 0 File(s) 0 bytes 1 Dir(s
Change directory (cd) to efi\boot, as follows:
fs2:> cd efi\boot
List the directory contents, as follows:
fs2:\EFI\BOOT> dir Directory of: fs2:\EFI\BOOT 09/22/10 06:03p 2,048 . 09/22/10 06:03p 2,048 .. 09/22/10 06:03p 429 BOOTX64.conf 09/22/10 06:03p 232,971 BOOTX64.efi 09/22/10 06:03p 1,341 splash.xpm.gz 3 File(s) 234,741 bytes 2 Dir(s)
Run bootx64, as follows:
fs2:\EFI\BOOT> bootx64
At this stage you should be at the GRUB prompt, as follows:
grub>
Set root to the device that corresponds to the CD/DVD device, in this example
grub> root (hd2) Filesystem type is iso9660, using whole disk
If you have selected the correct device, grub will report that it recognized as an iso9660 filesystem. If unsure about which device to select, type root <TAB> to see a list of possible devices.
Select the kernel to boot and choose the boot options, as follows:
grub> kernel /images/pxeboot/vmlinuz virtefi console=ttyS0,115200 askmethod vnc ip= netmask= gateway= dns=,,...,
If executing correctly, GRUB should display something similar to the following:
Trying to allocate 923 pages for VMLINUZ [Linux-EFI, setup=0x1034, size=0x39a580]
Some important considerations for boot options are, as follows:
virtefi
Note: For Altix UV 100 and Altix UV 1000 systems, you must specify the virtefi option the or installer's kernel will panic.
console=ttyS0,115200
Setup the serial console
askmethod
Use this option only if you want make sure that the installer will ask you which install method (local DVD, NFS, HTTP and so on) should be used.
vnc
Use vnc instead of text mode
Some network related parameters are, as follows:
ip=<IPADDR>
Tells the installer which IPADDR should be assigned to the Network Interface Card (NIC).
netmask=<NETMASK>
Tells the installer which NETMASK to assign.
gateway=<GW>
Gateway IP address
dns=<DNS1>,<DNS2>,...,<DNS3>
DNS servers IP addresses
Note: The network parameters are not needed if you have a Dynamic Host Configuration Protocol (DHCP) server.
Select the initrd image, as follows:
grub> initrd /images/pxeboot/initrd.img
If executing correctly, GRUB should display something similar to the following:
[Initrd, addr=0x726bf000, size=0x1cab9cc]
You should now be ready to boot the kernel, as follows:
grub> boot
Once the kernel booted, the installer will start and ask you which NIC, in case you have several, to use. After you answer few questions, the installer will announce that the Virtual Network Computing (VNC) is up and that the graphical installation has started. You need to manually connect your vnc client to <hostname:1> (IPADDR) to begin the install, such as, <IPADDR:1> depending on whether a hostname has been set by DHCP.
Press Enter for a shell to start the graphical installation.
Connect to the installer's VNC server from another machine, similar to the following:
vncviewer IPADDR:1
You should now be able to continue the installation, in graphic mode, from within the VNC session.
Once the installation complete and the system rebooted, stop at the GRUB menu, enter a to modify the kernel boot cmdline parameter and add virtefi option to it.
Note: If you do not add the virtefi parameter the kernel will panic.
Once the system booted, edit the /etc/grub.conf file, which is a symlink to /boot/efi/EFI/redhat/grub.conf ) to make the change permanent.
To enable a remote console to your UV Altix system, see “Enabling a Remote Console” in Chapter 3 of the SGI Altix UV System Management Node Administrator's Guide.