Chapter 1. Altix UV Software Installation

This chapter describes how to install Windows Server 2008 R2 Datacenter software on an SGI Altix UV 100 or SGI Altix UV 1000 system.


Note: This manual does not apply to SGI Altix UV 10 systems.


It covers the following topics:

Overview

To install Windows Server 2008 R2 software on an SGI Altiv UV 100 or SGI Altix UV 1000 system, you need to disable the NL5_enable setting (see Procedure 1-1), ensure that the BaseIO BIOS version 1.45 (or later) is installed, and ensure that the system is either in legacy or x2apic mode. The install procedure assumes a clean disk. You need to completely wipe the partition tables, both MBR and GPT (see Chapter 2, “Disk Preparation for Windows”). Once the install is complete, you need to enable the NL5_enable setting and change the system to extended mode. If you do not completely remove the partition tables before doing the Windows server install procedure, the system may hang or fail on reboot.

Installing Windows

This section describes how to install Windows Server 2008 R2 Datacenter on an Altix UV 100 or Altix UV 1000 system.

Procedure 1-1. Installing Windows Server Software

    To install Windows Server 2008 R2 software on an SGI Altix UV 100 or SGI Altix UV 1000 system, perform the following steps:


    1. Note: The Windows Server 2008 R2 SP1 release includes these hotfixes by default so you can skip this step if you are installing the SP1 release.


      The hotfixes listed below are available from Microsoft, Inc. They are needed later in this procedure. The hotfixes have passwords which will be contained in emails from Microsoft. The passwords are needed to extract the files from the zip archives. Put zipped software on a USB stick or burn them into a data CD. The hotfixes required are, as follows:

    2. Make sure the system disk drives are setup for Windows. Windows 2008 R2 Datacenter, when booted from EFI, should allow you to manage the disks without any external help. Each disk must have a proper GPT label. If the disks have msdos labels, see Chapter 2, “Disk Preparation for Windows”.

      If you run into problems, you may have to wipe out the master boot record (MBR), create a new msdos disk label, create a 100 MB system partition and a second boot/system partition (greater than or equal to 46 GB). Then format those targeted to Windows as new technology file system (NTFS). For more information, see “Disk Setup with Linux” in Chapter 2.

    3. Attach a USB keyboard, USB mouse, and VGA monitor to the BaseIO blade (see Figure 1-1 and Figure 1-2).


      Note: You can also use the display/keyboard and media redirection feature of the BaseIO BMC firmware (BMC firmware versions 1.2.2 or later). See “Enabling a Remote Console”.


      Figure 1-1. Altix UV 1000 Individual Rack Unit (IRU)

      Altix UV 1000 Individual Rack Unit (IRU)

      Figure 1-2. Altix UV 1000 BaseIO Riser Enabled Blade Front Panel Example

      Altix UV 1000 BaseIO Riser Enabled Blade Front Panel Example

      Figure 1-3. Altix UV 100 Individual Rack Unit (IRU)

      Altix UV 100 Individual Rack Unit (IRU)

    4. Attach USB DVD-ROM drive if needed (see Figure 1-2 ), and insert Windows Server 2008 R2 Datacenter DVD. For the location of the Altix UV 100 DVD drive, see Figure 1-3.

    5. Use the ssh command to connect to the Chassis Management Controller (CMC), similar to the following example:


      Note: This assumes the system CMC SMN port is connected and an address is assigned. For more information, see Chapter 2 in the SGI Altix UV CMC Controller Software User's Guide.


      % ssh root@hostname-cmc
      
      SGI Chassis Manager Controller, Firmware Rev. 1.3.11
      
      system-cmc CMC:r001i01c>

      Typically, the default password set out of the factory is root. The CMC prompt appears. r001i01c refers to rack 1, IRU 1, CMC.

    6. Use the hwcfg command on the console to disable NUMAlink, as follows:

      system-cmc CMC:r001i01c>hwcfg nl5_enable=no

    7. From the CMC, perform a power reset, as follows:

      system-cmc CMC:r001i01c>power reset
      

    8. When prompted on the console, hit the space bar for the Boot Menu, as shown in Figure 1-4.

      Figure 1-4. Console Prompt for Boot Menu Screen

      Console Prompt for Boot Menu Screen

      From the boot manager screen, select Device Manager, as shown in Figure 1-5.

      Figure 1-5. Device Manager Screen

      Device Manager Screen

      Navigate the Device Manager, as follows:

      • F1=Scroll Help

      • Shift-6(^),v or arrow keys to Move Highlight

      • F9=Reset to Defaults

      • Enter=Select Entry

      • F10=Save

      • Esc= Exit without Save

      • Change setting values with plus (+) and minus (-) keys

      • Use CTRL-] q to exit the console.

    9. From the Device Manager screen, as shown in Figure 1-6, select Advanced.

      Figure 1-6. Device Manager Advanced Option Screen

      Device Manager Advanced Option Screen

    10. From the Advanced screen, select SGI Custom Configuration, as shown in Figure 1-7.

      Figure 1-7. SGI Custom Configuration Option Example

      SGI Custom Configuration Option Example

      From the Advanced screen, as shown in Figure 1-8, perform the following:

      • Change Address Map Configuration Mvalue from 37/7 (44bit) to 36/4 (40bit). Use the plus and minus keys to change these values.

      • Change APIC Mode from Scalable to Legacy.

      • Use F10 to save changes (alt-0 in some terminals)

      Figure 1-8. Advanced Screen APIC Mode Option

      Advanced Screen APIC Mode
Option

    11. You can either boot to the EFI shell and run \efi\boot\bootx64 or enter the boot menus and use the Boot Manager to select the boot loader. Be sure to navigate into the DVD's file hierarchy to find the Windows boot loader. Do not just boot from the DVD at the top level or you will get a legacy boot and Windows will not be able to use X2APIC mode or access more than eight Nehalem-EX sockets (or four Westmere-EX sockets.) Note that the DVD may not be at fs0, depending on what other bootable partitions are available on other drives.

    12. Once the install completes and Windows has booted (using a VGA monitor and attached keyboard and mouse on the Altix UV system), copy the downloaded hotfixes onto the system. Unzip each one to the desktop.


    13. Note: The Windows Server 2008 R2 SP1 release includes the hotfixes by default so you can skip this step if you are install the SP1 release.


      Install the first hotfix, 980598, by clicking on it and going through the dialog steps.

    14. Reboot the system when prompted. The hotfix install completes after reboot. If the system appears to hang on reboot, after at least two minutes, reset the machine as follows:

      system-cmc CMC:r001i01c>power reset
      


    15. Note: The Windows Server 2008 R2 SP1 release includes the hotfixes by default so you can skip this step if you are installing the SP1 release.


      Install the first x2apic hotix - 2303458. Complete the dialogs and reboot when prompted. The hotfix install completes after reboot. Login as the administrator. Start a cmd shell and enter the following command:
      bcdedit x2apicpolicy enable

    16. Reboot the system. When prompted on the console, perform the following:

      • Hit the space bar for the Boot Menu (see Figure 1-4).

      • Select Device Manager (see Figure 1-5), Advanced (see Figure 1-6), SGI Custom Configuration (see Figure 1-7).

      • Change Mvalue back to 37 from 36. Change Nvalue back to 7 from 4. Change APIC Mode to Extended (see Figure 1-9). Use F10 to save changes (alt-0 in some terminals). Use F2 to return to Advanced screen. Use F10 to save changes. Use Esc to return to Device Manager window. Select Continue and accept the reset request.

        Figure 1-9. APIC Mode Extended Example Screen

        APIC Mode Extended Example Screen


    17. Note: The Windows Server 2008 R2 SP1 release includes the hotfixes by default so you can skip this step if you are installing the SP1 release.


      Install the second x2apic hotfix - 2398906. Complete the dialog steps and reboot when prompted. The hotfix install completes after reboot. Shutdown windows to prepare to enable the rest of the system.

    18. Use the hwcfg command on the console to enable NUMAlink, as follows:

      system-cmc CMC:r001i01c>hwcfg nl5_enable=yes

    19. From the CMC perform a power reset, as follows:

      system-cmc CMC:r001i01c>power reset
      

    20. Boot into Windows. This should take about 10 minutes on a 128 processor, one TB memory Altix UV 100 system. A 256 core, 2TB Alitx UV 1000 system takes about 28 minutes to boot Windows. The disk activity LED on the Base I/O, just to the right of the SATA connector, should be flashing. This will be the only indication of activity for most of the wait time.

      The SAC> command will show up on the serial console towards the end of the 10 minutes. It will take a few more minutes before the login screen appears on the VGA monitor. Windows will now take another 20 to 30 minutes, depending on the speed of the root filesystem, to discover new hardware and write their configuration to disk. It will prompt for reboot when complete. Reboot once more to complete the install.

    Enabling a Remote Console

    This section describes how to enable remote console access.


    Note: Remote console is now available and is in BaseIO BMC firmware versions 1.2.2 or later. The node BMC firmware and the BaseIO BMC firmware typically have different version numbers. Make sure the BaseIO BMC firmware is 1.2.2 or later to have the remote console capability.


    You can use the ssh command to connect to the BaseIO blade BMC and use the version command to determine the firmware version, as follows:

    uv44-cmc CMC:r001i01c> ssh ibmc0
    
    SGI UV BMC, Firmware Rev. 1.3.9
    r001i01b00i> version
    SGI Chassis Manager Controller, Firmware Rev. 1.3.9   [Bootloader 0.3.0]
    

    Procedure 1-2. Enabling a Remote Console


      Note: Make sure your workstation has Java 6 (Java 1.6.0 or later). You can also use a VNC viewer or similar remote desktop to make sure you get Firefox running on the SMN and not running on your local system.


      To enable a remote console to your SGI Altix UV system, perform the following steps:

      1. Establish a network connection to the SMN, as follows:

        # ssh -X sysco@uv-system-smn

        The password set out of the factory is sysco.

      2. From the SMN, launch the Mozilla Firefox web browser, as follows:

        sysco@system-smn:~/hw>firefox -no-remote 2>dev/null &

      3. Enter the IP address of the BaseIO node baseboard management controller (BMC), similar to the following:

        http://192.168.1.200

        To determine the IP address of the BaseIO node BMC, see “Determining the IP Address of the BaseIO BMC”.

      4. The BaseIO node BMC login screen appears, as shown in Figure 1-10. Login as root with the password superuser.

        Figure 1-10. BMC Login Screen

        BMC Login Screen

      5. The System Information screen appears, as shown in Figure 1-11. Click on the Remote Control tab.

        Figure 1-11. BMC System Information Screen

        BMC System Information Screen

      6. The Remote Control screen appears, as shown in Figure 1-12. Click on the Console Redirection button.

        Figure 1-12. BMC Remote Control Screen

        BMC Remote Control Screen

      7. The Console Redirection screen appears, as shown in Figure 1-13. Click on the Launch Console button.

        Figure 1-13. BMC Remote Control Console Redirection Screen

        BMC Remote Control Console Redirection Screen

        You may need to enable pop-ups for this address for the Firefox browser first. From the Preferences dialog box, allow access from the IP address of the BaseIO.

      8. A Java application under Firefox will open. Select Execute Java Web Start file (default) from the pop-up menu, as shown in Figure 1-14. You can also click the checkbox to keep it from showing up later.

        Figure 1-14. Select Execute Java Web Start file (default) Pop-up Menu

        Select Execute Java Web Start file (default)
Pop-up Menu

      9. You will then get a screen about The web site's certificate cannot be verified. Do you want to continue? , as shown in Figure 1-15. Click the Yes button to continue. You can also click the checkbox to keep it from showing up later.

        Figure 1-15. Web site Warning-Security Screen

        Web site Warning-Security Screen

      10. You will then get a screen about The application's digital signature cannot be verified. Do you want to run the application?, as shown in Figure 1-16. Click the Run button to continue. You can also click the checkbox to keep it from showing up later.

        Figure 1-16. Application Warning-Security Screen

        Application Warning-Security Screen

      11. The JViewer KVM redirection window appears with a Shell> prompt, as shown in Figure 1-17.

        Figure 1-17. KVM JViewer Shell Prompt Screen

        KVM JViewer Shell Prompt Screen


        Note: You may need to expand the JViewer window in order to see the entire desktop of the system in the following steps.


      12. From the Shell> prompt issue the following command, as shown in Figure 1-18:

        Shell>fs0:\efi\boot\bootx64
        

        Figure 1-18. Booting the Operating System from the JViewer Shell> Prompt Screen

        Booting the Operating System from the  JViewer Shell>
 Prompt Screen

      Determining the IP Address of the BaseIO BMC

      This section describes how to determine the IP address of the BaseIO BMC.

      By default, the external Ethernet port on the BaseIO BMC is configured to use DHCP. In the case that it is connected to an SMN, the SMN will serve it an IP address mapped to a known host name.

      Procedure 1-3. Determining the IP address of the BaseIO BMC without an SMN

        To determine the IP address of the BaseIO, perform the following steps:

        1. Login to the CMC, as follows:

          ssh root@hostname-cmc
          SGI Chassis Manager Controller, Firmware Rev. 1.1.11
          
          CMC:r1i1c>  

        2. Run the iodcon command on the CMC to get to the BaseIO BMC' ibmc0, as follows:

          uv14-cmc CMC:r001i01c> iodcon
          

          The iodcon tool brings up an internal serial connection to the BaseIO BMC (it is serial, which is why you need to hit ENTER to see the prompt in the next step).

          Press the Enter key to get the r001i01b00i> prompt

        3. Run the ifconfig eth1 command to obtain the inet addr, as follows:

          r001i01b00i> ifconfig eth1
          eth1      Link encap:Ethernet  HWaddr 08:00:69:15:C0:57
                    inet addr:172.20.255.254  Bcast:172.20.255.255  Mask:255.255.0.0
                    UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
                    RX packets:1508779 errors:204 dropped:28 overruns:0 frame:204
                    TX packets:2262923 errors:0 dropped:0 overruns:0 carrier:0
                    collisions:0 txqueuelen:1000
                    RX bytes:924130317 (881.3 MiB)  TX bytes:3017177156 (2.8 GiB)
                    Interrupt:1

        4. Enter CTRL-X to exit the console.

        5. The output also displays the MAC address as the HWaddr. You can use this information to configure a known MAC to IP address/host name mapping in your DHCP server.

        Procedure 1-4. Determining the IP address of the BaseIO BMC with an SMN

          To determine the IP address of the BaseIO, perform the following steps:

          1. Establish a network connection to the SMN, as follows:

            # ssh -X sysco@uv-system-smn

            The password set out of the factory is sysco.

          2. Perform the following command to find the IP address of the BaseIO BMC, as follows:

            sysco@uv14-smn:~/hw> cmclist -a
            sysco@uv14-smn:~/hw> cmclist -a
            # File: /var/lib/dhcp/db/dhcpd.leases
            # UID              IP Address
            d \001\010\000i\025\300W 172.20.255.254
            sysco@uv14-smn:~/hw>   

          Setting an Administrative Password on the Console

          This section describes how to use the SGI BIOS setup utility software residing on the BaseIO blade to set an administrative password on the console.

          Procedure 1-5. Setting an Administrative Password on the Console

            To set an administrative password on the console using the BIOS setup utility, perform the following steps:

            1. From the CMC, perform a power reset, as follows:

              system-cmc CMC:r001i01c>power reset
              

            2. When prompted on the console, hit the space bar for the Boot Menu, as shown in Figure 1-19.

              Figure 1-19. Console Prompt for Boot Menu Screen

              Console Prompt for Boot Menu Screen

            3. From the boot manager screen, select Device Manager, as shown in Figure 1-23.

              Figure 1-20. Device Manager Screen

              Device Manager Screen

              Navigate the Device Manager, as follows:

              • F1=Scroll Help

              • Shift-6(^),v or arrow keys to Move Highlight

              • F9=Reset to Defaults

              • Enter=Select Entry

              • F10=Save

              • Esc= Exit without Save

              • Change setting values with plus (+) and minus (-) keys

              • Use CTRL-] q to exit the console.

            4. From the Device Manager screen, select Security, as shown in Figure 1-21.

              Figure 1-21. Device Manager Screen Security Option

              Device Manager Screen Security
 Option

            5. From the Security screen, select the Admin Password option, as shown in Figure 1-22.

              Figure 1-22. Security Screen Admin Password Option

              Security Screen Admin 
Password Option

            6. From the Security Admin Password screen, enter the new administrative password, as shown in Figure 1-23.

              Figure 1-23. Security Screen for Setting Administrative Password

              Security Screen for Setting Administrative
Password

            7. When you reboot the system, the console will be locked and you will be prompted for a password before the system finishes booting and the Shell> prompt appears, as shown in Figure 1-24.

              Figure 1-24. Console Administrative Password Prompt

              Console Administrative Password Prompt

            8. The console password can be deleted by clearing the EFI variables, as follows:

              # power -c reset

              You can change the console password in the same manner you originally set it. If you delete the password with the power -c reset command, you need to go back through the setup screens to set it again.