You may want to reach network services outside of your SGI Altix ICE 8200 system. For this type of access, SGI recommends using Network Address Translation (NAT), also known as IP Masquerading or Network Masquerading. Depending on the amount of network traffic and your site needs, you may want to have multiple service nodes providing NAT services.

To enable NAT on your service node, perform the following steps:

system-1:/ # netstat -rn
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
128.162.244.0   0.0.0.0         255.255.255.0   U         0 0          0 eth0
172.16.0.0      0.0.0.0         255.255.0.0     U         0 0          0 eth1
169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 eth0
172.17.0.0      0.0.0.0         255.255.0.0     U         0 0          0 eth1
127.0.0.0       0.0.0.0         255.0.0.0       U         0 0          0 lo
0.0.0.0         128.162.244.1   0.0.0.0         UG        0 0          0 eth0

tcpdump -i eth1 ip proto ICMP # Dump ping packets on the public side of service node.
tcpdump -i ib1 ip proto ICMP # Dump ping packets on the IB fabric side of service node.
tcpdump -i eth1 port nfs # Dump NFS traffic on the eth1 side of service node.
tcpdump -i ib1 port nfs # Dump NFS traffic on the eth1 side of service node.

You may chose to connect your compute nodes using routable addresses on the house network. This requires planning before the installation by reserving a large block of routable IP addresses on the house network and the correct steps early in installation.

---

Note: Placing a fabric on the house network does make it more susceptible to bandwidth and latency fluctuations due to undesired or unexpected network traffic.

---

To connect your compute nodes using routable addresses on the house network, perform the following steps:

For information on setting up DNS, see “Installing Software on the System Admin Controller”.

Assuming the installation has either NAT or Gateway operations configured on one or more service nodes, the compute nodes can directly mount the house NFS server's exports (see the exports(5) man page).

To allow the compute nodes to directly mount the house NFS server's exports, perform the following steps:

This section describes two different ways to configure a service node for NIS, as follows:

Assuming the installation has either Network Address Translation (NAT) or Gateway operations configured on one or more service nodes, the compute nodes can directly access the house NIS servers. Broadcast operations for discovering NIS servers do not typically work. Therefore, you need to configure the compute images with the IP address of the NIS server to which you want them to connect.

To configure NIS on a compute node, perform the following steps:

To configure NIS with a service node as a NIS slave server to the house NIS master, perform the following steps:

---

Note: Multiple service nodes can be used as NIS slave servers.

---

---

Note: Steps 1 through 7 of this procedure are performed on the service node. Steps 8 and 9 are performed from the system admin controller (admin node).

---

For information on centrally managed user accounts, see “Setting Up a NIS Server for Your Altix ICE System”. It describes NIS master set up. In this design, the master server residing on the service node provides the filesystem and the NIS slaves reside on the rack leader controllers. If you have more than one home server, you need to export all home filesystems on all home servers to the server acting as the NIS master. You also need to export the filesystems to the NIS master using the no_root_squash exports flag.

If you want to use NAS server for scratch storage or make home filesystems available on NAS, you can follow the instructions in “Setting Up an NFS Home Server on a Service Node for Your Altix ICE System”. In this example, you need to replace service-ib1 with the ib1 InfiniBand host name for the NAS server and you need to know where on the NAS server the home filesystem is mounted to craft the sgi-fstab script properly.

In the procedures that follow in this section, here are some of the tasks you need to perform and system features you need to consider:

This section describes how to set up a service node as a NIS master. This section only applies to service nodes running SLES10.

To set up a service node as a NIS master, perform the following steps:

---

Note: These instructions use the text-based version of YaST. The graphical version of YaST may be slightly different.

---

This section describes how to use YaST to set up your other service nodes to be broadcast binding NIS clients. This section only applies to service nodes running SLES10.

---

Note: You do not do this on the NIS Master service node that you already configured as a client in “Setting Up a Service Node as a NIS Master”.

---

To set up a service node as a NIS client, perform the following steps:

This section provides two sets of instructions for setting up rack leader controllers (leader nodes) as NIS slave servers. One set of instructions uses YaST, the other uses a set of commands that could be scripted if you so choose. It is possible to make all these adjustments to the leader image in /var/lib/systemimager/images. Currently, SGI does not recommend using this approach.

---

Note: Be sure the InfiniBand interfaces are up and running before proceeding because the rack leader controller gets its updates from the NIS Master over the InfiniBand network. If you get a "can't enumerate maps from service0" error, check to be sure the InfiniBand network is operational.

---

Use the following set of commands to set up a rack leader controller (leader node) as a NIS slave server and client.

---

Note: Replace ice with your NIS domain name and service0 with the service node you set up as the master server.

---

# cexec --head chkconfig ypserv on
   # cexec --head chkconfig ypbind on
   # cexec --head chkconfig portmap on
   # cexec --head chkconfig nscd on
   # cexec --head rcportmap start
   # cexec --head "echo ice > /etc/defaultdomain"
   # cexec --head "ypdomainname ice"
   # cexec --head "echo ypserver 127.0.0.1 > /etc/yp.conf"
   # cexec --head "echo +::: >> /etc/group"
   # cexec --head "echo +:::::: >> /etc/passwd"
   # cexec --head "echo + >> /etc/shadow"
   # cexec --head /usr/lib/yp/ypinit -s service0
   # cexec --head rcportmap start
   # cexec --head rcypserv start
   # cexec --head rcypbind start
   # cexec --head rcnscd start

This section describes how to set up the compute nodes to be NIS clients. You an configure NIS on the clients to use a server list that only contains the their rack leader controller (leader node). All operations are performed from the system admin controller (admin node).

To set up the compute nodes to be NIS clients, perform the following steps:

The NAS cube needs to get configured with each InfiniBand fabric interface in a separate subnet. These fabrics will be separated from each other logically, but attached to the same physical network. For simplicity, this guide assumes that the -ib1 fabric for the compute nodes has addresses assigned in the 10.149.0.0/16 network. This guide also assumes the lowest address the cluster management software has used is 10.149.0.1 and the highest is 10.149.1.3 (already assigned to the NAS cube).

For the NAS cube, you need to configure the large physical network into four, smaller subnets, each of which would be capable of containing all the nodes and service nodes. It will have subnets 10.149.0.0/18 , 10.149.64.0/18, 10.149.128.0/18 , and 10.149.192.0/18.

After the discovery of the storage node has happened, SGI personnel will need to log onto the NAS box and change the network settings to use the smaller subnets, and then define the other three adapters with the same offset within the subnet; for example: Initial configuration of the storage node had set ib0 fabric's IP to 10.149.1.3 netmask 255.255.0.0. After the addresses are changed, ib0=10.149.1.3:255.255.192.0, ib1=10.149.65.3:255.255.192.0 , ib2=10.149.129.3:255.255.192.0, ib3=10.149.193.3:255.255.192.0 . The NAS cube should now have all four adapter connections connected to the fabric with IP addresses which can be pinged from the service node.

---

Note: The service nodes and the rack leads will remain in the 10.149.0.0/16 subnet.

---

For the compute blades, log into the admin node and modify /opt/sgi/share/per-host-customization/global/sgi-setup-ib-configs file. Following the line iruslot=$1, insert:

# Compute NAS interface to use
IRU_NODE=`basename ${iruslot}`
RACK=`cminfo --rack`
RACK=$(( ${RACK} - 1 ))
IRU=`echo ${IRU_NODE} | sed -e s/i// -e s/n.*//`
NODE=`echo ${IRU_NODE} | sed -e s/.*n//`
POSITION=$(( ${IRU} * 16 + ${NODE} ))
POSITION=$(( ${RACK} * 64 + ${POSITION} ))
NAS_IF=$(( ${POSITION} % 4 ))
NAS_IPS[0]="10.149.1.3"
NAS_IPS[1]="10.149.65.3"
NAS_IPS[2]="10.149.129.3"
NAS_IPS[3]="10.149.193.3"

Then following the line $iruslot/etc/opt/sgi/cminfo add:

IB_1_OCT12=`echo ${IB_1_IP} | awk -F "." '{ print $1 "." $2 }'`
IB_1_OCT3=`echo ${IB_1_IP} | awk -F "." '{ print $3 }'`
IB_1_OCT4=`echo ${IB_1_IP} | awk -F "." '{ print $4 }'`
IB_1_OCT3=$(( ${IB_1_OCT3} + ${NAS_IF} * 64 ))
IB_1_NAS_IP="${IB_1_OCT12}.${IB_1_OCT3}.${IB_1_OCT4}"

Then change the IPADDR='${IB_1_IP}' and NETMASK='${IB_1_NETMASK}' lines to the following:

IPADDR='${IB_1_NAS_IP}'
NETMASK='255.255.192.0'

Then add the following to the end of the file:

# ib-1-vlan config
cat << EOF >$iruslot/etc/sysconfig/network/ifcfg-vlan1
# ifcfg config file for vlan ib1
BOOTPROTO='static'
BROADCAST=''
ETHTOOL_OPTIONS=''
IPADDR='${IB_1_IP}'
MTU=''
NETMASK='255.255.192.0'
NETWORK=''
REMOTE_IPADDR=''
STARTMODE='auto'
USERCONTROL='no'
ETHERDEVICE='ib1'
EOF
if [ $NAS_IF -eq 0 ]; then
    rm $iruslot/etc/sysconfig/network/ifcfg-vlan1
fi

To update the fstab for the compute blades, edit /opt/sgi/share/per-host-customization/global/sgi-fstab file. Perform the equivalent steps as above to add the # Compute NAS interface to use section into this file. Then to specify mount points, add lines similar to the following example:

# SGI NAS Server Mounts
${NAS_IPS[${NAS_IF}]}:/mnt/data/scratch     /scratch nfs    defaults 0 0

If you add or remove a rack leader controller (leader node), for example, if you use discover command to discover a new rack of equipment, you will need to configure the new rack leader controller to be an NIS slave server as described in “Setting Up a Service Node as a NIS Client”.

In addition, you need to add or remove the leader from the /var/yp/ypservers file on NIS Master service node. Remember to use the -ib1 name for the leader, as service nodes cannot resolve r2lead style names. For example, use r2lead-ib1.

# cd /var/yp && make

The example used in this section assumes that the home directory is mounted on the NIS Master service and that the NIS master is able to create directories and files on it as root. The following example use command line commands. You could also create accounts using YaST.

To create user accounts on the NIS server, perform the following steps:

SGI supplies updates to SGI Tempo software via the SGI update server at https://update.sgi.com/. Access to this server requires a Supportfolio login and password. Access to SUSE Linux Enterprise Server updates requires a Novell login account and registration.

The initial installation process for the SGI Altix ICE system set up a number of package repositories in the /tftpboot directory on the admin node. The SGI Tempo related packages are in the /tftpboot/oscar directory and SUSE Linux Enterprise Linux (SLES) packages are in the /tftpboot/distro/sles-10-x86_64 directory.

When SGI releases updates, you may run a script or the wget(1) command to download the updated packages (for example, patches) to the appropriate package repository in /tftpboot directory and then update the repository metadata.

Once the local repositories contain the updated packages, it is possible to update the various SGI Altix ICE admin, leade, and managed service node images using the yum(8) command. The yum update and yum install commands are used for all updates, including updates to existing images.

If you are upgrading from SGI ProPack 5 SP2 to SGI ProPack 5 SP3, and have never set the system up to retrieve online updates, you will have to follow the instructions documented on this web page: https://update.sgi.com/sles/SGI-ProPack/5/x86_64/tempo/docs/ice_patch_doc.txt and upgrade using the online update mechanism; just like you would do to get the latest patches. SGI does not support going from SGI ProPack 5 SP2 GA with no patches installed to SGI ProPack 5 SP3 because the update mechanism might not be set up.

For additional information on updating your system, see “Upgrading from SGI ProPack 5 SP2 to SGI ProPack 5 SP3 ”.

There is a small amount of preparation required, in order to setup an SGI Altix ICE system, so that updated packages can be downloaded from the SGI update server and then installed with the yum command.

This following sections describe these steps:

In order to prepare the SGI Altix ICE system to download and install updated packages, you first need to manually upgrade the sgi-admin-node and sgi-cluster packages on the admin node. These packages contain scripts and tools for managing updates.

---

Note: The SGI Altix ICE systems pre-installed by SGI will already have versions of the sgi-admin-node and sgi-cluster packages that meet the minimum requirements. Check to see if the package versions installed on your admin node are the same or newer than those mentioned below. If you are re-installing an SGI Altix ICE system using the initial SGI Altix ICE System Installation Media (part number 811-1251-001), you will need to follow this procedure.

---

Do not attempt to configure an admin node for updates before you have performed the steps in the configure-cluster script. The configure-cluster script must be run first because it creates the initial repositories described in the “Overview”. It is possible to prepare the admin node for updates before discovering cluster components as long as the steps in the configure-cluster script has completed successfully.

To download and install the sgi-admin-node and sgi-cluster packages, perform the following steps:

Once the sgi-admin-node and sgi-cluster update packages are installed, you will have everything you need to update the local package repositories on your admin node with the latest updates for SGI Tempo, SGI ProPack and SLES.

A copy of update instructions are also available at https://update.sgi.com/sles/SGI-ProPack/5/x86_64/tempo/docs/ice_patch_doc.txt that you can download to your system admin controller for quick reference.

This section explains how to update the local product package repositories needed to share updates on all of the various nodes on an SGI Altix ICE system.

Update the SGI Package Repositories on the Admin Node

SGI provides an example script called sync-repo-updates to help keep your local package repositories on the admin node synchronized with available updates for the SGI Tempo, SGI ProPack and SLES products. You can use this script directly or use it as a template to develop more customized solutions. The script is located in /opt/sgi/sbin/sync-repo-updates on the admin node.

The sync-repo-updates script requires two parameters; your Supportfolio user name and password. Using that information, the script contacts SGI's update server and downloads the updated packages into the appropriate local package repositories. If you installed and configured the yup tool as described in “Update the SLES Package Repository”, the sync-repo-updates script will also download any updates to SLES from the Novell update server. When all package downloads are complete, the script runs the yume command to update the repository metadata.

Once the script completes, the local package repositories on the admin node should contain the latest available package updates and be ready to use with the yum(8) command to update node images.

---

Note: If you manually add updates to any of the local package repositories on the admin node, remember to run the yume --prepare --repo command to update the package repository metadata. Failure to do so, will cause the yum command to report checksum failures

---

“Update the SLES Package Repository” contains further details on using the sync-repo-updates script to get SLES package updates. The sync-repo-updates script will only try to get updates if certain conditions are met, so it's safe to use the script to update the local SGI Tempo and SGI ProPack repositories before completing the tasks listed in that section.

To install updates on the admin node, run the following command as root on the admin node:

# yum update

Because SGI pre-installs certain package updates on systems before they leave the factory, there may not be any package updates updates for the admin node.

To install the SGI Altix ICE System Administrator's Guide and other SGI ProPack related documentation, run the following command:

# yum install sgi-propackdocs

This sections explain how to configure existing node images in the /var/lib/systemimager image directories and node images in use on live nodes so that you can install update packages for the SGI Tempo, SGI ProPack and SLES products.

If you want to quickly configure node images already running on a live system so that you can update packages on those nodes right away, start with “Prepare Running Leader and Managed Service Nodes for Updates”. If you would rather configure images in the image directories first so that you can update them before re-deploying those images to the system nodes, skip down to “Prepare Existing Images for Updates”.

# scp /opt/sgi/sbin/setup-for-updates r1lead:/tmp
# ssh r1lead /tmp/setup-for-updates -r

# scp /opt/sgi/sbin/setup-for-updates service0:/tmp
# ssh service0 /tmp/setup-for-updates -r

Prepare Existing Images for Updates

The setup-for-updates script is used to prepare images and nodes for updates by installing yum and an updated version of the sgi-cluster package.

---

Note: If you are managing an admin node that was pre-installed by SGI, the images and nodes may already be setup properly for updates. If you are installing an admin node and/or your entire SGI Altix ICE system from scratch, you should follow the instructions in this section.

---

If the nodes or images are already setup to handle updates, the setup-for-updates script will safely exit. The script will prepare all SGI- supplied images for updates by installing yum and updating the sgi-cluster package if it does not meet the minimum requirements.

---

Note: Updating an image and updating a node are not synonymous. To have leader and managed service nodes use an updated image, you must either re-discover the components or update the nodes while they are running (see section "5.1 Update Packages on Running Leader and Managed Service Nodes" for more details). To update the compute node image, you must use the cimage command to deploy the changes to the compute image on the admin node.

---

The following examples illustrate how to use the setup-for-updates script to prepare existing images for updates:

# setup-for-updates -i /var/lib/systemimager/images/lead-sles10sp1/ # setup-for-updates -i /var/lib/systemimager/images/service-sles10sp1/ # setup-for-updates -i /var/lib/systemimager/images/compute-sles10sp1/

For more information about using cimage to manage compute images, “cimage Command” in Chapter 3. The cimage --help command also provides some help text.

The following sections explain how to update leader, service and compute node images in the /var/lib/systemimager image directories with newer packages, how to update leader and service node images in use on live nodes with new packages and the additional steps required when updating the kernel package on compute node images.

If you want to quickly install the latest updates to the node images already running on a live system, start with “Update Packages on Running Leader and Managed Service Nodes”. If you would rather install the latest update packages to node images in the image directories first and then redeploy those images to the system nodes, go to “Update Packages Inside Images”.

It is possible to update live images running on leader and managed service nodes using the yum command.

---

Note: Changes made to live node images are not reflected in the node images in the /var/lib/systemimager/images/{image}-sles10sp1 image directories. This means that changes made to live images may not get carried forward the next time you add or re-discover leader and/or service nodes.

---

The following examples show how to update a live leader node and/or managed service node using the yum command:

# ssh r1lead yum update   (update live leader node)
# ssh service0 yum update      (update live service node)

If you have not already prepared your images for updates as described in sections “Prepare Existing Images for Updates” and/or “Prepare Running Leader and Managed Service Nodes for Updates”, go back and complete those steps first.

Assuming you have prepared your images for updates, there are couple different ways to update packages inside images.

SGI provides the yum-image-wrapper script, which makes using the yum command inside an image directory fairly straight forward. You can use the script directly, or use the script as a template for a more customized solution.

---

Note: Changes to the kernel package inside the compute image directory require some additional steps before the new kernel can be used on compute nodes (see section "5.3 Additional Steps for Compute Image Kernel Updates" for more details). This note does not apply to leader or managed service nodes.

---

The following examples show how to upgrade the packages inside three SGI- supplied node images :

# yum-image-wrapper /var/lib/systemimager/images/lead-sles10sp1 update
# yum-image-wrapper /var/lib/systemimager/images/service-sles10sp1 update
# yum-image-wrapper /var/lib/systemimager/images/compute-sles10sp1 update

---

Note: Changes to the compute image on the admin node are not seen by the compute nodes until the updates have been pushed to the leader nodes with the cimage command. Updating leader and managed service node images ensure that the next time you add or re-discover a leader or service node, it will already contain the updated packages.

---

While not recommended, it is possible to manually perform all the steps the yum-image-wrapper does for you. The following series of steps will walk you through the manual process:

It is also possible to use standard rpm (8) commands to update images. This can be quite cumbersome depending on the number of images you need to update and the number of packages you intend to update. The steps are as follows:

In general, SGI recommends using the yum-image-wrapper script to update the node images in the image directory.

Any time a compute image is updated with a new kernel, you will need to run some additional steps in order to make the new kernel available. The following example assumes that the compute node image name is compute-sles10sp1 and that you have already updated the compute node image in the image directory per the instructions in “Update Packages Inside Images”. If you have named your compute node image something other than compute-sles10sp1, replace this in the example that follows: