This chapter describes how to configure the Network Load Balancing Software, in these sections:
“Preparing for Network Load Balancing Software Configuration”
“Network Load Balancing Software and the Shell Command File /etc/init.d/network”
“Configuring the Network Load Balancing Software and Driver”
“Making the Network Load Balancing Interface the Primary Interface”
“ Configuring Multiple Network Load Balancing Software Interfaces”
“Configuring Network Load Balancing Software for Failure Detection Only”
Use the inst(1M) command to install the Network Load Balancing Software. (For information on using inst, see IRIX Admin: Software Installation and Licensing in the Technical Publications Library.)
If you are installing the Network Load Balancing Software for the first time, the subsystems marked default are the ones that are installed if you use the go command from the Inst menu.
To install a different set of subsystems, use the install, remove, keep, and step commands in inst to customize the list of subsystems to be installed, then choose go.
Your system must have an IP address and a network connection name for each Network Load Balancing Software network interface and for each device to be assigned to the Network Load Balancing Software.
IP addresses are usually assigned by a site's network administrator, who obtains them from the Network Information Center.
The network connection name is entered in the /etc/hosts file[1] . This name is paired with an IP address. Each network interface must have one network connection name that is unique to the domain and one globally unique IP address. The following example line that might appear in an /etc/hosts file shows an IP address (from the test network, 192.2.0) and a network connection name, followed by its short form (or alias).
192.0.2.5 sys2.mrktg.group1.com sys2
Generally, network interface IP addresses must always be unique and cannot share a subnet on the same system. (For more information, see information on IP addresses in the latest version of IRIX Admin: Networking and Mail.)
However, network devices assigned to a Network Load Balancing Software device can share a subnet address with the Network Load Balancing Software device or the other network devices assigned to it. For Network Load Balancing Software, these addresses must be unique only within the subnet.
The IP addresses for network devices assigned to a Network Load Balancing Software device must be addresses that can legally appear on the wire. Although these addresses are not used or advertised, they appear in ARP packets.
These addresses need not be on the same network or subnet as the Network Load Balancing Software; you can reuse addresses currently in use on other subnets.
The IP addresses of the devices configured for load balancing and that of the Network Load Balancing Software need not be related, however, if there are multiple systems with Network Load Balancing Software interfaces on the same subnet, all of the attached Ethernet devices for all of these Network Load Balancing Software systems must also have IP addresses on the same subnet as the Network Load Balancing Software devices for the systems to be able to communicate with each other using Network Load Balancing Software.
If there are multiple systems with Network Load Balancing Software interfaces on the same subnet, all of the attached Ethernet devices for these systems must also have IP addresses on the same subnet as the Network Load Balancing Software devices for the systems to be able to communicate with each other using Network Load Balancing Software. When properly configured, the systems will be able to communicate with each other, but there will be no effective increase in the bandwidth between them.
The network portion of the IP address for each local area network must be unique; you cannot use the same network address for your system's Ethernet and Network Load Balancing connections.
To get IP addresses for the Network Load Balancing Software and the devices you are associating with it, follow these steps:
Determine which real devices you are associating with the Network Load Balancing Software.
Obtain an IP address for the Network Load Balancing Software. This address must be unique.
Obtain IP addresses for the real (Ethernet) devices if they have not already been assigned.
If necessary, configure the Ethernet devices with ifconfig. For information on this command, see its man page, ifconfig(1M) .
Note: For specifics, see the sections on planning and setting up a network in the latest version of IRIX Admin: Networking and Mail.
Before starting Network Load Balancing Software configuration, collect the necessary information and perform housekeeping so that the process goes smoothly. Follow these steps:
Determine the number of networks to which your system will be connected after the Network Load Balancing Software is installed.
If the system is to have more than one network connection, decide which will be the primary network. The primary network interface should be the one where all or most of the system's network services or clients reside.
For each network connection, select a network connection name and IP address.
The network connection name of the primary network connection must be the same as the system's hostname. You can display your system's hostname by using the hostname command within a shell window:
% /usr/bsd/hostname
You can display the current IP address associated with the network connection name hostname by typing one of the following commands in a shell window:
% /sbin/grep hostname /etc/hosts % /usr/bin/ypmatch hostname hosts
The names you create for non-primary network interfaces can be anything you want. To facilitate recognition, the names usually include both the hostname and an indication of the protocol (for example, nlb-mars or nlb2-mars).
Determine if any of your system's network interfaces require special configuration for the subnetwork mask (netmask), broadcast address, and route metric.
Table 2-1 summarizes the default operational parameters for the network interfaces.
Table 2-1. Default Network Interface Parameters
Parameter
Default
Description
No subnet.
That is, the bits in the standard network portion of the Internet address are set to 1; the bits in the standard host portion of the Internet address are set to 0 (for class B addresses, 0xFFFF0000; for class C, 0xFFFFFF00).
The netmasks for all of the Ethernet devices to be attached to Network Load Balancing Software devices should be set to 0xffffffff to avoid routing problems.32-bit value used to create two or more subnetworks from a single Internet address, by increasing the number of bits used as the network portion and decreasing the number of bits used as the host portion. When creating the mask, assign 1 to each network bit and 0 to each host bit.
For the Internet address family, the host portion of the IP address is set to 1s (for class B addresses, x.x.255.255; for class C addresses, x.x.x.255).
Address used by this interface for contacting all systems on the local area network.
0
Hop count value advertised by the routing daemon (routed) to other routers. Higher numbers make the route less desirable and less likely to be selected as a route. Settings range from 0 (most favorable) to 16 (least favorable, infinite).
Address Resolution Protocol (ARP) is enabled and used by the interface.
Address Resolution Protocol translates IP addresses to link-layer (hardware) addresses. When this parameter is disabled, the interface does not use ARP.
debug
Disabled
When debugging is enabled, a wider variety of error messages is displayed when errors occur.
If any of these operational parameters needs special configuration, you must create or edit an /etc/config/ifconfig-#.options file, where the pound sign (#) matches the network interface's order in the netif.options file. For example, for the netif.options line if3name=lb0, create or edit the file /etc/config/ifconfig-3.options.
Insure that the proper /etc/config/ifconfig-#.options file specifies the correct subnetwork mask for the associated network.
For complete instructions for configuring operational parameters, see the device configuration instructions in IRIX Admin: Networking and Mail.
Update the site's hosts and ethers databases to include the correct information about this system.
During system startup and any time it is invoked specifically, the shell command file /etc/init.d/network configures and initializes the network interfaces and software. Some of the script's procedures are accomplished by calling other utilities and reading configuration files. Among other tasks, /etc/init.d/network performs the following:
Determines the system's hostname, as defined in the /etc/sys_id file.
Determines the network hardware and interfaces available in the operating system.
This information can be viewed with the hinv command.
Determines the ordering for the network interfaces.
This information is defined in the /etc/config/netif.options file. If the netif.options file has not been altered, the default ordering is configured (as defined in the network script).
Determines the network connection name for each network interface.
This information is defined by the if#name lines in the /etc/config/netif.options file. If the netif.options file has not been altered, the default names (as defined in the network script) are used.
Determines the IP address for each interface by looking up each network connection name in the /etc/hosts file.
Determines the settings for each network interface's operational parameters.
This information is defined in the /etc/config/ifconfig-#.options. If an ifconfig-#.options file does not exist for the interface, the default settings are assigned.
Configures and starts (enables) the number of network interfaces specified by the if_num variable in the network script.
Invokes /etc/init.d/network.ls, which handles the device-specific configuration of the load-balancing devices.
The results of the network script's configuration can be viewed with the /usr/etc/netstat -i and /usr/etc/ifconfig commands.
This section describes how to configure a new network load-balancing network connection for a system with multiple functional Ethernet connections. Configuration consists of assigning Ethernet devices to the network load-balancing device and performing the normal network device configuration for it. Follow these steps:
| Note: You may want to define the lb0 interface BEFORE any of the physical interfaces that make it up. For example, if lb0 is made up of ef0,ef1 and ef2, define lb0 first, ef0, then ef1 and so on. If you do this, Network Load Balancing Software comes up with no incorrect or extra routes that can degrade performance. |
Edit /etc/config/nlb.options to assign the Ethernet devices for which load balancing is to be performed. This file initially contains the following lines:
lb1name=lb0 lb1devs= lb_num=1
To configure one network load-balancing device on the system, set the shell variable lb1devs to a comma-separated list of Ethernet devices to be attached to the network load-balancing device lb0. For example, if four Ethernet devices in the system, ef1, ef2, ef2, and ef4 are to be assigned to the network load-balancing device, nlb.options looks like the following:
lb1name=lb0 lb1devs=ef1,ef2,ef3,ef4 lb_num=1
Note: The lb1devs shell variable comma-separated list cannot contain leading spaces, tabs, or white spaces. If it does, on start up the circuits will not aggregate and an error message similar to the following appears: Configuring Network Load Balancing devices /etc/init.d/network[587]: ef2,: not found
This section provides instructions for adding a Network Load Balancing device to a system with previously configured Ethernet devices. In these instructions, one of the existing Ethernet interfaces serves as the primary network interface; the Network Load Balancing interface serves as the secondary network interface.
| Note: If your site uses an NIS service, the changes described in this section must also be made to the database on the NIS server. |
Follow these steps:
Make sure you have completed preliminary steps:
Obtain network connection names and IP addresses for each Network Load Balancing Software network interface, as described in “IP Addresses and the Network Load Balancing Software”.
Prepare for configuration, as described in “Preparing for Network Load Balancing Software Configuration”.
Configure the software and driver, as described in “Network Load Balancing Software and the Shell Command File /etc/init.d/network”.
If necessary, determine your system's hostname:
# /usr/bsd/hostname
As superuser, open the /etc/hosts file and find the line containing your system's hostname. This line configures your system's Ethernet network interface. For multiport Ethernet configuration, see step 4.
If you find the Network Load Balancing Software name or address, verify that the IP address and the name are correct for the Network Load Balancing Software connection.
If you do not find an entry for this name, search for the Network Load Balancing Software IP address. Searching for all instances of the system's hostname usually identifies all the network connection names for the system.
If the line is not correct or is missing, edit the file so that there is a line containing the IP address and network connection name for the Network Load Balancing Software network interface.
A typical format for an entry in /etc/hosts file is as shown:
IPaddress full_network_connectionname alias
For example, a portion of an /etc/hosts file might look like this, where the host sys2 has two entries and sys3 has one:
192.0.2.1 sys3.mrktg.group1.com sys3 192.0.2.5 sys2.mrktg.group1.com sys2 192.0.2.8 nlb-sys2.engr.group1.com nlb-sys2
To configure your system for multiple Ethernet ports, edit the /etc/hosts similar to the following example:
# Default IP address for a new IRIS. It should be changed immediately to # the address appropriate for your network. # (The '192.0.2' network number is the officially blessed 'test' network.) 134.16.229.48 sqns3.engr.sgi.com sqns3 # This entry must be present or the system will not work. 127.0.0.1 localhost # # lb0: interfaces # 165.154.216.30 nlb-sqns3.engr.sgi.com nlb-sqns3 165.154.216.31 nlb1-sqns3.engr.sgi.com nlb1-sqns3 165.154.216.32 nlb2-sqns3.engr.sgi.com nlb2-sqns3 165.154.216.33 nlb3-sqns3.engr.sgi.com nlb3-sqns3 165.154.216.34 nlb4-sqns3.engr.sgi.com nlb4-sqns3
Edit the /etc/config/netif.options similar to the following example:
if1name=ef0 if1addr=$HOSTNAME if5name=ef1 if5addr=nlb1-$HOSTNAME if6name=ef2 if6addr=nlb2-$HOSTNAME if7name=ef3 if7addr=nlb3-$HOSTNAME if8name=ef4 if8addr=nlb4-$HOSTNAME if9addr=nlb-$HOSTNAME
Edit the /etc/config/nlb.options similar to the following example:
lb1name=lb0 lb1devs=ef1,ef2,ef3,ef4 lb_num=4
If your system uses more than one network connection, for each one (in addition to Network Load Balancing Software), verify that the name and IP address are the correct; follow instructions in step 3.
Copy the line containing your system's hostname and place the copy immediately below the original in the /etc/hosts file. This new line configures your system's Network Load Balancing network interface.
On the new line, change each instance of your system's hostname to nlb-hostname. That is, precede your system's hostname with nlb-.
On the same (new) line, change the address (digits on the left) to the Network Load Balancing IP address.
For example, the lines for a system with a hostname of sys3 in the domain group1.com looks like this:
x.x.x.x sys3.group1.com sys3 #Ether primary x.x.x.x nlb-sys3.group1.com nlb-sys3 #Load Bal. secondary
In these entries, each x represents one to three digits.
Save and close /etc/hosts.
Edit /etc/config/netif.options so that the network startup script can configure the load-balancing devices. Find the first two pairs of lines showing interface names that are not currently occupied.
In the following example, the first two pairs are in use:
if1name= ef0 if1addr=$HOSTNAME if2name= ef1 if2addr=gate-$HOSTNAME : if3name= : if3addr=gate2-$HOSTNAME : if4name= : if4addr=gate3-$HOSTNAME
In this case, you change
: if3name= : if3addr=gate2-$HOSTNAME
to
if3name= lb0 if3addr=nlb-$HOSTNAME
Note: Make sure the names or name formats you enter correspond to entries in the /etc/hosts file. If this file has, for example, five pairs of entries for five Ethernets already configured on the system, you would add the following two lines below those entries:
if6name=lb0 if6addr=nlb-$HOSTNAME
Note: You do not need to edit the entries for the existing Ethernet interfaces. Each Ethernet to be attached to the Network Load Balancing device must have an IP address assigned, even though these addresses are not actually used. You might need to change the addresses of client systems previously on the subnets accessed via the Ethernets that are being assigned to the Network Load Balancing interface. Save and close /etc/config/netif.options.
If applicable, edit /etc/config/nlb.options to attach additional Ethernets to the Network Load Balancing device. (See “Configuring the Network Load Balancing Software and Driver” for information on this file.)
Build your changes into the operating system:
# /etc/autoconfig Automatically reconfiguring the operating system.
Note: When setting up Network Load Balancing software, you can edit etc/config/nlb.options or /etc/config/netif.options as appropriate and then apply all of your changes using the autoconfig command. Turn on the Network Load Balancing Software using chkconfig.
Reboot the system to start using the reconfigured kernel.
Once the system is rebooted, output from netstat -in might look like that in Figure 2-2.
This section provides instructions for configuring a system so that the Network Load Balancing connection serves as the primary network interface and an Ethernet interface serves as the secondary one.
Follow these steps:
Open a shell window and log in as superuser.
If necessary, determine your system's hostname:
# /usr/bsd/hostname
Open the /etc/hosts file and find the line containing your system's hostname.
If the file does not contain a line for your hostname, enter it; for example:
192.0.2.1 sys3.mrktg.group1.com sys3
Copy the line and place the copy immediately below the original.
On the original line, change the address (the digits on the left) to the IP address for the Network Load Balancing network. This line configures your system's Network Load Balancing network interface. Do not change the hostname.
On the new line, change the hostname to the new connection name, which is used for the original primary Ethernet (such as gate5-sys3 in the example below).
On this new line, do not change the IP address (which represents the Ethernet connection).
The following example shows entries in /etc/hosts for a system with a hostname of sys3 in the domain group1.com:
x.x.x.x sys3.group1.com sys3 #NLBS primary x.x.x.x gate5-sys3.group1.com gate5-sys3 #Ether secondary x.x.x.x gate1-sys3.group1.com gate1-sys3 #NLBS attached to lb0 x.x.x.x gate2-sys3.group1.com gate2-sys3 #NLBS attached to lb0 x.x.x.x gate3-sys3.group1.com gate3-sys3 #NLBS attached to lb0 x.x.x.x gate4-sys3.group1.com gate4-sys3 #NLBS attached to lb0
In this example, each x represents one to three decimal digits.
Save and close /etc/hosts.
If your site uses an NIS service, the changes described above must also be made to the database on the NIS server.
Open the /etc/config/netif.options file .
Any Network Load Balancing Software device listed in /etc/config/netif.options must appear after the Ethernets that are to be attached to it. For example, you might be attaching ef1 and ef2 to lb0, so they might appear as follows:
if2name=ef1 if2addr=gate-$HOSTNAME if3name=ef2 if3addr=gate2-$HOSTNAME if4name=lb0 if4addr=nlb-$HOSTNAME
where NLBinterface is the name of the Network Load Balancing interface. For example:
if4name=lb0 if4addr=nlb-$HOSTNAME
To make an Network Load Balancing Software device be the primary, you have to use the appropriate /etc/config/ifconfig-*.options file. In the example above, you can make lb0 the primary interface by adding the ifconfig(1M) keyword primary to /etc/config/ifconfig-4.options. For example, the ifconfig-4.options file might contain the following:
netmask 0xffffff00 primary
Save and close /etc/config/netif.options.
If applicable, edit /etc/config/nlb.options to attach additional Ethernets to the Network Load Balancing device. (See “Configuring the Network Load Balancing Software and Driver” for information on this file.)
Build your changes into the operating system:
# /etc/autoconfig Automatically reconfiguing the operating system.
Turn on the Network Load Balancing Software using chkconfig.
Reboot the system to start using the reconfigured kernel.
Figure 2-2 shows netstat -in output with a load-balancing device as the primary Ethernet interface.
To configure multiple network load-balancing devices, follow these steps:
Follow instructions for configuring and adding the load-balancing device to the system, as described in this chapter.
Use systune to change the tunable parameter, lb_devices. Enter
systune lb_devices numberofdevices
For example, this command sets the number of load-balancing devices to 2:
systune lb_devices 2
For information on this command, see its man page, systune(1M) .
Edit /etc/config/nlb_options:
Set lb1name and lb2name to the load-balancing device names you are using.
Set the shell variable lb1devs to a comma-separated list of Ethernet devices to be attached to the network load-balancing device lb0.
Change the shell variable lb_num from the default of 1 to the number of network load-balancing devices configurable on the system.
Save and close the file.
The following example shows entries for two network load-balancing devices:
lb1name=lb0 lb1devs=ef1,ef2 lb2name=lb1 lb2devs=ef3,ef4 lb_num=2
For changes in the systune lb_devices parameter to take effect, build your changes into the operating system:
# /etc/autoconfig Automatically reconfiguring the operating system.
Turn on the Network Load Balancing Software using chkconfig.
Reboot the system to start using the reconfigured kernel.
Network Load Balancing Software can be configured with the load balancing features disabled, leaving only the failure detection and recovery operational. You can do this by setting the appropriate lbXopt value in the /etc/config/nlb.option file, where X is the unit number. For example, the /etc/config/nlb.options file may contain the following:
lb1name=lb0 lb1devs=ef1,ef2 lb1opt=-F lb_num=1 |
This will configure the Network Load Balancing Software device with load balancing disabled. For a system with multiple Network Load Balancing Software devices, the nlb.options file may contain the following:
lb1name=lb0 lb1devs=ef1,ef2 lb1opt=-F lb2name=lb1 lb2devs=ef3,ef4 lb_num=2 |
In the example above, lb0 is configured with load balancing disabled while lb1 is configured normally (load balancing enabled). By default, Network Load Balancing Software devices will be configured with load balancing enabled. In the above example, the absence of lb2opt results in a default configuration for lb1.
To verify that the system's Network Load Balancing Software connection is working, follow these steps:
List all network interfaces on the system. A Network Load Balancing Software interface, such as lb#, must be listed.
% /usr/etc/netstat -in
If the listing does not include a Network Load Balancing Software interface, the Network Load Balancing Software may not have been installed, may not be configured, or may not be built into the operating system. Fix the problem before continuing.
Check that the load-balancing configuration is correct by entering
% /usr/etc/lbconfig interfacename
where interfacename is the name of a load-balancing interface, such as lb0. For example:
# lbconfig lb0 Interfaces configured for lb0: ef1,ef2,ef3,ef4
Compare the output of this command to the configuration in /etc/config/nlb.options. If it does not correspond, reboot and check again. If it still does not correspond, contact your service provider.
Use ifconfig to get the broadcast address:
% /etc/ifconfig interfacename
Use the broadcast address in a ping command, as follows:
% /usr/etc/ping broadcastaddress
Verify that all clients on the subnet respond.
You can use lbstat -C to further evaluate the state of the load-balancing device(s) on the system. In particular, the state for each Ethernet port attached to the load-balancing device should be LINK_GOOD. If a port does not show this status, check its cabling.
[1] For more information about the /etc/hosts file, see the hosts(4) man page and IRIX Admin: Networking and Mail.