Questions you must answer during the planning process are as follows:

As an example of the configuration planning process, suppose that you have a two-node FailSafe cluster that is a departmental server. You want to make four XFS filesystems available for NFS mounting and have two Netscape FastTrack servers, each serving a different set of documents. These applications will be HA services.

You decide to distribute the services across two nodes, so each node will be the primary node for two filesystems and one Netscape server. The filesystems and the document roots for the Netscape servers (on XFS filesystems) are each on their own plexed XLV logical volume. The logical volumes are created from disks in a Fibre Channel storage system connected to both nodes.

There are four resource groups:

NFSgroup1 and NFSgroup2 are the NFS resource groups; Webgroup1 and Webgroup2 are the Web resource groups. NFSgroup1 and Webgroup1 will have one node as the primary node. NFSgroup2 and Webgroup2 will have the other node as the primary node.

Two networks are available on each node, ef0 and ef1. The ef1 interfaces in each node are connected to each other to form a private network.

Figure 2-1 depicts this configuration.

Figure 2-1. Example Configuration with Four Resource Groups

For each disk in a FailSafe cluster, you must choose whether to make it a shared disk, which enables it to be failed over, or a non-shared disk. Non-shared disks are not failed over.

The nodes in a FailSafe cluster must follow these requirements:

Only HA application data and configuration data can be placed on a shared disk. Choosing to make a disk shared or non-shared depends on the needs of the HA services that use the disk. Each HA service has requirements about the location of data associated with the service:

The figures in the remainder of this section show the basic disk configurations on FailSafe clusters before and after failover. A cluster can contain a combination of the following basic disk configurations:

---

Note: In each of the before and after failover diagrams, each disk shown can represent a set of disks.

---

Figure 2-2 shows two nodes in a cluster, each of which has a non-shared disk with two resource groups. When non-shared disks are used by HA applications, the data required by those applications must be duplicated on non-shared disks on both nodes. The clients should access the data in the shared disk using HA IP alias. When a failover occurs, HA IP aliases fail over.

---

Note: The hostname is bound to a different IP address that never moves.

---

The data that was originally available on the failed node is still available from the replacement node by using the HA IP alias to access it.

The configuration in Figure 2-2 contains two resource groups:

Figure 2-2. Non-Shared Disk Configuration and Failover

Figure 2-3 shows a two-node configuration with one resource group, Group1:

In this configuration, the resource group Group1 has a primary node, which is the node that accesses the disk prior to a failover. It is shown by a solid line connection. The backup node, which accesses the disk after a failover, is shown by a dotted line. Thus, the disk is shared between the nodes. In an active/backup configuration, all resource groups have the same primary node. The backup node does not run any HA resource groups until a failover occurs.

Figure 2-3. Shared Disk Configuration for Active/Backup Use

Figure 2-4 shows two shared disks in a two-node cluster with two resource groups, Group1 and Group2:

In this configuration, each node serves as a primary node for one resource group. The solid line connections show the connection to the primary node prior to fail over. The dotted lines show the connections to the backup nodes. After a failover, the surviving node has all the resource groups.

Figure 2-4. Shared Disk Configuration for Dual-Active Use

Other sections in this chapter and similar sections in the IRIS FailSafe 2.0 Oracle Administrator's Guide and IRIS FailSafe 2.0 INFORMIX Administrator's Guide provide more specific information about choosing between shared and non-shared disks for various types of data associated with each HA service.

There are no configuration parameters associated with non-shared disks. They are not specified when you configure a FailSafe system. Only the XLV logical volumes on shared disks are specified at configuration. For more information, see “Resource Attributes for Logical Volumes”.

For information on using CXFS filesystems (which use XVM logical volumes) in a FailSafe configuration, see “Coexecution of CXFS and FailSafe”.

All shared disks must contain XLV logical volumes. You can work with XLV logical volumes on shared disks as you would work with other disks. However, you must follow these rules:

The FailSafe software relies on the XLV naming scheme to operate correctly. A fully qualified XLV volume name uses one of the following formats:

pathname/volname
pathname/nodename.volname

The components are these:

For example, if volume vol1 is created on node ha1 using disk partitions located on a shared disk, the raw character device name for the assembled volume is /dev/rxlv/vol1. On the peer ha2, however, the same raw character volume appears as /dev/rxlv/ha1.vol1, where ha1 is the nodename component and vol1 is the volname component. As can be seen from this example, when the nodename component is the same as the local hostname, it does not appear as part of the device node name.

One nodename is stored in each disk or LUN volume header. This is why all volumes with volume elements on any single disk must have the same nodename component.

---

Caution: If this rule is not followed, FailSafe does not operate correctly.

---

FailSafe modifies the nodename component of the volume header as volumes are transferred between nodes during failover and recovery operations. This is important because xlv_assemble assembles only those volumes whose nodename matches the local hostname. Some of the other XLV utilities allow you to see (and modify) all volumes, regardless of which node owns them.

The resource name for a resource of resource type volume is the XLV volume name.

If you use XLV logical volumes as raw volumes (that is, with no filesystem) for storing database data, the database system may require that the device names in /dev/xlv have specific owners, groups, and modes. See the documentation provided by the database vendor to determine if the XLV logical volume device names must have owners, groups, and modes that are different from the default values (the defaults are root, sys, and 0600, respectively).

As an example of logical volume configuration, say that you have the following logical volumes on disks that we will call Disk1 through Disk5:

VolA and VolB must be part of the same resource group because they share a disk. VolC could be part of any resource group. Figure 2-5 describes this.

Figure 2-5. Example Logical Volume Configuration

Table 2-1 lists the resource attributes for XLV logical volumes.

See the section “Create XLV Logical Volumes and XFS Filesystems” in Chapter 3, for information about creating XLV logical volumes.

FailSafe supports the failover of XFS filesystems on shared disks. Shared disks must be either Fibre Channel or SCSI JBOD or in RAID storage systems that are shared between nodes in the FailSafe cluster. Fibre Channel and SCSI JBOD storage systems must use XLV mirroring.

The following are special issues that you must be aware of when you are working with filesystems on shared disks in a cluster:

The name of a resource of the filesystem resource type is the mount point of the filesystem.

When clients are actively writing to a FailSafe NFS filesystem during failover of filesystems, data corruption can occur unless filesystems are exported with the mode wsync. This mode requires that local mounts of the XFS filesystems use the wsync mount mode as well. Using wsync affects performance considerably.

---

Caution: Do not cross-mount filesystems using NFS in a FailSafe cluster (that is, do not mount a locally mounted filesystem on a different node using NFS). This configuration is not reliable and will not work with FailSafe. Instead, you should use the CXFS (clustered XFS) plug-in, which provides this functionality. For more information, see IRIS FailSafe Version 2 NFS Administrator's Guide. Use of NFS over TCP is not recommended. If the client loses the TCP connection and does not reconnect, it can cause the client to hang on a failover. You should use UDP rather than TCP. Note that TCP may be the default for your NFS clients, requiring you to reconfigure them to use UDP. One method to accomplish this is to create the /etc/config/nfsd.options file with the content -p UDP, which will prevent the server from accepting TCP mount requests.

---

Continuing with the scenario from in “Example Logical Volume Configuration”, suppose you have the following XFS filesystems:

Table 2-2 lists a label and configuration parameters for each filesystem.

Figure 2-6 shows the following:

Figure 2-6. Filesystems and Logical Volumes

See “Create XLV Logical Volumes and XFS Filesystems” in Chapter 3, for information about creating XFS filesystems.

Use the following guidelines when planning interface configuration for the private control network between nodes:

Use the following guidelines when planning the interface configuration for one or more public networks:

Use the following procedure to determine whether re-MACing is required. It requires the use of three nodes: node1, node2, and node3. node1 and node2 can be nodes of a FailSafe cluster, but they need not be. They must be on the same subnet. node3 is a third node. If you must verify that a router accepts gratuitous ARP packets (which means that re-MACing is not required), node3 must be on the other side of the router from node1 and node2. For more information about re-MACing, see “Network Interfaces and IP Addresses” in Chapter 1.

Table 2-3 shows the FailSafe configuration parameters you could specify for these example HA IP addresses.

You can configure your system so that an HA IP address will fail over to a second interface within the same node, for example from ef0 to ef1. A configuration example that shows the steps you must follow for this configuration is provided in “Example: Local Failover of HA IP Address” in Chapter 6.

A coexecution cluster is supported with as many as 32 nodes. All of these nodes must run CXFS and up to eight can also run FailSafe. As many as 16 of the nodes can be CXFS administration nodes and all other nodes can be client-only nodes. FailSafe must be run on a CXFS administration node; FailSafe cannot run on a client-only node.

It is recommended that a production cluster can be configured with a minimum of three CXFS metadata server-capable administration nodes. (A cluster with serial hardware reset cables and only two server-capable nodes is supported, but there are inherent issues with this configuration; see the CXFS Version 2 Software Installation and Administration Guide.)

Even when you are running CXFS and FailSafe, there is still only one pool, one cluster, and one cluster configuration.

The cluster can be one of three types:

All potential metadata server nodes must be of one of the following types:

The metadata server list must exactly match the failover domain list (the names and the order of names).

FailSafe provides the CXFS resource type, which can be used to fail over applications that use CXFS filesystems. CXFS resources must be added to the resource group that contain the resources that depend on a CXFS filesystem. The name of the CXFS resource is the CXFS filesystem mount point.

The CXFS resource type has the following characteristics:

The CXFS filesystems that an NFS server exports should be mounted on all nodes in the failover domain using the CXFS GUI or the cmgr(1M) command.

For example, following are the commands used to create resources named NFS, CXFS and statd_unlimited based on a CXFS filesystem mounted on /FC/lun0_s6. (This example assumes that you have defined a cluster named test-cluster and that you have already created a failover policy named cxfs-fp and a resource group named cxfs-group based on this policy.)

cmgr> define resource /FC/lun0_s6 of resource_type CXFS in cluster test-cluster
Enter commands, when finished enter either "done" or "cancel"

Type specific attributes to create with set command:

Type Specific Attributes - 1: relocate-mds


No resource type dependencies to add

resource /FC/lun0_s6 ? set relocate-mds to false
resource /FC/lun0_s6 ? done


============================================

cmgr> define resource /FC/lun0_s6 of resource_type NFS in cluster test-cluster
Enter commands, when finished enter either "done" or "cancel"

Type specific attributes to create with set command:

Type Specific Attributes - 1: export-info
Type Specific Attributes - 2: filesystem


No resource type dependencies to add

resource /FC/lun0_s6 ? set export-info to rw
resource /FC/lun0_s6 ? set filesystem to /FC/lun0_s6
resource /FC/lun0_s6 ? done


============================================
cmgr> define resource /FC/lun0_s6/statmon of resource_type statd_unlimited in cluster test-cluster
Enter commands, when finished enter either "done" or "cancel"

Type specific attributes to create with set command:

Type Specific Attributes - 1: ExportPoint


Resource type dependencies to add:

Resource Dependency Type - 1: NFS


resource /FC/lun0_s6/statmon ? set ExportPoint to /FC/lun0_s6
resource /FC/lun0_s6/statmon ? add dependency /FC/lun0_s6 of type NFS
resource /FC/lun0_s6/statmon ? done

==============================================
cmgr> define resource_group cxfs-group in cluster test-cluster
Enter commands, when finished enter either "done" or "cancel"

resource_group cxfs-group ? set failover_policy to cxfs-fp
resource_group cxfs-group ? add resource /FC/lun0_s6 of resource_type NFS
resource_group cxfs-group ? add resource /FC/lun0_s6 of resource_type CXFS
resource_group cxfs-group ? add resource /FC/lun0_s6/statmon of resource_type statd_unlimited
resource_group cxfs-group ? done

There is one cmgr(1M) command but separate GUIs for CXFS and for FailSafe. You must manage CXFS configuration with the CXFS GUI and FailSafe configuration with the FailSafe GUI; you can manage both with cmgr.

Using the CXFS GUI or cmgr(1M), you can convert an existing FailSafe cluster and nodes to type CXFS or to type CXFS and FailSafe. You can perform a parallel action using the FailSafe GUI. A converted node can be used by FailSafe to provide application-level high-availability and by CXFS to provide clustered filesystems.

However:

For FailSafe, you must have at least two network interfaces. However, CXFS uses only one interface for both heartbeat and control messages.

When using FailSafe and CXFS on the same node, only the priority 1 network will be used for CXFS and it must be set to allow both heartbeat and control messages.

---

Note: CXFS will not fail over to the second network. If the priority 1 network fails, CXFS will fail but FailSafe services may move to the second network if the node is CXFS and FailSafe. If CXFS resets the node due to the loss of the priority 1 network, it will cause FailSafe to remove the node from the FailSafe membership; this in turn will cause resource groups to fail over to other FailSafe nodes in the cluster.

---

FailSafe provides the XVM resource type, which can be used to fail over applications that use local XVM volumes without CXFS. (Do not use the XVM resource type with the CXFS resource type.)

For each local XVM resource, the name of the resource is the name of the XVM volume without the preceding vol/ characters. The resource name must be unique for all XVM domains in the FailSafe cluster.

Table 2-4 provides the XVM resource attributes. There are no resource type dependencies for the XVM resource type.

Following is an example local XVM volume configuration:

cmgr> show resource lun1 of resource_type XVM in cluster lab-cluster

fs_xvm_owner: fake_owner
devname_group: sys
physvol_names: lun1,lun2,lun3
devname_mode: 600
devname_owner: root