MVAPICH is installed as part of the OFED 1.2 software. MVAPICH is open source software developed largely by the Network-Based Computing Laboratory (NBCL) at Ohio State University. MVAPICH develops the Message Passing Interface (MPI) style of process-to-process communications for computing systems employing Infiniband and other Remote Direct Memory Access (RDMA) interconnects.

For more descriptions including the MVAPICH User Guide and other MVAPICH publications, see http://mvapich.cse.ohio-state.edu.

MVAPICH applications use the Infiniband network of SGI Altix ICE 8200 systems for interprocess RDMA communications. SGI Altix ICE 8200 systems are configured with two Infiniband fabrics, designated as ib0 and ib1. In order to maximize performance, SGI advises that the ib0 fabric be used for all MPI traffic, including MVAPICH MPI. The ib1 fabric is reserved for storage related traffic. The default configuration for MVAPICH MPI is to use only the ib0 fabric.

A simple test to insure installation of MVAPICH is to enter the command:

% rpm -qa | grep mvapich
mvapich_gcc

MVAPICH executables, libraries, and man pages can be found in the following directory:

/usr/mpi/gcc/mvapich-0.9.9

Build options for MVAPICH are those assembled in the OFED1.2 distribution and include single-rail ch_gen2 (ibverbs) with ROMIO.

Use of MVAPICH is a multiple step effort in that you must first build your MPI-based application using MVAPICH libraries and then execute your application with the MVAPICH-supplied launch tool, mpirun_rshl. A large number of tuning options and launch tool options are available. See the MVAPICH man pages, the MVAPICH User Guide, and the MVAPICH web site for information on these options.

It is strongly recommended that MVAPICH applications be built with an MVAPICH-supplied compiler script: mpicc, mpiCC, mpicxx, mpif77, or mpif90. For example, to build an MVAPICH application perform the following:

% /usr/mpi/gcc/mvapich-0.9.9/bin/mpicc -o myapp mmpi.c

The compiler scripts provide links to all necessary libraries and include files. Applications can be launched using the mpirun_rsh command, which is available in the same MVAPICH executables directory. There are a number of options available for launching distributed applications across a cluster. For more information on the use of this tool, enter the following command:

% /usr/mpi/gcc/mvapich-0.9.9/bin/mpirun_rsh --help

A very simplified example of its use is, as follows:

% /usr/mpi/gcc/mvapich-0.9.9/bin/mpirun_rsh -rsh -np 4 sv0 sv1 sv2 sv3 myapp

where you have requested the creation of an MPI process named myapp on each of the four hosts sv0, sv1, sv2, and sv3 using rsh.