Supererogatory Penguin Image
Supererogatory Penguin Image		   Myrinet GM-1 Software
for Linux

GM Software Available
Linux
Kernel		 Processor
IA32 IA64 Alpha PowerPC
2.4 check mark check mark check mark check mark
2.2 check mark   check mark check mark
2.0 check mark   check mark  

Note: PowerPC64 and AMD64 are not supported in GM-1. If you wish to use these processors, you must use GM-2.

To download Myrinet software, you need a login and password.
If you don't already have these, please request them.

The Linux GM software is distributed only in source form. Which GM version you use may depend on which Myrinet NICs are used in your hosts.

Recommended-version GM Source Downloads for Linux
NIC
Operating System/
Processor		 PCI64B and PCI64C
(LANai 9)		 PCI64 and PCI64A
(LANai7)		 PCI32C
(1MB)
(LANai4)		 PCI32B
(512KB)
(LANai4)		 PCI32A
(256KB)
(LANai 4)
Linux 2.4
IA64		 GM-1.6.4 GM-1.6.4 - - -
Linux 2.4
IA32, Alpha, PowerPC		 GM-1.6.4 GM-1.6.4 GM-1.5.2.1 GM-1.5.2.1 -
Linux 2.2
IA32, Alpha,
PowerPC		 GM-1.6.4 GM-1.6.4 GM-1.5.2.1 GM-1.5.2.1 GM-1.2.3
Linux 2.0
IA32, Alpha		 GM-1.4 GM-1.4 GM-1.4 GM-1.4 GM-1.2.3

Linux 2.4 requires GM 1.5 or higher. Certain of the very early PCI32 NICs cannot run GM 1.5.x for lack of sufficient memory, thus, cannot be used with Linux 2.4.

Important note: gm-1.5.2 or later is required for PCI64C NICs running on machines with 33MHz PCI buses!

VERY IMPORTANT NOTE: The installation procedure for GM has changed significantly between gm-1.5.2.1 and gm-1.6.3 and later. Refer to the abbreviated installation instructions below. Automated installation scripts will need to be updated!

Abbreviated Installation Instructions

To compile GM on Linux platforms, you will only need GNU make and C/C++ compilers. Please be sure to read the README files for detailed installation instructions and architecture-specific troubleshooting. Unresolved questions may also be addressed in the FAQ.

These abbreviated instructions presume that you are installing GM 1.6.4. (Here is the README-linux for GM 1.6.4.)

Abbreviated instructions for installing GM 1.5.2.1 are still available. (Here is the README-linux for GM 1.5.2.1.)

Current cautions and common problems:

  • GM is not in the critical performance path so it does not need to be built with specialized compilers and flags. GM should be built with gcc and only built with -O level of optimization.
  • When building/linking GM applications, you must do so on a Linux box that matches the OS version of the machine on which you will be running. For instance you cannot compile on a glibc-2.2 based OS and run the executable on a glibc-2.1 based one.
  • GM_INSTALL fails? Refer to this FAQ entry for troubleshooting.
  • AGP (nVidia and ATI) conflicts? Refer to the "Caveats" section of the README-linux.
  • APIC IRQ conflicts? Refer to the "Caveats" section of the README-linux.
  • If you're using SuSE 8 with highmem/highpte support, you will need gm-1.6.4 or later.
  • If you are using an Apple Xserve mother board, please read this FAQ entry for important performance information.
  • To install GM on RedHat Linux Advanced Server (also known as Pensacola Gold Master), you will need gm-1.6.3 or later.
  • If you have PCI64C NICs running on machines with 33MHz PCI buses, you must use gm-1.5.2 or later.
  • If you're installing GM on an Itanium IA64 McKinley (Tiger motherboard), you will need gm-1.5.2 or later.
  • If you're installing GM on a PAE-enabled x86 Linux kernel with more than 4GB of physical memory, you will need gm-1.5.2 or later.
  • PCI32 NICs are NOT supported in gm-1.6.3 or later.
  • PCI32 NICs are NOT supported on IA64.
  • For Alphas, if you have 2GB or more of memory, we recommend kernel version 2.4.18 or later. You must use kernel version 2.4.14 or later (2.4.9 also works).
  • By default, GM-1.5 and higher has security features enabled. Thus, you must now be root to run the GM Mapper (or gm_simpleroute).
  • Do not use the configure flag --enable-directcopy with GM 1.6.4. This flag is not a valid option to GM 1.6.4. It will be re-enabled in a future release.
  • If the host is rebooted, you must reload the GM driver and rerun the GM Mapper.
  • The GM Mapper must be run before any communication over Myrinet can occur.
  • In most cases, the GM Mapper should not be run continuously. It is intrusive and can interfere with network traffic intensive jobs (e.g, MPI applications).

    • GM installation is performed in four easy steps:

    1. Configuring and compiling GM.

            gunzip -c gm-1.6.4_Linux_and_AIX.tar.gz | tar xvf -
        cd gm-1.6.4_Linux_and_AIX
        ./configure
        make

    By default, we assume that the kernel header files for your Linux installation are located in /usr/src/linux. If your Linux installation is not located in /usr/src/linux, you must configure with the following option:

     
        ./configure --with-linux=<linux-source-dir>

    where <linux-source-dir> specifies the directory for the Linux kernel source. The kernel header files MUST match the running kernel exactly: not only should they both be from the same version, but they should also contain the same kernel configuration options.

    2. Installing the GM driver.

    Select an installation directory path <install_path>. It is usually best for <install_path> to be the path to an NFS directory available on all machines that are to share this GM installation. The directory must be accessible using <install_path> on all machines that are to share the installation. <install_path> must be an absolute path; it must start with /. However, <install_path> may contain symbolic links. 

            cd binary
        ./GM_INSTALL <install_path>

    You may omit <install_path> to install the driver in /opt/gm/.

    Next, you must run 

            su root
        <install_path>/sbin/gm_install_drivers
        /etc/init.d/gm start

    on each machine to install/copy the drivers on that machine.

    If you wish for the driver to auto-load at boot, you must create appropriate links in the /etc/rcN directories to the /etc/init.d/gm and /etc/init.d/myri scripts. Alternatively, you may start and stop the drivers manually using 

     
        su root
        /etc/init.d/gm start
        /etc/init.d/gm stop

    or 

     
        su root
        /etc/init.d/gm restart

    to start, stop, or restart the driver, respectively.

    For directions on how to uninstall the GM driver, refer to the "Miscellaneous" section of the README-linux.

    Note: If the host is rebooted, you must reload the GM driver (and rerun the GM mapper).

    3. Running the GM Mapper.

            cd  <install_path>/sbin/
        su root
        ./mapper ../etc/gm/map_once.args

    There are 3 ways in which the GM Mapper can be run. Refer to the README-linux for details.

    4. Testing the GM Installation.

    A variety of test scripts are available in bin/ to test your GM installation. We recommend the five tests described in bin/README.

    Congratulations! You have successfully installed GM.

    Refer to the Software and Documentation Page for details of GM-based software (MPICH-GM, VI-GM, PVM-GM, Sockets-GM). If you encountered difficulties during any phase of this installation process, we recommend that you first consult the README-linux and FAQ, and if you still have unresolved questions, please feel free to contact technical support at help@myri.com.


    Last updated: 20 May 2006