************************************************************************ * Myricom GM networking software and documentation * * Copyright (c) 2002 by Myricom, Inc. * * All rights reserved. See the file `COPYING' for copyright notice. * ************************************************************************ README-solaris for gm-1.6.4 Supported platforms: Solaris 7 and higher in 32-bit and 64-bit modes on UltraSPARC. GM 1.6.4 software for Solaris on x86 is not supported. Supported interfaces: LANai7 (PCI64, PCI64A), and LANai9 (PCI64B, PCI64C) (If you have LANai4, you will need to upgrade your interface, or use a previous version of GM. For LANai4 with 1MB (PCI32, SBus), use GM of version gm-1.6_Solaris. For LANai4 with 256K or 512K, use GM of version gm-1.2.3. For installation instructions of an earlier GM version please refer to the respective README and README- files. Table of Contents: ----------------- I. GM Binary Installation Instructions a. Unpacking and installing the GM driver b. Running the GM Mapper c. Testing the GM Installation II. GM Source Installation Instructions a. Configuring and compiling GM b. Installing the GM driver c. Running the GM Mapper d. Testing the GM Installation III. Verifying GM performance IV. IP support via Ethernet Emulation V. Miscellaneous a. De-installation of the GM driver b. 64-bit Shared Library patch for Sun C++ ************************************************************************ If difficulties are encountered, please consult the FAQ http://www.myri.com/scs/GM_FAQ.html and all technical support questions should be directed to help@myri.com. ************************************************************************ ====================================== I. GM Binary Installation Instructions ====================================== These GM binaries are provided for 64-bit and 32-bit installations, and assume the use of LANai9 or LANai7 interfaces. If you have LANai4 interfaces, you will not be able to use these precompiled binaries and must compile GM from the source distribution. 1. Unpacking and installing the GM driver: ----------------------------------------- gunzip -c gm-1.6.4_Solaris64-sun4u-SunOS-5.7-8port-heart-beat-on.tar.gz | tar xvf - cd gm-1.6.4_Solaris64-sun4u-SunOS-5.7 Select an installation directory path . It is usually best for to be the path to an NFS directory available on all machines that are to share this GM installation. The directory must be accessible using on all machines that are to share the installation. must be an absolute path; it must start with "/". However, may contain symbolic links. cd binary ./GM_INSTALL You may omit to install the driver in /opt/gm/. Next, you must run su root /sbin/gm_install_drivers on each machine to install the drivers on that machine. If you wish for the driver to auto-load an boot, you must create appropriate links in the /etc/rcN directories to the /etc/init.d/gm and /etc/init.d/myri scripts. Alternatively, you may start and stop the drivers manually using su root /etc/init.d/gm start /etc/init.d/gm stop or su root /etc/init.d/gm restart to start, stop, or restart the driver, respectively. For directions on how to uninstall the GM driver, refer to the "Miscellaneous" section. 2. Running the GM Mapper. ------------------------ After installing and starting the driver on all the nodes in your cluster, you need to map your network to configure it and make it usable. To run the mapper, execute su root /sbin/mapper /etc/gm/map_once.args There are 3 ways in which the GM Mapper can be run. Refer to section 2 'Running the Mapper' of the README-linux for details. 3. Testing the GM Installation. ------------------------------ A variety of test scripts are available in binary/bin/ to test your GM installation. We recommend the five tests described in tests/README. ======================================= II. GM Source Installation Instructions ======================================= Building GM requires the Sun C++ compiler and Gnu make. Current cautions and common problems: * If the host is rebooted, you must rerun the GM Mapper. * The GM Mapper must be run before any communication over Myrinet can occur. * In most cases, the GM Mapper should not be run continuously. It is intrusive and can interfere with network traffic intensive jobs (e.g, MPI applications). GM installation is performed in four easy steps: 1. Configuring and compiling GM: -------------------------------- gunzip -c gm-1.6.4_Solaris.tar.gz | tar xvf - cd gm-1.6.4_Solaris autoconf autoheader ./configure Add the "--disable-64b" option if you are building a 32-bit driver on a machine running a 64-bit kernel, or "--enable-64b" if you are building a 64-bit driver on a machine running a 32-bit kernel. Without these options, a driver matching the running kernel on the compilation machine will be built. make depend make to build a GM binary distribution in the binary/ subdirectory. Note: By default, we assume that you have LANai9 or LANai7 interfaces. If you have LANai4 (with 1MB of memory), you will need to configure with: ./configure --disable-new-features We also assume that you have a PCI-based machine. If you have an SBus-based machine, you must configure with: ./configure --disable-pci (As previously noted, if you have LANai4 with 256K or 512K, you cannot install gm-1.6.4. You will need to upgrade your interface or use a previous version of GM (gm-1.2.3). If you have a mixture of hosts with LANai4 and LANai7 (or LANai9) interfaces that need to talk to each other, you must configure with --disable-new-features on all of the hosts. For a complete listing of all options to configure, type: ./configure --help Please be sure to read the README file in the GM distribution for further details of GM. 2. Installing the GM driver: --------------------------- Select an installation directory path . It is usually best for to be the path to an NFS directory available on all machines that are to share this GM installation. The directory must be accessible using on all machines that are to share the installation. must be an absolute path; it must start with "/". However, may contain symbolic links. cd binary ./GM_INSTALL You may omit to install the driver in /opt/gm/. Next, you must run su root /sbin/gm_install_drivers on each machine to install the drivers on that machine. If you wish for the driver to auto-load an boot, you must create appropriate links in the /etc/rcN directories to the /etc/init.d/gm and /etc/init.d/myri scripts. Alternatively, you may start and stop the drivers manually using su root /etc/init.d/gm start /etc/init.d/gm stop or su root /etc/init.d/gm restart to start, stop, or restart the driver, respectively. For directions on how to uninstall the GM driver, refer to the "Miscellaneous" section. 3. Running the GM Mapper. ------------------------ After installing and starting the driver on all the nodes in your cluster, you need to map your network to configure it and make it usable. To run the mapper, execute su root /sbin/mapper /etc/gm/map_once.args There are 3 ways in which the GM Mapper can be run. Refer to section 2 'Running the Mapper' of the README-linux for details. 4. Testing the GM Installation. ------------------------------ A variety of test scripts are available in binary/bin/ to test your GM installation. We recommend the five tests described in tests/README. ============================= III. Verifying GM Performance ============================= We recommend the following test to verify the GM performance. View the results of the hardware benchmark test of the PCI bus with the DMA engine of the Myrinet adapter. cd /bin ./gm_debug --no-counters Note: The output of this command gives the maximum sustained bandwidth that can be obtained from the PCI bus. Refer to the section entitled "GM Performance" in the README file in the GM distribution for complete details on expected GM performance. ===================================== IV. IP support via Ethernet Emulation ===================================== The IP device is accessed via /sbin/ifconfig myri0 plumb (installs the driver) /sbin/ifconfig myri0 up (configures the driver) where you must replace 'myri0' with the appropriate name (myri1, myri2, etc.) if you have more than one Myrinet interface per host. ================ V. Miscellaneous ================ ------------------------------------ a. De-installation of the GM driver ------------------------------------ The gm_install_drivers script generates the script /sbin/gm_uninstall_drivers, which can be used to uninstall the drivers. The GM_INSTALL script generates the script /sbin/GM_UNINSTALL, which can be used to uninstall GM. ------------------------------------------- b. 64-bit Shared Library patch for Sun C++ ------------------------------------------- Solaris 7 may have a missing symbol (_ex_register64) in C++ compiler's library /usr/lib/sparcv9/libCrun.so.1, which causes the mapper to fail. Patch the machine with patch ID# 106300-14 at .