Performance of MPICH-GM 1.2.5..10
SMP case (two processes per node)
MPICH is a portable implementation of MPI, developed by Argonne National Laboratory. It is designed to be highly portable and is currently used by a large number of providers of MPI implementations. MPICH-GM is a "port" of MPICH on top of GM (ch_gm) developed and supported by Myricom.
The current MPICH-GM software release is MPICH-GM 1.2.5..10, and is based on MPICH 1.2.5. For download information, please refer to the Myricom Software and Customer Support Page.
Performance data is presented for the Pallas Benchmark Suite, available from the Pallas MPI Benchmarks Version 2.2.
Each benchmark is run with varying message lengths, and timings are averaged over multiple samples. Details of each test can be found in Section 4.2.1 of the Pallas MPI Benchmarks -- documentation MPI-1 part. The raw performance data plotted in the graphs is available, as well as the MPICH-GM host file used for the tests.
Important Note: For these performance graphs, two processes per node was used, so intranode communication was using shared memory by default.
The test environment consists of 16 dual P4 2.4GHz machines with a SerwerWorks Grand-Champion GC-LE chipset, a M3F-PCIXD-2 NIC on a 133MHz 64bit PCI-X bus, and connected by a Myrinet-2000 switch. Each machine has 1GB of memory and is running Linux 2.4.21, GM version 2.0.5 and MPICH-GM 1.2.5..10. The Pallas MPI Benchmark is compiled with gcc 3.2.3 with -O2.
Point to point communication performance is measured between two processes. Latency is measured in micro-second (us) and bandwidth is measured in MB/s. The latency scale is logarithmic and the bandwidth scale is linear.
PingPong is the classical pattern used for measuring startup (latency) and throughput (bandwidth) of a single message sent between two processes.
As PingPong, PingPing measures the startup and throughput of a single message sent between two processes, with the crucial difference that messages are obstructed by oncoming messages. For this, two processes communicate (MPI_Isend/MPI_Recv/MPI_Wait) with each other, with the MPI_Isend's issued simultaneously.
Based on MPI_Sendrecv(), the processes form a periodic communication chain. Each process sends to the right and receives from the left neighbor in the chain. The turnover count is 2 messages sample (1 in, 1 out) for each process.
For 2 processes, Sendrecv will report the bi-directional bandwidth of the system, as obtained by the (optimized) MPI_Sendrecv function.
Results are presented for 2, 4, 8 and 16 processes.
Exchange is a communications pattern that often occurs in grid splitting algorithms (boundary exchanges). The group of processes is seen as a periodic chain, and each process exchanges data with both left and right neighbor in the chain.
The turnover count is 4 messages per sample (2 in, 2 out) for each process.
Results are presented for 2, 4, 8 and 16 processes.
Collective communication performance is measured between all or a subset of the nodes in the system.
Benchmark of the MPI_Reduce() function. Reduces a vector of length L = X/sizeof(float) float items. The MPI datatype is MPI_FLOAT; the MPI operation is MPI_SUM. The root of the operation is changed cyclically.
Results are presented for 2, 4, 8 and 16 processes.
Benchmark of the MPI_Reduce_scatter() function. Reduces a vector of length L = X/sizeof(float) float items. The MPI datatype is MPI_FLOAT; the MPI operation is MPI_SUM. In the scatter phase, the L items are split as evenly as possible between all processes. Exactly, when
np = #processes, L = r*np+s (s = L mod np),
then the process with rank i gets r+1 items when i < s, and r items when i > = s.
Results are presented for 2, 4, 8 and 16 processes.
Benchmark of the MPI_Allreduce() function. Reduces a vector of length L = X/sizeof(float) float items. The MPI datatype is MPI_FLOAT; the MPI operation is MPI_SUM.
Results are presented for 2, 4, 8 and 16 processes.
Benchmark of the MPI_Allgather() function. Every process sends X bytes and receives the gathered X*(#processes) bytes.
Results are presented for 2, 4, 8 and 16 processes.
Functionally the same as Allgather, however with the MPI_Allgatherv() function. Shows whether MPI produces overhead due to the more complicated situation as compared to MPI_Allgather().
Results are presented for 2, 4, 8 and 16 processes.
Benchmark of the MPI_Alltoall() function. Every process inputs X*(#processes) bytes (X for each process) and receives X*(#processes) bytes (X for each process).
Results are presented for 2, 4, 8 and 16 processes.
Benchmark of MPI_Bcast. A root process broadcasts X bytes to all. The root of the operation is changed cyclically.
Results are presented for 2, 4, 8 and 16 processes.
This is a benchmark of the MPI_Barrier() function.
Last updated: 02 June 2006