at ISC'07

Dresden, Germany 26-29 June 2007

A Remarkable New Series of Myri-10G Switches
Up to 512 ports from a single enclosure, and scalable to thousands of host ports
Multi-protocol Switching, 10-Gigabit Ethernet and 10-Gigabit Myrinet

Myricom's principal announcement at ISC'07 is a new series of economical, low-latency, modular, Myri-10G switches. These switches, based on a Myri-10G 32-port Myrinet-protocol crossbar-switch chip, are remarkable in several ways.

High port count. High port density. The high degree of the XBar32 chip allows full-bisection diameter-3 Clos networks up to 512 host ports. For economy and to minimize cabling, any cluster up to 512 host ports requires only a single enclosure. The 512-port switch network pictured to the right is 21U high (half of a standard 42U rack). A similar 12U enclosure accepts any of the line cards in this series for switches up to 256 host ports, a 7U enclosure for switches up to 128 host ports, and a 2U enclosure for switches up to 32 ports. In order to allow for enclosures with different capacities, the line cards are horizontal.

The standard internal network topology is a full-bisection Clos network. In the switch network pictured to the right, the 16 XBar32 switches that form the spine of the Clos network are on the 8 central line cards, and the 32 leaf XBar32 switches are on the line cards with external ports. All of the XBar32-to-XBar32 links of the Clos network are carried by the enclosure backplane.

Low cost. In a configuration with 512 10GBase-CX4 Myrinet-protocol ports, this complete switch has a list price of only $204,800 ($400 per host port). Fully populated configurations of the smaller switches and switch networks in this series are also available at a list price of $400 per 10GBase-CX4 Myrinet-protocol host port.

Multi-protocol switching. The line cards with external ports have 16 front-panel ports and 16 (XAUI) ports to the internal backplane. Different line cards are available for any of the standard Myri-10G 10-Gigabit Ethernet PHYs. The internal switching employs highly scalable, low-latency, Myrinet-protocol switching, but line cards can have both 10-Gigabit Ethernet and 10-Gigabit Myrinet external ports. This Myrinet-Ethernet interoperability was first demonstrated at SC06 with the original XBar16-based series of Myri-10G switches, and is used in many Myri-10G sites today, such as the DAS-3 grid of clusters in the Netherlands.

The 10-Gigabit Ethernet capacity is greatly extended in this new series of XBar32-based switches. Myricom is supplying line cards initially with 8 external 10-Gigabit Ethernet ports (10GBase-CX4 or 10GBase-R) along with 8 external 10-Gigabit Myrinet ports. Myricom will later introduce line cards with 16 external 10-Gigabit Ethernet ports.

In addition to grids, another important application that has emerged for large-scale multi-protocol switching is high-performance cluster file systems with MX drivers connected to 10-Gigabit Ethernet clients.

Low latency. Low power. The Myri-10G XBar32 chip has a cut-through latency of only 0.11µs, and consumes only 11 Watts. The Myri-10G XBar32 chip is also ideal for blade switches, currently in development.

Scaling up to 8192 host ports. Full-bisection Clos networks with more than 512 host ports up to 8192 host ports require a network diameter of 5. These networks are readily implemented using multiple enclosures with switch-to-switch cabling between the Clos-network enclosures and Edge-switch enclosures, and either 10-Gigabit Myrinet or 10-Gigabit Ethernet host ports.

Enterprise features. All of the switches in this series come standard with advanced monitoring on multiple 10/100/1000Base-T Ethernet ports. Cooling is front-to-back with multiple redundant fan modules. The power modules for the 21U, 12U, and 7U enclosures have separate IEC inlets for connection to redundant AC power, and operate in an auto-parallel mode for redundancy. Line cards, fan modules, and power modules can all be hot-swapped.

These new products will be added soon to the Myri-10G product list with links to descriptions and specifications. First customer shipments are scheduled for July.

26-29 June 2007