64 LAN host ports
M2LM-Clos64 Clos Myrinet "Network in a Box" of 16-port switches |
This product packages 16, 16-port, Myrinet switches in a
Clos network:
64 SAN inter-switch ports
64 LAN host ports
This network topology provides full (maximal) bisection between the 64 host ports, and preserves 64 links of data rate between the host ports and the deeper part of a Clos-network fabric. One Clos64 component can be used to support 64 hosts with full bisection. Two Clos64 components can be connected "back-to-back" by 32 SAN cables (64 links) to support up to 128 hosts with full bisection. Larger numbers of Clos64 components can be connected through additional switches to provide full-bisection networks of up to thousands of hosts, or a large variety of networks with nearly full bisections. For applications exceeding 128 hosts, please contact info@myri.com for assistance in developing a network topology suitable for your applications.
A microcontroller with an Ethernet connection (the left-most "line card" in the photo) provides complete visibility and monitoring of the internal parts of the M2LM-Clos64. The SNMP-based monitoring and control through a 10-Base-T Ethernet port includes counts of CRC-correct and CRC-incorrect packets on each switch port, the state of each port, and any error conditions detected. The control includes enabling/disabling any port. A large amount of 'vital stats' information is also available through the Ethernet port, including internal voltages and temperature.
This product takes advantage of several "economies of scale" to make high- and full-bisection networks practical for clusters with large numbers of hosts. A Myrinet 16-port switch, complete with all protocol support, is a single chip, the "XBar16." The 16 XBar16 chips in one enclosure share a single power supply and a single microcontroller. The internal links, which with separate switch components would be run on cables with connectors, are instead run more economically on a large circuit board. The LAN ports to the hosts allow the network to be centralized, such that inter-switch connections can be made two-per-cable with Myrinet-SAN cables.
Myrinet-LAN ports (64): 1.28+1.28 Gb/s or 0.64+0.64 Gb/s data
rates selected automatically by the LAN
cable type.
Myrinet-SAN ports (64): 1.28+1.28 Gb/s on the A and B links of the
32 SAN connectors on the rear of the enclosure.
High-Availability features: Myrinet heartbeat and illegal-control-symbol detection selectable by port, with out-of-band monitoring and control via the Ethernet switch-management port.
Ethernet switch-management port: Standard RJ-45 10-Base-T Ethernet. The Ethernet MAC address is indicated on a label below the RJ-45 connector. An internal microcontroller monitors the XBar16 scan paths and LAN-SAN-conversion circuits, and communicates externally using SNMP. Click here for the SNMP MIBS and other documentation on the switch-monitoring port.
Cut-through latency: ~100ns for each XBar16, plus ~100ns for each
LAN-SAN conversion. Worst-case latency for a 64-host network: ~500ns. Worst-case
latency for a 1024-host full-bisection Clos network: ~700ns.
Physical characteristics: Rack mount, 12U high, 7 inches deep, ~20
Kg. Maximum-operating ambient temperature 40C. Forced-air cooling, with
air entry and exit on the back of the enclosure. The enclosure material
is stainless steel, unpainted.
Approvals: For in-cabinet applications only, due to the EMI generated
from the SAN inter-switch ports and cables at the rear of the enclosure.
Power: Power input 85-265VAC, 47-63Hz, single-phase, ~160W maximum,
with a single IEC plug and standard 3-wire power cord.
Last significant revision: 26 June 2000