The M3-CLOS-ENCL-B and M3-SPINE-ENCL-B are identical except for their backplanes, and hence are described in this single specification.

These "B" versions of the 14U enclosures for Myrinet-2000 switches for large clusters are functionally identical to the original versions, and accommodate the same line cards. However, the "B" enclosures have the network ports and power connections on the back rather than on the front in order to simplify cabling to servers that similarly have front-to-back airflow and ports and power connections on the back. In addition, the "B" enclosures employ more efficient and quieter power modules, and are RoHS (Reduction of Hazardous Substances) compliant under the server exemption.

The photographs below are of an M3-CLOS-ENCL-B with line cards in the Clos256 configuration (see below).

Front: inlet air side Fan units, product and regulatory labels

Back: exhaust air side Ports, status display, power modules

The enclosures provide power and cooling for 21 vertical line cards in the card cage at the back of the enclosure. The line cards may be of the following types:

Each enclosure includes the following Field-Replaceable Units (FRUs), which may be ordered separately for replacements.

• M3-MONITOR - monitoring line card (quantity 1)
• M3-POWER-B - 840W power module (quantity 2)
• M3-FAN-B - Fan assembly (quantity 2)
• M3-RACK-KIT - Rack-mount kit (quantity 1)

The left-most line-card slot is reserved for the M3-MONITOR card. The enclosures also include a color TFT display for status information. The TFT display is driven from the monitoring line card, which includes a turn/push knob to control the display mode and certain internal settings.

Up to four M3-POWER-B auto-parallel power modules may be installed. One power module is sufficient to power any of the common configurations listed below, but two power modules are provided for redundancy and power derating. The IEC inlets for the M3-POWER-B power modules are on the power modules themselves. A separate power cord is provided for each power module. For installations with dual-redundant AC power, plug the power cords into separate AC sources. Of course, you must disconnect all power cords to completely remove power from the system.

Backplane of the M3-CLOS-ENCL-B

For the M3-CLOS-ENCL-B, the line-card slots are denoted as follows left to right, and accommodate the following types of line cards:

The M, S, and Q slot types are readily distinguished in the photograph above. A line card of a given type will fit only in the type of slot allowed, but the M3-AIRDAM card will fit in any slot. Both for airflow and for EMI shielding, any otherwise unused line-card slots must include M3-AIRDAM line cards.

The M3-CLOS-ENCL-B backplane connects the 16 Myrinet ports from each of the 16 "Single" line cards to a set of ports on the 4 central "Quad" line cards to form a Clos network. The backplane also includes the connections for a low-speed network from each line card to/from the monitoring line card.

Backplane of the M3-SPINE-ENCL-B

For the M3-SPINE-ENCL-B, the line-card slots are denoted as follows left to right, and accommodate the following types of line cards

The M3-SPINE-ENCL backplane is very simple. The backplane Myrinet-serial ports of adjacent pairs of slots, (0, 1), (2, 3), ..., (18, 19), are connected. The only allowed use is with M3-4SW32-16Q line cards in the even-numbered slots and M3-THRU-16Q line cards in the odd-numbered slots. The pairing of M3-4SW32-16Q and M3-THRU-16Q line cards brings out all 128 ports of the 4 32-port switches to the two front panels. These quad 32-port switches are used as the spine switches of a diameter-5 Clos network.

Common configurations

• Clos256: 256 host ports, no inter-switch ports. The line cards in an M3-CLOS-ENCL-B, left to right, are 1 M3-MONITOR, 8 M3-SW32-16F, 4 M3-2SW32, 8 M3-SW32-16F. The list price of the components for this configuration is $80,000 ($312.50 per host port). For fewer than 256 host ports, substitute M3-AIRDAM blank line cards for M3-SW32-16F line cards.
• Clos256+256: 256 host ports, 256 interswitch ports on 64 quad-fiber ports. The line cards in an M3-CLOS-ENCL-B, left to right, are 1 M3-MONITOR, 8 M3-SW32-16F, 4 M3-4SW32-16Q, 8 M3-SW32-16F. The list price of the components for this configuration is $128,000.
• Spine768: 6 quad (24) 32-port switches presented on 192 quad-fiber ports (768 ports total). This configuration may be used as the spine for 3 Clos256+256 units. The line cards in an M3-SPINE-ENCL-B, left to right, are 1 M3-MONITOR, 6 repetitions of 1 M3-4SW32-16Q and 1 M3-THRU-16Q, and 8 M3-AIRDAM. The list price of the components for this configuration is $157,600.
• Spine1024: 8 quad (32) 32-port switches presented on 256 quad-fiber ports (1024 ports total). This configuration may be used as the spine for 4 Clos256+256 units. The line cards in an M3-SPINE-ENCL-B, left to right, are 1 M3-MONITOR, 8 repetitions of 1 M3-4SW32-16Q and 1 M3-THRU-16Q, and 4 M3-AIRDAM. The list price of the components for this configuration is $205,200.
• Spine1280: 10 quad (40) 32-port switches presented on 320 quad-fiber ports (1280 ports total). This configuration may be used as the spine for 5 Clos256+256 units. The line cards in an M3-SPINE-ENCL-B, left to right, are 1 M3-MONITOR, and 10 repetitions of 1 M3-4SW32-16Q and 1 M3-THRU-16Q. The list price of the components for this configuration is $252,800.

Configurations for more than 256 hosts up to 512 hosts are a special case. Contact sales@myri.com for efficient implementations.

Specifications

Physical Dimensions:

• Rack mounting conforms to the EIA-310 standard (19-inch rack mount). However, different mounting-hole patterns on the rack-mount ears can be supplied on special order.
• The nominal width of the enclosures is 17.25 inches (438.2mm), and the width to the edges of the rack-mount ears is 19 inches (482.6mm).
• The nominal depth of the enclosures is 18 inches (457 mm).
• The height of the enclosures is 24.34 inches (618mm). This height is 0.16 inches (4mm) less than the nominal 24.500 inches (622.3mm) for a 14U rack-mount unit in order to allow for the 0.13 inch (3.3mm) height of the rack-mount shelf bracket (M3-RACK-KIT), plus a small tolerance.

Weight and Mounting:

• Enclosures are shipped completely assembled and tested, including power supplies, fans, and the monitoring line card, but without other line cards, which ship separately. The weight of the unit in this shipping configuration is approximately 55 pounds (25 kg).
• The weight with line cards in all slots can be as high as approximately 101 pounds (46 kg).
• Due to the weight of the unit, it is recommended that the unit be installed in a rack while it is in the shipping configuration (55 pounds, 25 kg), before installing the line cards.
• Use of the M3-RACK-KIT rack-mount shelf backet is highly recommended, both to assist during installation and to support the enclosure in the rack against shock and vibration that may be encountered when shipping racks with switches installed.
• The rack-mount "ears" are normally in the front, but can be relocated to the back.

Environmental:

• Operating: Temperature 15C to 40C at the air inlets. Relative humidity 15% to 80% @ 40C, non-condensing.
• Storage: Temperature -40C to 70C. Relative humidity 90% @ 65C.

Thermal and Cooling:

• Air flow is front-to-back, with separate air paths for the power-module and card-cage cooling. Each of the power modules has a pair of high-speed 1.5-inch (38mm) fans. The card cage cooling is accomplished by two M3-FAN-B assemblies, each of which has a series pair of 4.5-inch (114mm) fans. Fan rotation speed for the card-cage cooling is monitored, and can be read along with the exhaust air temperature through the M3-MONITOR monitoring card. The small microcontroller card for each M3-FAN-B unit also controls the fan speed according to the exhaust-air temperatures. The M3-FAN-B units operate at maximum speed at exhaust-air temperatures of 40C and higher.
• Internal temperatures are measured both upstream and downstream in the air flow on each line card. The temperatures can be monitored through the M3-MONITOR monitoring card. The line card will shut down autonomously on an over-temperature condition (55C circuit-board temperature).

Power:

• Two power modules are standard, but the enclosure accommodates up to four for additional redundancy. There is an IEC power inlet on each power module. All power modules are supplied with an IEC power card appropriate to the shipping destination. Note: The 100-127V~ IEC power cord for the M3-POWER-B is rated for 15A, whereas all of the other IEC power cords are rated for 10A.
• Power input: 100-127V~ / 200-240 V~, 50 Hz / 60 Hz.
• The M3-POWER-B power modules are rated at 840W each (12VDC @ 70A each), and auto-parallel. If one power module fails, the other can supply sufficient power. The power modules can be individually hot-swapped.
• The power factor of the power modules at full (840W) load is typically 0.98, but at typical load is ~0.90.
• Typical total power with line cards in all slots is approximately 700W.
• Maximum DC power (exclusive of power-supply inefficiency) is 840W.
• Maximum total power including power-supply inefficiency is 960W.
• See the specifications for the M3-POWER-B power modules for the maximum input power and current.

Safety Approvals:

The M3-CLOS-ENCL-B and M3-SPINE-ENCL-B are certified by TUV of North America, Inc., in accordance with the IEC System for Conformity Testing and Certification of Electrical Equipment (IECEE), CB Scheme. In addition, TUV certified these products to the US and Canadian safety standards. The physical enclosures are the same except for the backplane and set of line cards supported as the 10G-CLOS-ENCL, and thus are the same from the standpoint of safety approvals. You can download the full TUV test report and this addendum to the CB certificate.

EMC Approvals:

All configurations described above are within Class A limits for emission of and susceptibility to electromagnetic interference (EMI). You can download the full EMC test report.

Last updated: 5 January 2007