<01> Wed Jun 10 13:45:03 1998 F I 3 2 - 1.0 ============= This note defines the FI32-1.0 chip, the Myricom FIFO 32-bit interface, and provides information needed by users who wish to use it to connect to the Myricom SAN-links without using a LANai. A draft specification of the Data-Link layer of Myrinet (Level-2 of the ISO Reference Model for Open System Interconnection) is at http://www.myri.com:80/drafts/link-l2.html The specification of the Myrinet SAN link will be added, soon. The FI32 interfaces directly with Myrinet using the SAN-protocol on a byte-basis. Its host side interface operates on a 4-Byte-word-basis. The word interface is considered to be Big-Endian. The FI32 first communicates the most significant byte (Byte0, bits 24-31), and communicates the least significant byte (Byte3, bits 0-7) last. There is no synchronization between the sending and the receiving operations. For all practical purposes it could be two independent chips. All the transfers between the host and the FI32 are word-based, even when there are less than 4 bytes to communicate (typically at the beginning of SND, or at the end of RCV). In these cases these bytes are padded into full words that are presented with indications about the number of the added padding bytes. CONFIGURATION SIGNALS --------------------- Signals with a minus-sign at the end of their name (like "RST-") are active (i.e., do what their name suggest) when LOW. RST- chip reset - This input is synchronized to both RCLK and SCLK. The 3ns setup time and 0ns hold time (relative to either clock) only guarantee that a transition will be recognized during a specific clock cycle. These 3 configuration signals may be changed only while RESET (RST- = 0). CRC CRC-8 enable pin - When this signal is asserted the last byte of every message sent into the Myrinet network or received from the network is xor-ed with the CRC-8 of that message (not including the last byte); - For normal operation this signal should be asserted. WIN0,WIN1 SAN-protocol sampling-window-width selection - For normal operation these 2 signals should be HIGH. The configuration signals may be changed only at RESET (when RST- = 0). <02> THE CONTROL SIGNALS BETWEEN THE FI32 AND THE SYSTEM --------------------------------------------------- Synchronous send channel ------------------------ SCLK send clock - The output signals (SRDY,SAF-) change upon up-going SCLK edges, with the delay of 1ns min, 8ns max (0-30pF load). - The input signals (S00-S31,SP0,SP1,ST,SEN) are latched on up-going SCLK edges, with setup time of 3ns, 0ns hold time. SEN send enable - The request to advance the FIFO (it has no effect unless SRDY is asserted). The asserted SEN signal and its corresponding output data are latched on the same up-going SCLK edge. SRDY send ready - When this signal is asserted, the send channel can accept at least one word (a partial word counts as one regardless of the padding bits). SAF- send almost full - High: The FI can accept at least 4 words (16 bytes) for the SAN (a partial word counts as one regardless of the padding bits). - Low: The FI can accept less than 4 words for the SAN. S00-S31: 32 data bits: from S00 (the least significant bit) to S31 (the most significant bit) SP0 padding count (least significant bit) SP1 padding count (most significant bit) ST tail - interpretation of the tail and padding bits +------+-------+-------+-------+-------+-------+-------+ | | | |MSByte LSByte| | ST | SP1 | SP0 |S31-S24|S23-S16|S15-S08|S07-S00| +------+-------+-------+-------+-------+-------+-------+ | 0 | 0 | 0 | byte 0| byte 1| byte 2| byte 3| | 0 | 0 | 1 | (-) | byte 0| byte 1| byte 2| | 0 | 1 | 0 | (-) | (-) | byte 0| byte 1| | 0 | 1 | 1 | (-) | (-) | (-) | byte 0| | 1 | 0 | 0 | byte 0| byte 1| byte 2| tail 3| | 1 | 0 | 1 | byte 0| byte 1| tail 2| (-) | | 1 | 1 | 0 | byte 0| tail 1| (-) | (-) | | 1 | 1 | 1 | tail 0| (-) | (-) | (-) | +------+-------+-------+-------+-------+-------+-------+ (-) this byte will be ignored. It may have any value. <03> Synchronous receive channel --------------------------- RCLK receive clock - The output signals (R00-R31,RP0,RP1,RT,RRDY,RAE-,RBLCK) change upon up-going RCLK edges, with the delay of 1ns min, 8ns max (0-30pF load). - The input signal (REN) is latched on up-going RCLK edges, with setup time of 3ns, 0ns hold time. REN receive enable - The request to advance the FIFO (it has no effect unless RRDY is asserted) RRDY receive ready - When this signal is asserted there is at least one word in the receive channel, and that word is driven to the output data pins upon the same up-going RCLK edge (a partial word counts as one regardless of the padding bits). RAE- receive almost empty - High: The FI has at least 4 words (16 Bytes) from the SAN (a partial word counts as one regardless of the padding bits). - Low: The FI has less than 4 words from the SAN. RBLCK receive blocked - If the synchronous receiver blocks an incoming packet longer than 1/16 of a second, the RBLCK signal is asserted for one clock cycle. Any messages arriving from the network may be dropped until the receiver starts accepting packets again. R00-R31: 32 read data bits, from R00 (the least significant bit) to R031 (the most significant bit) <04> RT tail RP0 padding count (least significant bit) RP1 padding count (most significant bit) - interpretation of the tail and padding bits +------+-------+-------+-------+-------+-------+-------+ | RT | RP1 | RP0 |R31-R24|R23-R16|R15-R08|R07-R00| +------+-------+-------+-------+-------+-------+-------+ | 0 | 0 | 0 | byte 0| byte 1| byte 2| byte 3| | 0 | 0 | 1 | illegal | | 0 | 1 | 0 | illegal | | 0 | 1 | 1 | illegal | | 1 | 0 | 0 | byte 0| byte 1| byte 2| tail 3| | 1 | 0 | 1 | byte 0| byte 1| tail 2| (-) | | 1 | 1 | 0 | byte 0| tail 1| (-) | (-) | | 1 | 1 | 1 | tail 0| (-) | (-) | (-) | +------+-------+-------+-------+-------+-------+-------+ (-) this byte should be ignored. It may have any value. <05> The SYNCHRONOUS INTERFACE (aka the GIVING/TAKING (RDY/EN) Interface) -------------------------------------------------------------------- The data interface between a data source and a destination, used by the FI32 and the IDT-FIFOs, consists of Clock, data and GIVING (from the source to the destination) and TAKING (in the opposite direction). The data is 32 bit wide. The GIVING and TAKING are control signals. At every rising clock-edge the receiver samples the GIVING and the data, and presents TAKING, at the same time the sender samples the TAKING and presents the GIVING and the data. An asserted GIVING at the clock edge signifies that the data that is presented at that edge is valid. An asserted TAKING at the clock edge signifies that if, and only if, GIVING is also asserted, then the data presented by the sender (at the same edge) is accepted by the receiver, and the sender may change it. The actual behavior is symmetric: data transfer occurs only when both GIVING and TAKING are HIGH. Because the FI32 follows the IDT naming convention, RRDY and SEN are GIVING signals, whereas REN and SRDY are TAKING signals. Timing examples: __ __ __ __ __ __ _ Clock _| |__| |__| |__| |__| |__| |__| | _|_____|_____|___ | | | GIVING________________|___/ | | | \_|_____|_____|__ | | | | | | | __|_____|_____|_____|_____|_____|_____|__ TAKING_____________/ | | | | | | | | | | | | | | _ _____ _____ _____ _____ _____ _____ ____ Data presented _X__?__X__D1_X__D2_X__D3_X__?__X__?__X__?_ | | | | | | | Data accepted | | D1 | D2 | D3 | | | | | | | | | | Next, suppose that the receiver can handle immediately only the first word, and needs to take some time before being able to accept the other 2 words. | _|_____|_____|_____|_____|___ | | | GIVING________________|___/ | | | | | \_|_____|_____|__ | | | | | | | | | __|_____|___ | | _|_____|_____|_____|_____|__ TAKING_____________/ | | \_|_____|___/ | | | | | | | | | | | | | | _ _____ _____ _________________ _____ _____ _____ ____ Data presented _X__?__X__D1_X__D2____D2____D2_X__D3_X__?__X__?__X__?_ | | | | | | | | | Data accepted | | D1 | | | D2 | D3 | | | | | | | | | | | | <06> The Control Signals Between the FI32 and the SAN Link ----------------------------------------------------- ** Configuration Signals * TCLK (I) SAN Transmit Clock: This clock input sets the data rate of the SAN output link. A byte is sent on every transition, so, for the nominal Myrinet SAN link rate of 160 megabytes per second, this is an 80 MHz clock. The alowed range for duty cycle is 45-55%. Here is no restriction on the phase of TCLK with respect to any other clock (including TCLK on the other end of the SAN link). The frequency of TCLK must be within 1000ppm (0.1%) of 80MHz for optimum performance. * LVDD (I) SAN Low-Voltage-Driver-Supply Voltage: This pin should be connected to a 1.25V +/- 2% supply if it is to be compatible with other Myrinet SAN links. Power consumption varies with channel usage, 20mA minimum, 90mA maximum. If the SAN output drivers are shorted to GND, current draw can exceed 240mA. * VTH (I) SAN Input-Threshold Reference: This reference input should be LVDD/2 +/- 1%. Current draw is 1uA max. * BIAS (I) SAN Bias Reference: For 3.3V operation, this pin should be fed 1.0mA. At that current level, the voltage on the pin will be approxmately 1.4V. A 1.87KOhm resistor to 3.3V supply will achieve this. * OAH (O) SAN Impedence Reference: High: This pin should be connected to GND through a 50 ohm resistor. * OAL (O) SAN Impedence Reference: Low: This pin should be connected to LVDD through a 50-ohm resistor. <07> ** Input Channel * I0 - I7 and ID (SAN, I) SAN Input: The 8 data bits (I0-I7, I7 most significant) and the control (bit ID) are transition encoded. A transition on a data bit corresponds to the value of 1, no transition to the value of 0. A transition on the control bit corresponds to the data byte, no transition to the control symbol (see Myrinet SAN Link Specification). Each of these pins have a built-in 20KOhm pulldown resistor. * OB (SAN, O) SAN Output Block: This output is asserted to notify the connecting output SAN link channel that it must stop transmitting (see Myrinet SAN Link Specification). ** Output Channel * O0 - O7 and OD (SAN, O) SAN Output: The 8 data bits (O0-O7, O7 most significant) and the control (bit OD) are transition encoded. A transition on a data bit corresponds to the value of 1, no transition to the value of 0. A transition on the control bit corresponds to the data byte, no transition to the control symbol (see Myrinet SAN Link Specification). * IB (LVDS, I) SAN Input Block: When this input is asserted, the output channel stops transmitting (see Myrinet SAN Link Specification). This pin has a built-in 20KOhm pulldown resistor. <08> PIN ASSIGNMENT -------------- Vdd 1 GND 37 Vdd 73 GND 109 ST 2 S06 38 R07 74 I0 110 SP1 3 S05 39 R08 75 I1 111 SP0 4 S04 40 R09 76 I2 112 S31 5 S03 41 R10 77 I3 113 GND 6 Vdd 42 GND 78 Vdd 114 S30 7 S02 43 R11 79 I4 115 S29 8 S01 44 R12 80 I5 116 S28 9 S00 45 R13 81 I6 117 S27 10 SEN 46 R14 82 I7 118 Vdd 11 GND 47 Vdd 83 ID 119 S26 12 SRDY 48 R15 84 GND 120 S25 13 SAF- 49 R16 85 IB 121 S24 14 SCLK 50 R17 86 Vth 122 S23 15 TCLK 51 R18 87 OB 123 GND 16 Vdd 52 GND 88 OD 124 S22 17 WIN1 53 R19 89 Vdd 125 S21 18 WIN0 54 R20 90 GND 126 S20 19 RST 55 R21 91 LVDD 127 S19 20 CRC 56 R22 92 O7 128 Vdd 21 GND 57 Vdd 93 O6 129 S18 22 RCLK 58 R23 94 O5 130 S17 23 RBLCK 59 R24 95 GND 131 S16 24 RAE- 60 R25 96 BIAS 132 S15 25 RRDY 61 R26 97 LVDD 133 GND 26 Vdd 62 GND 98 O4 134 S14 27 REN 63 R27 99 O3 135 S13 28 R00 64 R28 100 O2 136 S12 29 R01 65 R29 101 Vdd 137 S11 30 R02 66 R30 102 GND 138 Vdd 31 GND 67 Vdd 103 LVDD 139 S10 32 R03 68 R31 104 O1 140 S09 33 R04 69 RP0 105 O0 141 S08 34 R05 70 RP1 106 GND 142 S07 35 R06 71 RT 107 OAL 143 GND 36 Vdd 72 GND 108 OAH 144 ****************************************************************************** <09> SPECIFICATIONS -------------- Vdd 3.3V Voh 2.7V min @4ma Vol 0.4V max @4mA Vih 2.1V min Vil 0.7V max SCLK 10 to 67 MHz RCLK 10 to 67 MHz TCLK 80 MHz Send-Rate min of (2*TCLK, 4*SCLK) Bytes Receive-Rate min of (2*tclk, 4*RCLK) Bytes (tclk is TCLK of the other end) Die Size 4.53 x 4.53 mm Package size 144-lead MQFP (MQuad) 28x28mm body, with lead pitch of 0.65mm (?). For engineering drawings of the package, please contact . ****************************************************************************** <10> APPLICATION NOTES ----------------- The FI is symmetric and could be "looped-back" at either end. It can also be cascaded (back to back or front to front, e.g., for testing or for repeaters). The FI32 may also be directly connected to an IDT FIFO for applications that require more than just few bytes of FIFO, as shown below. The FI32 uses 32 bits of data plus 3 control bits for framing (tails) and padding count, namely the ST, SP0, and SP1 for sending (system to FI32) and RT, RP0, and RP1 for receiving (FI32 to System). Since the data path of the IDT FIFO is 36 bits wide, embedding the control with the data is straight forward. SCLK TCLK | | +-v-----v-+ SAF-| <=== Configuration IDT FIFO <----| | Signals ---+-+-+----+-+-+OR SEN| | | | |....| | |-------------->| | ST 35 | | |....| | |B 35 SData| | SP0,SP1 ==/==>| | |....| | |======/=======>| | S00-S31 | | |....| | |ENB SRDY| | FIFO controls | | |....| | |<--------------| | ---+-+-+----+-+-+ \ | 10 | +====/=======> | FI32 | SAN-link IDT FIFO | <====/======== +-+-+----+-+-+---IR REN/ | 10 | | |....| | |-------------->| | RT 35 | | |....| | |A 35 RData| | RP0,RP1 <==/==| | |....| | |<=====/========| | R00-R31 | | |....| | |ENA RRDY| | FIFO controls | | |....| | |<--------------| | +-+-+----+-+-+--- RAE-| |RBLCK <----| |---> +----^----+ : | : RRDY and SEN are "GIVING" : RCLK : REN and SRDY are "TAKING" Host: :Myrinet side: :side (words): :(Bytes) The 35 bits of SData and RData are the 32 data bits, the tail, and the 2 bits of padding-count. Some IDT parts could be used here are IDT723631 (2KB: 512 x 36), IDT723641 (4KB:1K x 36), and IDT723651 (8KB: 2K x 36). However, these IDT FIFOs are 5V devices. Since these IDT devices (still?) use 5V, level-converters have to be used in between. <11> RCLK SCLK TCLK | | | RBLCK+----v----+ +-v-----v-+ <---| |RAE- SAF-| <=== Configuration | |----> <----| | Signals | |RRDY SEN| | | |------------------>| | | |RData 35 SData| | | |=========/========>| | | |REN SRDY| | | |<------------------| | 10 | / \ | 10 =====/=======> | | +====/=======> SAN-link | FI32 | | FI32 | SAN-Link <====/=======+ | | <====/======== 10 | \ SRDY REN/ | 10 | |------------------>| | | |SData 35 RData| | | |<========/=========| | | |SEN RRDY| | | |<------------------| | | |SAF- RAE-| |RBLCK Configuration ===>| |----> <----| |---> Signals +-^-----^-+ +----^----+ | | | SCLK TCLK RCLK A SAN-link Repeater RRDY and SEN are "GIVING" REN and SRDY are "TAKING" ..............................................................................