This is /ufs/gmbin/src/doc/gm.info, produced by Makeinfo version 3.12h from /ufs/gmbin/src/doc/gm.texi. ************************************************************************* * * * Myricom GM myrinet software and documentation * * * * Copyright (c) 2000 by Myricom, Inc. * * All rights reserved. * * * * Myricom, Inc. Email: info@myri.com * * 325 N. Santa Anita Ave. World Wide Web: http://www.myri.com/ * * Arcadia, CA 91024 * *************************************************************************  File: gm.info, Node: High Availability Extensions, Next: Utility Modules, Prev: Alarms, Up: Top High Availability Extensions **************************** While GM automatically handles transient network errors such as dropped, corrupted, or misrouted packets, and while the GM mapper automatically reconfigures the network if links or nodes appear or disappear, GM cannot automatically handle catastrophic errors such as crashed hosts or loss of network connectivity without the cooperation of the client program. When GM detects a catastrophic error, it temporarily disables the delivery of all messages with the same sender port, target port, and priority as the message that experienced the error, and GM informs the client of catastrophic network errors by passing a status other than `GM_SUCCESS' to the client's send completion callback routine. The client program is then expected to call either `gm_resume_sending()' or `gm_drop_sends()', which reenable the delivery of messages with the same sender port, target port, and priority. This mechanism preserves the message order over the prioritized connection between the sending and receiving ports, while allowing the client to decide if the other packets that it has already enqueued over the same connection should be transmitted or dropped. Simpler GM programs, such as MPI programs, will typically consider GM send errors to be fatal and will typically exit when they see a send error. This is reasonable for applications running on small or physically robust clusters where errors are rare and when users can tolerate restarting jobs in the rare event of a network error. Poorly written GM programs may simply ignore the error codes, which will cause the program to eventually hang with no error indication when catastrophic errors are encountered. This poor programming practice is strongly discouraged: Developers should always check the send completion status. More sophisticated applications, such as high availability database applications, will respond to the network faults, which appear to the client as send completion status codes other than `GM_SUCCESS'. The send completion status codes are as follows: `GM_SUCCESS' The send succeeded. This status code does not indicate an error. `GM_SEND_TIMED_OUT' The target port is open and responsive and the message is of an acceptable size, but the receiver failed to provide a matching receive buffer within the timeout period. This error can be caused by the receive neglecting its responsibility to provide receive buffers in a timely fashion or crashing. It can also be caused by severe congestion at the receiving node where many senders are contending for the same receive buffers on the target port for an extended period. This error indicates a programming error in the client software. `GM_SEND_REJECTED' The receiver indicated (in a call to `gm_set_acceptable_sizes()') the size of the message was unacceptable. This error indicates a programming error in the client software. `GM_SEND_TARGET_PORT_CLOSED' The message cannot be delivered because the destination port has been closed. `GM_SEND_TARGET_NODE_UNREACHABLE' The target node could not be reached over the Myrinet. This error can be caused by the network becoming disconnected for too long, the remote node being powered off, or by network links being rearranged when the Myrinet mapper is not running. `GM_SEND_DROPPED' The send was dropped at the client's request. (The client called `gm_drop_sends()'.) This status code does not indicate an error. `GM_SEND_PORT_CLOSED' Clients should never see this internal error code. When the send completion status code indicates an error a sophisticated client program may respond by calling `gm_resume_sending()' or `gm_drop_sends()'. Calling `gm_resume_sending()' causes GM to simply reenable delivery of subsequent messages over the connection, including those that have already been enqueued. This would be the typical response of a distributed database that assumes the underlying network is unreliable and layers its own reliability protocol over GM. Calling `gm_drop_sends()' causes GM to drop all enqueued sends over the disabled connection, return them to the client with status `GM_SEND_DROPPED', and reenable the connection. This would be the typical response of a program that wishes to reorder subsequent communication over the connection in response to the error. Note that each of the fault response functions (`gm_drop_sends()' and `gm_resume_sending()') requires a send token. This send token is implicitly returned to the caller when the callback function passed to `gm_drop_sends()' or `gm_resume_sending()' is called by GM.  File: gm.info, Node: Utility Modules, Next: Additional Features, Prev: High Availability Extensions, Up: Top Utility Modules *************** Some of GM's internal modules may be useful to GM developers, so their APIs are exposed. These modules include the following: * Menu: * CRC Functions:: * Hash Table:: * Lookaside List:: * Page Allocation::  File: gm.info, Node: CRC Functions, Next: Hash Table, Prev: Utility Modules, Up: Utility Modules CRC Functions ============= GM provides the following functions, which compute 32-bit CRCs on the contents of memory. These functions are not guaranteed to perform any particular variant of the CRC-32, but these functions are useful for creating robust hashing functions. - : unsigned long gm_crc (void *PTR, unsigned long LEN) computes a CRC-32 of the indicated range of memory. - : unsigned long gm_crc_str (char *PTR) computes a CRC-32 for the indicated string.  File: gm.info, Node: Hash Table, Next: Lookaside List, Prev: CRC Functions, Up: Utility Modules Hash Table ========== * Menu: * Hash Table Introduction:: * Hash Table API::  File: gm.info, Node: Hash Table Introduction, Next: Hash Table API, Prev: Hash Table, Up: Hash Table Introduction ------------ GM implements a generic hash table with a flexible interface. This module can automatically manage storage of fixed-size keys and/or data, or can allow the client to manage storage for keys and/or data. It allows the client to specify arbitrary hashing and comparison functions. For example, hash = gm_create_hash (gm_hash_compare_strings, gm_hash_hash_string, 0, 0, 0, 0); creates a hash table that uses null-terminated character string keys residing in client-managed storage, and returns pointers to data in client-managed storage. In this case, all pointers to hash keys and data passed by GM to the client will be the same as the pointers passed by the client to GM. As another example, hash = gm_create_hash (gm_hash_compare_ints, gm_hash_hash_int, sizeof (int), sizeof (struct my_big_struct), 100, 0); creates a hash table that uses `ints' as keys and returns pointers to copies of the inserted structures. All storage for the keys and data is automatically managed by the hash table. In this case, all pointers to hash keys and data passed by GM to the client will point to GM-managed buffers. This function also preallocates enough storage for 100 hash entries, guaranteeing that at least 100 key/data pairs can be inserted in the table if the hash table creation succeeds. The automatic storage management option of GM not only is convenient, but also is extremely space efficient for keys and data no larger than a pointer, because when keys and data are no larger than a pointer, GM automatically stores them in the space reserved for the pointer to the key or data, rather than allocating a separate buffer. Note that all keys and data buffers are referred to by pointers, not by value. This allows keys and data buffers of arbitrary size to be used. As a special (but common) case, however, one may wish to use pointers as keys directly, rather than use what they point to. In this special case, use the following initialization, and pass the keys (pointers) directly to the API, rather than the usual references to the keys. hash = gm_create_hash (gm_hash_compare_ptrs, gm_hash_hash_ptr, 0, DATA_LEN, MIN_CNT, FLAGS); While it is possible to specify a KEY_LEN of `sizeof (void *)' during initialization and treat pointer keys just like any other keys, the API above is more efficient, more convenient, and completely architecture independent.  File: gm.info, Node: Hash Table API, Prev: Hash Table Introduction, Up: Hash Table Hash Table API -------------- Some day the GM hash table API may be extended, but the current API is as follows: - : struct gm_hash * gm_create_hash (long (*GM_CLIENT_COMPARE) (void *key1, void *key2), gm_u32_t (*GM_CLIENT_HASH) (void *key1), unsigned long KEY_LEN, unsigned long DATA_LEN, unsigned long MIN_CNT, int FLAGS) returns a newly-created `gm_hash' structure or `0' if the hash table could not be created. The parameters are as follows: GM_CLIENT_COMPARE the function used to compare keys and may be any of `gm_hash_compare_ints' `gm_hash_compare_longs' `gm_hash_compare_ptrs' `gm_hash_compare_strings' or may be a client-defined function. GM_CLIENT_HASH the function to be used to hash keys and may be any of `gm_hash_hash_int' `gm_hash_hash_long' `gm_hash_hash_ptr' `gm_hash_hash_string' or may be a client-defined function. KEY_LEN specifies the length of the keys to be used for the hash table, or `0' if the keys should not be copied into GM-managed buffers. DATA_LEN specifies the length of the data to be stored in the hash table, or `0' if the data should not be copied into GM-managed buffers. MIN_CNT specifies the number of entries for which storage should be preallocated. FLAGS should be `0' because no flags are currently defined. - : void gm_destroy_hash (struct gm_hash *H) frees all resources associated with the hash table, except for any client-allocated buffers. - : void gm_hash_rekey (struct gm_hash *HASH, void *OLD_KEY, void *NEW_KEY) finds each entry with key OLD_KEY and changes the key used to store the data to NEW_KEY. This call is guaranteed to succeed. - : void * gm_hash_remove (struct gm_hash *HASH, void *KEY) removes an entry associated with KEY from the hash table HASH and returns a pointer to the data associated with the key, or `0' if no match exists. If the data resides in a GM-managed buffer, it is only guaranteed to be valid until the next operation on the hash table. - : void * gm_hash_find (struct gm_hash *HASH, void *KEY_PTR) finds an entry associated with KEY from the hash table HASH and returns a pointer to the data associated with the key, or `0' if no match exists. - : gm_status_t gm_hash_insert (struct gm_hash *HASH, void *KEY, void *DATA) stores the association of KEY and DATA in the hash table HASH. The key `*'KEY (or data `*'DATA) is copied into the hash table unless the table was initialized with a KEY_LEN (or DATA_LEN) of 0.  File: gm.info, Node: Lookaside List, Next: Page Allocation, Prev: Hash Table, Up: Utility Modules Lookaside List ============== GM implements a lookaside list, which may be used to manage small fixed-length blocks more efficiently than `gm_malloc()' and `gm_free()'. Lookaside lists can also be used to ensure that at least a minimum number of blocks are available for allocation at all times. GM lookaside lists have the following API: - : struct gm_lookaside * gm_create_lookaside (unsigned long ENTRY_LEN, unsigned long MIN_ENTRY_CNT) returns a newly created lookaside list to be used to allocate blocks of ENTRY_LEN bytes. MIN_ENTRY_CNT entries are preallocated. - : void gm_destroy_lookaside (struct gm_lookaside *L) frees a lookaside list and all associated resources, including any buffers currently allocated from the lookaside list. - : void * gm_lookaside_alloc (struct gm_lookaside *L) returns a buffer of size ENTRY_LEN specified when the entry list L was created, or `0' if the buffer could not be allocated. - : void gm_lookaside_free (void *PTR) frees a block of memory previously allocated by a call to `gm_lookaside_alloc()'. The contents of the block of memory are guaranteed to be unchanged until the next operation is performed on the lookaside list.  File: gm.info, Node: Page Allocation, Prev: Lookaside List, Up: Utility Modules Page Allocation =============== The following GM API allows pages to be allocated and freed. - : void * gm_page_alloc () allocate a page-aligned buffer of length `GM_PAGE_LEN'. - : void gm_page_free (void *ADDR) frees the page at ADDR previously allocated by `gm_page_alloc()'.  File: gm.info, Node: Additional Features, Next: GM Constants and Macros, Prev: Utility Modules, Up: Top Additional Features ******************* The function `(char *)_gm_get_route(PORT, NODE_ID, LENGTH_P)' returns a pointer to the route to the network host with GM ID NODE_ID and stores the length of the route in `*LENGTH_P' (1). In the future, a standard mechanism for determining the type of a remote node will be specified. It will involve sending a request to the node to return the type to avoid storing types for all remote nodes in local memory. ---------- Footnotes ---------- (1) The syntax of this call may change.  File: gm.info, Node: GM Constants and Macros, Next: Function Summary, Prev: Additional Features, Up: Top GM Constants and Macros *********************** - Enum: GM_HIGH_PRIORITY The priority of high priority messages (1). - Enum: GM_LOW_PRIORITY The priority of low priority messages (0).  File: gm.info, Node: Function Summary, Next: Token Reference, Prev: GM Constants and Macros, Up: Top Function Summary **************** The following miscellaneous library functions are provided. Several are simply cover functions for standard Unix library functions, but are provided to simplify the creation of portable GM programs, or to provide the ANSI functionality on non-ANSI systems, such as Windows NT. - : void gm_abort () Cover function for `abort()'. Aborts the current process. - : void * gm_alloc_pages (unsigned long LEN) Allocate LEN bytes, aligned on a page boundary. Any fractional page following the buffer is wasted. - : int gm_alloc_send_token (struct gm_port *PORT, unsigned int PRIORITY) This trivial utility function allocates a send token of priority `priority' previously freed with `gm_free_send_token()' or returns `0' if no token is available. Clients may choose to maintain their own send token counts without using this utility function. - : void gm_allow_remote_memory_access (struct gm_port *PORT) Allow _any_ remote GM port to modify the contents of _any_ GM DMAable memory using the `gm_directed_send()' function. This is a significant security hole, but is very useful on tightly coupled clusters on trusted networks. - : void gm_bcopy (void *FROM, void *TO, unsigned long LEN) Copy LEN bytes starting at FROM to location TO. Does not handle overlapping regions. - : union gm_receive_event * gm_blocking_receive (struct gm_port *PORT) Block until there is a receive event and then return a pointer to the event. If no send is immediately available, this call suspends the current process until a receive event is available. As an optimization for applications with one CPU per CPU-intensive thread, this function polls for receives for one millisecond before sleeping the process, so it is not suited for machines running more than one performance critical process or thread on the machine. - : union gm_receive_event * gm_blocking_receive_no_spin (struct gm_port *PORT) This function behaves just like `gm_blocking_receive()', only it sleeps the current thread immediately if no receive is pending. It is well suited to applications with more than one CPU-intensive thread per processor. - : void gm_bzero (void *PTR, int LEN) Clear the LEN bytes of memory starting at PTR. - : void * gm_calloc (unsigned long LEN, unsigned long CNT) Allocate and clear an array of CNT elements of length LEN. - : void gm_close (struct gm_port *PORT) Close a previously opened port, and free all resources associated with the port. - : void gm_datagram_send (struct gm_port *PORT, void *MESSAGE, unsigned int SIZE, unsigned int LEN, unsigned int PRIORITY, unsigned int TARGET_NODE_ID, unsigned int TARGET_PORT_ID, gm_send_completion_callback_t CALLBACK, void *CONTEXT) Queue MESSAGE of length LENGTH to be sent unreliably to a buffer of size SIZE at TARGET_PORT_ID on TARGET_NODE_ID. LENGTH must be no larger than `GM_MTU' If any network error is encountered while sending the packet, the packet is silently and immediately dropped. After the packet has been DMA'd from host memory, `CALLBACK(PORT,CONTEXT,STATUS)' is called inside a user invokation of `gm_unknown()', reporting the STATUS of the attempted send. - : gm_status_t gm_deregister_memory (struct gm_port *PORT, void *PTR, unsigned LEN) Deregister LEN bytes of user virtual memory starting at PTR that were previously registered for DMA transfers with a matching call to `gm_register_memory()'. - : void gm_directed_send (struct gm_port *P, void *SOURCE_BUFFER, gm_remote_ptr_t TARGET_BUFFER, unsigned long LEN, enum gm_priority PRIORITY, unsigned int TARGET_NODE_ID, unsigned int TARGET_PORT_ID) _This function is deprecated. Use `gm_directed_send_with_callback()' instead._ Transfer the LEN bytes at SOURCE_BUFFER to TARGET_PORT_ID on TARGET_NODE_ID with priority PRIORITY and store the data at the remote virtual memory address TARGET_BUFFER. The order of the transfer is preserved relative to messages of the same priority sent using `gm_send()' or `gm_send_to_peer()'. - : void gm_directed_send_with_callback (struct gm_port *P, void *SOURCE_BUFFER, gm_remote_ptr_t TARGET_BUFFER, unsigned long LEN, enum gm_priority PRIORITY, unsigned int TARGET_NODE_ID, unsigned int TARGET_PORT_ID, gm_send_completion_callback_t CALLBACK, void *CONTEXT) Transfer the LEN bytes at SOURCE_BUFFER to TARGET_PORT_ID on TARGET_NODE_ID with priority PRIORITY and store the data at the remote virtual memory address TARGET_BUFFER. Call `CALLBACK(PORT,CONTEXT,STATUS)' when the send completes or fails, with STATUS indicating the status of the send. The order of the transfer is preserved relative to messages of the same priority sent using `gm_send()' or `gm_send_to_peer()'. - : void * gm_dma_calloc (struct gm_port *PORT, unsigned LENGTH, unsigned LENGTH) Allocates and clears COUNT*LENGTH bytes of DMAable memory aligned on a 4-byte boundary. - : void gm_dma_free (struct gm_port *PORT, void *PTR) Frees PTR, which was allocated by a call to `gm_dma_calloc()' or `gm_dma_malloc()'. Note that the memory is not necessarily unlocked and returned to the operating system, but may be reused in future calls to `gm_dma_calloc()' and `gm_dma_malloc()'. - : void * gm_dma_malloc (struct gm_port *PORT, unsigned COUNT) Allocates LENGTH bytes of DMAable memory aligned on a 4-byte boundary. - : void gm_drop_sends (struct gm_port *PORT, unsigned int PRIORITY, unsigned int TARGET_NODE_ID, unsigned int TARGET_PORT_ID); Drop all enqueued sends for PORT of priority PRIORITY destined for TARGET_PORT_ID of TARGET_NODE_ID to be dropped, and reenable packet transmission on that connection. This function should only be called after an error is reported to a send completion callback routine, and only with parameters matching those of the failed send. It should be called only once per reported error. The dropped sends will be returned to the client with a status of `GM_SEND_DROPPED'. This function requires a send token, which will be returned to the client using a `GM_FREE_SEND_TOKEN_EVENT' or `GM_FREE_HIGH_SEND_TOKEN_EVENT', as determined by PRIORITY. - : void gm_exit (gm_status_t STATUS) Cause the current process to exit with a status appropriate to the GM status code `status'. - : void gm_finalize () Decrement the GM initialization counter, and if it becomes zero, free all resources associated with GM in the current process. Each call to `gm_finalize()' should be matched by a call to `gm_init()'. - : void gm_free (void *PTR) Free the memory buffer at PTR, which was previously allocated by `gm_malloc()', or `gm_calloc()'. - : void gm_free_pages (void *PTR) Free the pages at PTR, which were previously allocated with `gm_alloc_pages()'. - : void gm_free_send_token (struct gm_port *PORT, unsigned int PRIORITY) This trivial utility function increments a count of free send tokens of `priority' for `port' so that it can later be allocated using `gm_alloc_send_token()'. Clients may choose to maintain their own count of send tokens in the client's possession instead of using this utility function. - : void gm_free_send_tokens (struct gm_port *PORT, unsigned int PRIORITY, unsigned int COUNT) Like `gm_free_send_token()', but can be used to free zero or more tokens. - : gm_status_t gm_get_host_name (struct gm_port *PORT, char NAME[GM_MAX_HOST_NAME_LEN]) Copy the host name of the local node to NAME. - : gm_status_t gm_get_node_id (struct gm_port *PORT, unsigned int *N) Copy the GM ID of the network interface card associated with PORT to `*'N. - : gm_status_t gm_get_unique_board_id (struct gm_port *PORT, char UNIQUE[6]) Copy the 6-byte MAC address of the network interface card associated with PORT to the buffer at UNIQUE. - : void gm_handle_directed_send_notification (struct gm_port *port, gm_recv_event_t *EVENT) _Deprecated_. When a `GM_DIRECTED_SEND_NOTIFICATION' event is received, programs may optionally call this function instead of `gm_unknown()' as an optimization for handling send completions more efficiently. This function does not improve performance on newer hardware, which does not generate `GM_DIRECTED_SEND_NOTIFICATION' events. - : void gm_handle_sent_tokens (struct gm_port *PORT, gm_recv_event_t *EVENT); _Deprecated_. When a `GM_DIRECTED_SEND_NOTIFICATION' event is received, programs may optionally call this function instead of `gm_unknown()' as an optimization for handling send completions more efficiently. This function does not improve performance on newer hardware, which does not generate `GM_DIRECTED_SEND_NOTIFICATION' events. - : unsigned int gm_host_name_to_node_id (struct gm_port *PORT, char *HOST_NAME) Return the GM ID associated with HOST_NAME or `GM_NO_SUCH_NODE_ID' in case of an error. - : gm_status_t gm_init () Increment the GM initialization counter and ianitialize GM if it was uninitialized. This call must be performed before any other GM call and before any reference to a GM global variable (e.g.: `GM_PAGE_LEN'). Each call to `gm_init()' should be matched by a call to `gm_finalize()'. (1) - : unsigned long gm_log2_roundup (unsigned long N) Returns the logarithm, base 2, of N, rounded up to the next integer. - : void * gm_malloc (unsigned long LEN) Allocates and returns a pointer to LEN bytes of uninitialized memory. The memory should be freed with `gm_free()'. - : unsigned long gm_max_length_for_size (unsigned int SIZE) Return the maximum length of a message that will fit in a GM buffer of size SIZE. - : gm_status_t gm_max_node_id (struct gm_port *port, unsigned int *n) Store the maximum GM node ID supported by the the network interface card corresponding to PORT at `*'N. - : gm_status_t gm_max_node_id_in_use (struct gm_port *port, unsigned int *n) Store the maximum in-use node ID for the network attached to PORT at `*N'. - : int gm_memcmp (void *A, void *B, unsigned long LEN) Emulate the ANSI `memcmp()' function. - : void * gm_memorize_message (void *MESSAGE, void *BUFFER, unsigned int LEN) Function for optionally optimizing the handling "`FAST'" receive messages as described in *Note Receiving Messages::. If MESSAGE and BUFFER differ, `gm_memorize_message(PORT, MESSAGE,BUFFER)' copies the message pointed to by MESSAGE into the buffer pointed to by BUFFER. `gm_memorize_message()' returns BUFFER. This function is optimized for performing such aligned copies. - : unsigned int gm_min_message_size (struct gm_port *PORT) Returns the minimum message size supported by this GM implementation. - : unsigned int gm_min_size_for_length (unsigned long LENGTH) Returns the minimum GM message buffer size required to store a message of length LENGTH. - : char * gm_node_id_to_host_name (struct gm_port *PORT, unsigned int NODE_ID) Return a pointer to the host name of the host containing the network interface card with GM node id NODE_ID. The name referenced by the returned pointer is only valid until the next GM API call. - : gm_status_t gm_node_id_to_unique_id (struct gm_port *PORT, unsigned int N, char UNIQUE[6]) Store the MAC address for the interface with GM ID N at UNIQUE. - : unsigned int gm_num_ports (struct gm_port *P) Return the number of ports supported by the the network interface card associated with port P. - : unsigned int gm_num_receive_tokens (struct gm_port *P) Return the number of receive tokens implicitly owned by the port's client after the port is opened. - : unsigned int gm_num_send_tokens (struct gm_port *P) Return the number of send tokens implicitly owned by the port's client after the port is opened. - : gm_status_t gm_open (struct gm_port **P, int DEVICE, int PORT, char *PORT_NAME) Opens GM port PORT for LANai interface DEVICE, a pointer to the port's state at `*'P. This pointer must be passed to all subsequent functions that operate on the opened port. PORT_NAME is a null-terminated ASCII string that is used to identify the port client for debugging (and potentially other) purposes; pass in the name of your program. Note that unit and port numbers start at 0, and that ports 0 and 1 reserved, so clients will usually open ports 2 and higher. - : void * gm_page_alloc () Returns a pointer to a newly allocated aligned uninitialized page of memory. - void: gm_page_free (void *ADDR) Frees the page of memory at ADDR, an address returned by `gm_page_alloc()'. - : void gm_perror (char *MESSAGE, gm_status_t ERRNO) Similar to ANSI standard `perror', but takes the error code as a parameter to allow thread safety in future implementations, and only supports GM error numbers. Prints MESSAGE followed by a description of ERRNO. - : int gm_printf (char *FORMAT, ...) Emulate or invoke the ANSI standard `printf()' function. - : void gm_provide_receive_buffer (struct gm_port *PORT, void *MESSAGE, unsigned SIZE, unsigned PRIORITY) Equivalent to calling `gm_provide_receive_buffer_with_tag(...,0)', and no faster than doing so. It is included for backwards compatibility. Many new clients will want to use `gm_provide_receive_buffer_with_tag()' instead. - : void gm_provide_receive_buffer_with_tag (struct gm_port *PORT, void *MESSAGE, unsigned SIZE, unsigned PRIORITY, unsigned int TAG) This function provides GM with a buffer into which it can receive messages with matching `size' and `priority' fields. It is the client software's responsibility to provide buffers of each SIZE and PRIORITY that might be received; not doing so can cause program deadlock, which will eventually result in the port being closed after a timeout(2). The client software may provide up to `gm_num_receive_tokens()' different receive buffers into which messages may be received. Each buffer provided by the client software to GM via this function will be used only once to receive a message. In other words, calling `gm_provide_receive_buffer(PORT, BUFFER, SIZE, PRIORITY)' provides GM a token to receive a single message of size SIZE and priority PRIORITY into the receive buffer BUFFER. When a message is eventually received into this buffer, `gm_receive(PORT)' stores the buffer pointer BUFFER and TAG in the returned event, returning control of the buffer (token) to the client software. If the client software wishes for the buffer to be reused for a similar receive, it must call `gm_provide_receive_buffer()' again with the same or similar parameters. Once a buffer has been provided to GM, its content should not be changed until control of the buffer has been returned to the client software via `gm_receive()'. The TAG parameter must be in the range [0,255], and is returned in the receive event describing a receive into the buffer. It may be used in any way the client desires, and need not be unique. - : int gm_rand () Returns a pseudo-random integer, using a poor but fast random number generator. - : int gm_rand_mod (int MODULUS). Returns a pseudo-random number modulo MODULUS, using a poor but fast random number generator. - : union gm_recv_event * gm_receive (struct gm_port *P) Returns a receive event. If no significant receive event is pending, then an event of type `GM_NO_RECV_EVENT' is immediately returned. - : int gm_receive_pending (struct gm_port *PORT) Returns nonzero if a receive event is pending. If a receive event is pending A call to any `gm_receive*)()' function will return the event immediately, although `gm_receive()' is preferred in this case for efficiency. - : void * gm_register_memory (struct gm_port *PORT, void *PTR, unsigned LEN) Register LEN bytes of user virtual memory starting at PTR for DMA transfers. The memory is locked down (made nonpageable) and DMAs on the region of memory are enabled. Memory may be registered multiple times. Memory may be deregistered using matching calls to `gm_deregister_memory()'. Note that memory registration is an expensive operation relative to sending and receiving packets, so you should use persistent memory registrations wherever possible. Also note that memory registration is not supported on Solaris due to operating system limitations. - : void gm_resume_sending (struct gm_port *PORT, unsigned int PRIORITY, unsigned int TARGET_NODE_ID, unsigned int TARGET_PORT_ID); Reenable packet transmission of messages from PORT of priority PRIORITY destined for TARGET_PORT_ID of TARGET_NODE_ID. This function should only be called after an error is reported to a send completion callback routine, and only with parameters matching those of the failed send. It should be called only once per reported error. The dropped sends will be returned to the client with a status of `GM_SEND_DROPPED'. This function requires a send token, which will be returned to the client using a `GM_FREE_SEND_TOKEN_EVENT' or `GM_FREE_HIGH_SEND_TOKEN_EVENT', as determined by PRIORITY. - : void gm_send (struct gm_port *PORT, void *MESSAGE, unsigned int SIZE, unsigned int LEN, unsigned int PRIORITY, unsigned int TARGET_NODE_ID, unsigned int TARGET_PORT_ID) This function is _deprecated_ and included only for backwards compatibility with GM-1.0. Queues the MESSAGE of length LEN to be sent with priority PRIORITY to node TARGET_NODE_ID. Before calling `gm_send()', client software must first possess a send token of the same priority obtained by a successful call to `gm_alloc_send_token()' or `gm_get_sent_message()', and by calling `gm_send ()' the client implicitly relinquishes this send token. After a call to `gm_send(..., MESSAGE, LEN, ...)', the memory specified by MESSAGE and LEN must not be modified until MESSAGE is returned by `gm_get_sent_message()'. The buffer pointed to by MESSAGE is not modified by GM between the time `gm_send(PORT, MESSAGE,...)' is called and the time that the sent message pointer appears in a `GM_SENT_EVENT'. - : void gm_send_with_callback (struct gm_port *PORT, void *MESSAGE, unsigned int SIZE, unsigned int LEN, unsigned int PRIORITY, unsigned int TARGET_NODE_ID, unsigned int TARGET_PORT_ID, gm_send_completion_callback_t CALLBACK, void *CONTEXT) Queues the MESSAGE of length LEN to be sent with priority PRIORITY to node TARGET_NODE_ID. Before calling `gm_send()', client software must first possess a send token of the same priority obtained by a successful call to `gm_alloc_send_token()' or `gm_get_sent_message()', and by calling `gm_send ()' the client implicitly relinquishes this send token. After a call to `gm_send(..., MESSAGE, LEN, ...)', the memory specified by MESSAGE and LEN must not be modified until MESSAGE is returned by `gm_get_sent_message()'. After the send completes, `CALLBACK(PORT,CONTEXT,STATUS)' will be called inside `gm_unknown()', with STATUS indicating the status of the completed send. The buffer pointed to by MESSAGE should not be modified by GM between the time `gm_send(PORT, MESSAGE,...)' is called and the time that the sent completes. - : void gm_send_to_peer (struct gm_port *PORT, void *MESSAGE, unsigned int LEN, unsigned int PRIORITY, unsigned int TARGET_NODE_ID) This function is _deprecated_ and is included only for backwards compatibility with GM-1.0. Like `gm_send()', only with the TARGET_NODE_ID implicitly set to the same ID as PORT. This function is marginally faster than `gm_send()'. - : void gm_send_to_peer_with_callback (struct gm_port *PORT, void *MESSAGE, unsigned int LEN, unsigned int PRIORITY, unsigned int TARGET_NODE_ID, gm_send_completion_callback_t CALLBACK, void *CONTEXT) Like `gm_send_with_callback()', only with the TARGET_NODE_ID implicitly set to the same ID as PORT. This function is marginally faster than `gm_send_with_callback()'. - : int gm_send_token_available (struct gm_port *PORT, unsigned PRIORITY) Returns the value `gm_alloc_send_token(PORT, PRIORITY)' would return, without actually allocating a send token. This function allows client software to test for the availability of a send token without actually allocating the send token. - : gm_status_t gm_set_acceptable_sizes (struct gm_port *P, enum gm_priority PRIORITY, unsigned long MASK) Inform GM of the acceptable sizes of GM messages received on port P with priority PRIORITY. Each set bit of MASK represents indicates an acceptable size. While calling this function is not required, clients should call it during program initialization to errors involving the reception of badly sized messages to be reported nearly instantaneously, rather than after a substantial delay of 30 seconds or more. - : int gm_sleep (unsigned SECONDS) Emulate ANSI standard `sleep()', sleeping the entire process for SECONDS seconds. - : gm_status_t gm_node_id_to_unique_id (struct gm_port *PORT, unsigned int N, char UNIQUE[6]) Store the MAC address for the interface with GM ID N at UNIQUE. - : void gm_unknown (struct gm_port *P, union gm_recv_event *E) This functions handles all GM events not recognized or processed by the client software, allowing the GM library and network interface card firmware to interact. This functions also catches and reports several common client program errors, and converts some unrecognizable events into recognizable form for the client. ---------- Footnotes ---------- (1) Currently, `gm_open()' implicitly calls `gm_init()' for the caller and `gm_close()' implicitly calls `gm_finalize()', but developers should not depend on this. (2) For a future version of GM, an exception notification mechanism will report this exception, instead.  File: gm.info, Node: Token Reference, Next: Glossary, Prev: Function Summary, Up: Top Token Reference *************** The following functions require a send tokens: `gm_drop_sends()' `gm_resume_sending()' `gm_send()' `gm_send_to_peer()' `gm_send_to_peer_with_callback()' `gm_send_with_callback()' The send token is implicitly returned to the client when the function's callback is called or, for the GM-1.0 functions `gm_send()' and `gm_send_to_peer()', a send token is implicitly passed to the client with each pointer returned in a `GM_SENT_EVENT'. (The legacy `GM_SENT_EVENT's are generated if and only if the legacy `gm_send()' and `gm_send_to_peer()' functions are called.) The following functions require a receive token: `gm_provide_receive_buffer()' `gm_provide_receive_buffer_with_tag()' A single receive token is passed to the client with each of the following events: `GM_RAW_RECV_EVENT' `GM_RECV_EVENT' `GM_HIGH_RECV_EVENT' `GM_HIGH_PEER_RECV_EVENT' `GM_FAST_HIGH_RECV_EVENT' `GM_FAST_HIGH_PEER_RECV_EVENT' (However, if the client passes these event to `gm_unknown()', then the token is implicitly returned to GM.) Any of the GM receive functions can generate these types of events. These functions are: `gm_receive()' `gm_blocking_receive()' `gm_blocking_receive_no_spin()'  File: gm.info, Node: Glossary, Next: Variable Index, Prev: Token Reference, Up: Top Glossary ******** The following terms abbreviations are used in the GM documentation and source code. Some of these abbreviations are obvious to speakers of English, but are included for speakers of other languages. This section does not include architecture-specific abbreviations used in the architecture-specific GM driver code, as those are documented by the architecture's vendor and are not of interest to most GM developers. _accum_ accumulator _addr_ address _alloc_ allocate _arch(s)_ architecture(s) _buf_ _buff_ buffer _cnt_ count _create_ allocate and then initialize _destroy_ finalize and then free _dma_ direct memory access _hash_ hash table _hp_ host pointer (a pointer of the appropriate size for the host architecture in question) _insn(s)_ instruction(s) _intr_ interrupt _KVMA_ kernel virtual memory address _lookaside_ lookaside list _LSB(s)_ least significant byte(s) _lsb('s)_ least significant bit(s) _MAC_ Media Access Control. This is a commonly referred to sublayer of the datalink layer in the ISO network reference model. _MAC Address_ A 6-byte address unique to a Myrinet interface. It is equivalent to an ethernet address. _minor_ device minor number _num_ number _phys_ physical _pre_ prefetch or precompute _PTE_ page table entry _recv_ receive _ref_ reference _seg_ segment _sema_ semaphore _UVMA_ user virtual memory address _virt_ virtual _VM_ virtual memory _VMA_ virtual memory address _zalloc_ allocate and clear  File: gm.info, Node: Variable Index, Next: Concept Index, Prev: Glossary, Up: Top Variable Index ************** * Menu: * GM_HIGH_PRIORITY: GM Constants and Macros. * GM_LOW_PRIORITY: GM Constants and Macros.  File: gm.info, Node: Concept Index, Prev: Variable Index, Up: Top Concept Index ************* * Menu: * (: Function Summary. * Copyright Notice: Copyright Notice. * gm_abort: Function Summary. * gm_alloc_pages: Function Summary. * gm_alloc_send_token: Function Summary. * gm_allow_remote_memory_access: Function Summary. * gm_bcopy: Function Summary. * gm_blocking_receive: Function Summary. * gm_blocking_receive_no_spin: Function Summary. * gm_bzero: Function Summary. * gm_calloc: Function Summary. * gm_cancel_alarm: Alarms. * gm_close: Function Summary. * gm_crc: CRC Functions. * gm_crc_str: CRC Functions. * gm_create_hash: Hash Table API. * gm_create_lookaside: Lookaside List. * gm_datagram_send: Function Summary. * gm_deregister_memory: Function Summary. * gm_destroy_hash: Hash Table API. * gm_destroy_lookaside: Lookaside List. * gm_directed_send: Function Summary. * gm_directed_send_with_callback: Function Summary. * gm_dma_calloc: Function Summary. * gm_dma_free: Function Summary. * gm_dma_malloc: Function Summary. * gm_drop_sends: Function Summary. * gm_exit: Function Summary. * gm_finalize: Function Summary. * gm_free: Function Summary. * gm_free_pages: Function Summary. * gm_free_send_token: Function Summary. * gm_free_send_tokens: Function Summary. * gm_get_host_name: Function Summary. * gm_get_node_id: Function Summary. * gm_get_unique_board_id: Function Summary. * gm_handle_directed_send_notification: Function Summary. * gm_handle_sent_tokens: Function Summary. * gm_hash_find: Hash Table API. * gm_hash_insert: Hash Table API. * gm_hash_rekey: Hash Table API. * gm_hash_remove: Hash Table API. * gm_host_name_to_node_id: Function Summary. * gm_init: Function Summary. * gm_initialize_alarm: Alarms. * gm_log2_roundup: Function Summary. * gm_lookaside_alloc: Lookaside List. * gm_lookaside_free: Lookaside List. * gm_malloc: Function Summary. * gm_max_length_for_size: Function Summary. * gm_max_node_id: Function Summary. * gm_max_node_id_in_use: Function Summary. * gm_memcmp: Function Summary. * gm_memorize_message: Function Summary. * gm_min_message_size: Function Summary. * gm_min_size_for_length: Function Summary. * gm_node_id_to_host_name: Function Summary. * gm_node_id_to_unique_id: Function Summary. * gm_num_ports: Function Summary. * gm_num_receive_tokens: Function Summary. * gm_num_send_tokens: Function Summary. * gm_open: Function Summary. * gm_page_alloc <1>: Function Summary. * gm_page_alloc: Page Allocation. * gm_page_free: Page Allocation. * gm_perror: Function Summary. * gm_printf: Function Summary. * gm_provide_receive_buffer: Function Summary. * gm_provide_receive_buffer_with_tag: Function Summary. * gm_rand: Function Summary. * gm_rand_mod: Function Summary. * gm_receive: Function Summary. * gm_receive_pending: Function Summary. * gm_register_memory: Function Summary. * gm_resume_sending: Function Summary. * gm_send: Function Summary. * gm_send_token_available: Function Summary. * gm_send_to_peer: Function Summary. * gm_send_to_peer_with_callback: Function Summary. * gm_send_with_callback: Function Summary. * gm_set_acceptable_sizes: Function Summary. * gm_set_alarm: Alarms. * gm_sleep: Function Summary. * gm_unknown: Function Summary.