README for vxworks build-install 1. Introduction This file describes how to build and install the gm driver for vxworks. This build is still a little rough, so read this file carefully. 2. Building GM 2.1 Modify gm/drivers/vxworks/gm/gm_bsp_support.h The first thing that needs to be done is to modify the gm_bsp_support.h to suit your particular BSP. The file in gm/drivers/vxworks/gm/README.gm_bsp_support has more information on what needs to be defined in this file. Also, gm_bsp_support.h already includes configuration for several boards which serve as a good starting point for defining new entries. 2.2 Setting of environment variables Before running 'configure', several environment variables need to be set. Here is an example of the environment variables needed for building GM for the Motorolla MTX603 board. (Note long lines have been indented for readabilty and should be considered as single lines when setting up your environment). unsetenv CC unsetenv CFLAGS unsetenv CXX unsetenv CXXFLAGS unsetenv CPP setenv CC "ccppc" setenv CFLAGS "-B/ufs/wind/end2/host/sun4-sunos4/lib/gcc-lib/ -mstrict-align -nostdinc -O2 -fvolatile -fno-builtin -Wall -I/ufs/wind/end/target/h -I/ufs/wind/end/target/config/mtx603 -I/ufs/wind/end/target/config/all -DCPU=PPC603 -DMTX_603 -c" setenv CXX "c++ppc" setenv CXXFLAGS "-B/ufs/wind/end2/host/sun4-sunos4/lib/gcc-lib/ -mstrict-align -nostdinc -O2 -fvolatile -fno-builtin -Wall -I/ufs/wind/end/target/h -I/ufs/wind/end/target/config/mtx603 -I/ufs/wind/end/target/config/all -DCPU=PPC603 -DMTX_603 -c" setenv CPP "cppppc -nostdinc -I/ufs/wind/end/target/h -I/ufs/wind/end/target/config/mtx603 -I/ufs/wind/end/target/config/all -DCPU=PPC603 -DMTX_603" ******************** WARNING ******************** The Tornado 2 tools no longer include cppppc. As a replacement please use 'ccppc -E' instead. Forgetting to do this will create a very broken configuration in the configure step below, resulting in unusual errors during compile. Under Tornado 2, your CPP flag would look like: setenv CPP "ccppc -E -nostdinc -I/ufs/wind/end/target/h -I/ufs/wind/end/target/config/mtx603 -I/ufs/wind/end/target/config/all -DCPU=PPC603 -DMTX_603" ***************** END OF WARNING *************** Also look at drivers/vxworks/gm/README-* for specific information on configuring your BSP. Please return any modifications to myricom so that we can add them to the general distribution. 2.3 Configure To configure, run: ./configure --target=ppc-vxworks If you didn't get prebuilt mcps from Myricom, you will need to add "--enable-mcp-build" to the configure line above. 2.4 Make After running 'configure', simply type 'make' to build gm. If the code builds correctly, the vxworks loadable module will be at: gm/binary/sbin/gm . Also look at gm/binary/GM_INSTALL for a sample script on how to load the gm driver. 3. Configuring vxWorks to use the GM_SENS driver. To use the GM SENS driver with vxWorks you have to modify configNet.h and the Makefile in your BSP config directory and then rebuild vxWorks. You will also have to modify your vxWorks startup script. 3.1 configNet.h changes First add an entry for the GM SENS driver in the endDevTbl[]. In the example below, the second line in the table is the entry for the GM SENS driver. extern END_OBJ* geLoad(char*, void*); END_TBL_ENTRY endDevTbl [] = { { 0, DEC_LOAD_FUNC, DEC_LOAD_STRING, DEC_BUFF_LOAN, NULL, FALSE}, { 1, geLoad, "0", 1, NULL, TRUE}, { 0, END_TBL_END, NULL, 0, NULL, FALSE}, }; VxWorks uses the device unit number as an index, and these unit numbers have to be unique, even among different drivers. GM uses unit numbers to index different myrinet cards. There is no relationship between vxWorks unit numbers and GM unit numbers. You specify them in two different positions in the endDevTbl entry, below. Notice that the GM unit number is given as a string. ____________ TRUE = don't autoload. | FALSE = autoload. { 1, geLoad, "0", 1, NULL, TRUE}, | | | | | |______ GM unit number. | | |__________ vxWorks unit number (twice). You can have the GM SENS driver autoload or you can load it manually in your vxWorks startup script. THE AUTOLOAD FEATURE INVARIABLY CAUSES VXWORKS TO CRASH, SO MYRICOM RECOMMENDS THAT YOU DON'T USE IT. Finally you must tell vxWorks to allow more than one IP device. #define IP_MAX_UNITS 2 3.3 Makefile Changes You must edit the makefile in your BSP directory to add the GM driver and optionally GM tools to your vxWorks image. Add the following lines to the Makefile, and change the value of GM_BASE to fit your file system. You may also have to edit the MACH_EXTRA definitions to match your BSP or version of vxWorks. For instance, below, the MACH_EXTRA object files sysSpeed.o pcf8584i2c.o muxLib.o are vxWorks files and may be different in your build environment. GM_BASE = /ufs/wind/finucane/gm MACH_EXTRA = sysSpeed.o pcf8584i2c.o muxLib.o \ $(GM_BASE)/binary/sbin/gm If you wanted to include the GM mapper into the vxWorks image, you would instead use: MACH_EXTRA = sysSpeed.o pcf8584i2c.o muxLib.o \ $(GM_BASE)/binary/sbin/gm \ $(GM_BASE)/binary/sbin/mapper You may have problems if you try to include the mapper. It may cause vxsize to fail in the last step of the vxWorks build because of an image size restriction. You might work around this by recompiling the mapper objects in a space-conserving manner. Open gm/drivers/vxworks/make-os and remove -DINSIST and -g from the line CPPFLAGS += -g -DGM_OS_VXWORKS=1 -DVXWORKS=1 -DMT_ppc_vxWorks \ -DINSIST -fsigned-char There may be other ways of reducing the size of GM in the vxWorks image. For instance, recompiling the gm driver with fewer mcp images built in. 3.4 vxWorks startup script changes Here is an example startup script for the GM SENS driver and vxWorks: --------------------------------------------------------------- #change to fit your file system. cd "/ufs/wind/finucane/gm/binary/sbin" #initialize GM driver gmVxWorksInit #optional: cd "../sbin" ld < mapper cplusCtors "mapper" sp mapper, "active.args" #load and configure GM SENS driver. #change to suit your world. muxdev = muxDevLoad (1, geLoad, "0", 0, 0) muxDevStart muxdev ipAttach 1, "myri" ifAddrSet "myri1", "199.201.130.151" hostAdd "bigbox-t","199.201.130.100" hostAdd "beta-t","199.201.130.23" ifShow #optional: load and call gm_board_info #cd "../bin" #ld < gm_board_info #gm_board_info --------------------------------------------------------------- NOTES on the startup script 3.4.1 In running the GM mapper on vxWorks, you might want to modify the "active.args" file. First comment out the line -verbose. (semicolon and pound signs are comment chars.) This will disable the printing of diagnostic messages. If you are running the mapper in the foreground, you will also have to make it terminate at some time. Some combination of -map-once and -expected-nodes should be used in the modifed "active.args" file to acheive this. -map-once tells the mapper to configure the network and then terminate. -expected-nodes, when used with -map-once, tells the mapper to terminate when a certain number of nodes has been seen. As with any myrinet, only one mapper should be running at a time. If there is more than one mapper running, only one should be started with "active.args" All other mappers should be started with "passive.args". 3.4.2 In muxDevLoad, the first argument is the vxWorks unit number, and the third is a string containing the GM unit number. In muxDevStart the array index is the GM unit number. In ipAttach the first argument is the vxWorks unit number. In ifAddrSet the name is of the form myrid where "d" is the vxWorks unit number. See the discussion on endDevTbl above for more information on unit numbers.