Some additional instructions to build MCNPX 2.5.F with MPI.

Unix Systems
------------

(1) Download and install MPI. I suggest getting MPICH from ANL
    (http://www-unix.mcs.anl.gov/mpi/mpich/). Follow the instructions
    that come with the distribution. Since MCNPX is an F90 code,
    care must be taken to generate the F90 MPI libraries (as I
    recall F77 was the default). This is done with the -f90 flag on
    the configure command. The following example is for using the
    Portland Group compiler:

    cd mpich-1.2.5
    ./configure -cc=pgcc -fc=pgf90 -f90=pgf90

    of if you don't have the Portland Group C compiler:

    ./configure -cc=gcc -fc=pgf90 -f90=pgf90 -lib="-L/usr/local/pgi/linux86/lib -lpgf90 -lpgf90rtl -lpgftnrtl -lpgc ..."

    gmake

    Note the "..." should be replaced with other libraries that may be
    in the Portland Group library directory (the -lib option solves
    a problem that may be eliminated in future versions of MPICH).The
    MPICH libraries are put in the /lib subdirectory.

(2) Build the MPI version of MCNPX. The following steps will
    accomplish this:

    (a) Set the MPI environment variables:

    setenv MPI_ROOT ~/mpich-1.2.5
    setenv MPI_NAME mpich

    (b) Create a build directory, use the MPILIB configure option, and
        use the MPICH configure option (if using MPICH). If you are
        creating an 8-byte integer version, you may need to copy the
        mpif.h file into src/include and edit it to replace "integer"
        with "integer*4" (8-byte integer option compiles with "-i8"
        which sets the default integer size to 8 bytes; however MPI
        calls must provide 4-byte integers - you will get a linking
        error for various MPI routines if this is a problem).

    mkdir MCNPX_BLD
    cd MCNPX_BLD
    ../v25f/configure --with-FC=pgf90 --with-CC=pgcc --with-MPILIB
                      --with-MPICH

    (c) Run gmake to complete the MCNPX build.

(3) To run the test suite sequentially with this executable,
    you must edit the file MCNPX_BLD/src/Test/Makefile to
    alter the MCNPX variable:

    MCNPX = mpirun ..${/}mcnpx${/}mcnpx

    This assumes you have added ~/mpich-1.2.5/bin to your
    path (so the command "mpirun" can be found). Note, if you are
    anxious to run the test suite in parallel, then you must add
    "-np 4" after "mpirun" and modify the Makefile so test problems
    INP08, INP18, and INP36 are run sequentially (see 2.b.iv below).

(4) We have been able to use native MPI libraries in most
    cases (e.g., SGI, IBM). When linking to native MPI libraries,
    it may not be necessary to set the MPI_ROOT and MPI_NAME
    environment variables as these libraries will automatically
    be found and linked.

Windows Systems
---------------

    An MPI version of MCNPX can also be built on Windows
    platforms. Note that a compiler is not necessary to build
    either MPI or MCNPX. The following steps guide the user
    through this process:

(1) Download and install MPI. I suggest getting MPICH from ANL
    (http://www-unix.mcs.anl.gov/mpi/mpich/). The download for
    a Windows distribution does not require building the MPI
    libraries (they are provided for CVF). A typical install will
    put MPICH in

    C:\Program Files\MPICH

    Make sure you are the "Administrator" when performing this
    install. I believe it is necessary to use this default
    installation directory so required DLL files can be found
    (otherwise you must move these files into some other directory
    included in your PATH).

(2) There are two options for obtaining an MPICH version of
    MCNPX: (a) get the executable from our web site, or (b)
    build one.

    (a) To obtain the executable from our web site, click on
    "The Code", then "Beta Release", then "Version 2.5.f",
    then "Win32.MPI.ZIP" located near the bottom of the page.
    Extract the "mcnpx.exe" file from the "bin" directory and
    put it in your current MCNPX\BIN folder. If you don't have
    an existing download of MCNPX, you will need to get the
    source file (v25f.tar.gz) and follow the instructions in
    the README.w32 file (Section 1.0) first. Skip to step #3.

    (b) The following steps are required to build the MCNPX executable
    with MPI using the Intel compiler:

      (i) Obtain the source file "v25f.tar.gz" from the MCNPX
      web site (click on "The Code", then "Beta Release", then
      "Version 2.5.f"). Uncompress this file using WinZip or some
      similar utility. Typically a source directory will be
      created during this process (e.g., v25f). Create a build
      directory and run "configure":

      mkdir MCNPX_BLD
      cd MCNPX_BLD
      ..\v25f\configure MPI

      At the end of this process, you should be prompted to
      uncompress the file "Win32.exe". Click "UNZIP", then
      "OK", then "CLOSE" to complete the process. If MPICH was
      installed on a drive other than C:, you will need to edit
      the src\mcnpx\Makefile.h file to substitute the correct
      drive in refereces to "c:\progra~1\mpich\sdk\".

      (ii) Run "Make".

      (iii) You can run the test suite with some minor modifications to
      MCNPX_BLD\src\Test\Makefile. Modify the "MCNPX" variable to:

      MCNPX = mpirun ..${/}mcnpx${/}mcnpx

      This assumes you have added C:\Program Files\MPICH\mpd\bin to your
      path (so the command "mpirun" can be found). Note, if you are
      anxious to run the test suite in parallel, then you must use the
      "-np" option with "mpirun" and modify the Makefile so test problems
      INP08, INP18, and INP36 are run sequentially (i.e., with 1 CPU).
      To do this modify the MCNPX variable and add an MCNPX1 variable
      in the src\Test\Makefile:

      MCNPX = mpirun -np 3 -localonly ..${/}mcnpx${/}mcnpx fatal
      MCNPX1 = mpirun ..${/}mcnpx${/}mcnpx

      and make sure target "08s", "18s", and "36s" are run using MCNPX1.
      Note the test suite has been significantly altered from that of
      2.5.C (all of the MCNP5 test problems have been incorporated and
      many of the MCNPX test problems have been modified to provide
      more code coverage). Test problems INP102, INP118, and HTP112 will
      not track when run in parallel as these use the CEM or INCL 
      physics model (this is still on our list to track down).

(3) With MPICH, there are several ways to identify hosts to be used
    in a parallel execution: (a) MPICH Configuration Tool via the
    Start->Programs->MPICH->mpd; (b) using the "-hosts" option of
    MPIRUN; (c) using a configuration file. Here is an example
    of the latter (i.e., using a file called CONFIG):

exe C:\TEMP\MCNPX.EXE
env DATAPATH=C:\MCNPX\DATA
hosts
HOST1 2 C:\GWM\MCNPX\V2.5.F.MPI_BLD\SRC\MCNPX\MCNPX.EXE
HOST2 2 D:\MPI\MCNPX.EXE
HOST3 1

    Note this gives the ability to identify the executable location,
    environment variables, host names, and number of processes
    to run on each host. The master process is alwasy started on
    the first host listed, thus the input file(s) must be located
    on that machine (assuming a shared hard drive is not being used).
    This would be run using the following command:

    mpirun config inp=inp01 name=out1. ...

    Note that the standard MCNPX command-line arguments follow
    the "config" filename (or they can be within the "config"
    file itself - use "mpirun -help" for additional details). When
    using MPI, it is not necessary to use the MCNPX "TASKS" keyword
    which is required for PVM jobs. If you simply want to use 4
    CPUs within the same host, the command line becomes:

    mpirun -np 4 mcnpx inp=inp01 name=out1. ...

    In this case, there will be one master process and 3 subtask
    processes. This actually works even on a single CPU machine,
    however the processes will compete for the CPU and no decrease
    in wall-clock timing will be experienced. See the MPICH
    documentation for further details.