Bootstrap Options

-bootstrap <bootstrap server>

Use this option to select a built-in bootstrap server to use. A bootstrap server is the basic remote node access mechanism that is provided by the system. Hydra supports multiple runtime bootstrap servers such as ssh, rsh, pdsh, fork, persist, slurm, ll, lsf, sge, or jmi to launch the MPI processes. The default bootstrap server is ssh. By selecting slurm, ll, lsf, or sge, you use the corresponding srun, llspawn.stdio, blaunch, or qrsh internal job scheduler utility to launch service processes under the respective selected job scheduler (SLURM*, LoadLeveler*, LSF*, and SGE*).

Arguments

<arg>

String parameter

ssh

Use secure shell. This is the default value

rsh

Use remote shell

pdsh

Use parallel distributed shell

pbsdsh

Use Torque* and PBS* pbsdsh command

fork

Use fork call

persist

Use Hydra persist server

slurm

Use SLURM* srun command

ll

Use LoadLeveler* llspawn.stdio command

lsf

Use LSF blaunch command

sge

Use Univa* Grid Engine* qrsh command

jmi

Use Job Manager Interface (tighter integration)

To enable tighter integration with the SLURM* or PBS Pro* job manager, use the jmi bootstrap server. Tighter integration includes registration of the process identifiers by the respective job managers. This configuration enables better resource accounting by the respective job manager, and better node cleanup upon job termination.

Note

Some bootstrap servers that use parallel startup of remote processes (slurm and pdsh) might not work in heterogeneous environment, for example, when I_MPI_MIC is enabled.

See the -bootstrap jmi description and the I_MPI_HYDRA_BOOTSTRAP environment variable for details.

-bootstrap-exec <bootstrap server>

Use this option to set the executable to be used as a bootstrap server. The default bootstrap server is ssh. For example:

$ mpiexec.hydra -bootstrap-exec <bootstrap_server_executable> \

-f hosts.file -env <VAR1> <VAL1> -n 2 ./a.out

See the I_MPI_HYDRA_BOOTSTRAP environment variable for more details.

-bootstrap-exec-args <args>

Use this option to provide the additional parameters to the bootstrap server executable file.

$ mpiexec.hydra -bootstrap-exec-args <arguments> -n 2 ./a.out

See the I_MPI_HYDRA_BOOTSTRAP_EXEC_EXTRA_ARGS environment variable for more details.

-bootstrap persist

Use this option to launch MPI processes using Hydra persist server. Before running a job, start these servers on each host:

$ hydra_persist&

Note

Do not start the services under the root account. A server can be shutdown using the Linux shell kill command.

-bootstrap jmi

Use this option to enable tight integration with the SLURM* or PBS Pro* job schedulers. Tighter integration is implemented using a particular job scheduler application programming interface or utility. If you specify this option, the default libjmi.so library is loaded. You can overwrite the default library name through the I_MPI_HYDRA_JMI_LIBRARY environment variable.

See the I_MPI_HYDRA_JMI_LIBRARY environment variable for more details.