IntelĀ® MPI Library Reference Manual for Linux* OS
In the following example, an application is started and then an explicit signal (SIGUSR1) is passed to the application to take a checkpoint. The checkpoints are stored in a directory called ckptdir.
user@head $ mpiexec.hydra -ckpoint on -ckpoint-prefix /home/user/ckptdir -ckpointlib blcr -n 32 -f hosts /home/user/myapp
...
user@head $ kill -s SIGUSR1 <PID of mpiexec.hydra>
It is necessary and sufficient for you to signal the mpiexec.hydra process on node head.