TotalView* Debugger

Intel® MPI Library supports the use of the TotalView* debugger from Rogue Wave* Software, Inc. To debug an MPI program, add -tv to the global mpirun arguments, as in

$ mpirun -tv -n 4 ./testc

Note

In case of ssh communication, you need to set the TVDSVRLAUNCHCMD environment variable to ssh.

You will get a popup window from TotalView asking whether you want to start the job in a stopped state. If so, when the TotalView window appears, you may see assembly code in the source window. Click on main() in the stack window (upper left) to see the source of the main function. TotalView shows that the program (all processes) are stopped in the call to MPI_Init(). When debugging with TotalView using the above startup sequence, you need to exit TotalView before restarting an Intel MPI Library job.

To debug with TotalView an enable restarting the session, use the following command line:

$ totalview python -a ‘which mpirun‘ -tvsu <mpirun_args> <prog> <prog_args>

If you have TotalView 8.1.0 or later, you can use a feature called indirect launch.

  1. Invoke TotalView as:

    $ totalview <prog> -a <prog_args>

  2. Select the Process/Startup Parameters command.

  3. Choose the Parallel tab in the resulting dialog box and choose MPICH2 as the parallel system.

  4. Set the number of tasks using the Tasks field.

  5. Enter other needed mpirun arguments into the Additional Starter Arguments field.

If you want to be able to attach to a running MPI job using TotalView, you must use the -tvsu option in the mpirun command when starting the job. Using this option adds a barrier inside MPI_Init() and hence may affect startup performance slightly. After all tasks have returned from MPI_Init(), there is no performance degradation incurred from this option.