I_MPI_ADJUST Family

I_MPI_ADJUST_<opname>

Control collective operation algorithm selection.

Syntax

I_MPI_ADJUST_<opname>="<algid>[:<conditions>][;<algid>:<conditions>[...]]"

Arguments

<algid>

Algorithm identifier

>= 0

The default value of zero selects the reasonable settings

 

<conditions>

A comma separated list of conditions. An empty list selects all message sizes and process combinations

<l>

Messages of size <l>

<l>-<m>

Messages of size from <l> to <m>, inclusive

<l>@<p>

Messages of size <l> and number of processes <p>

<l>-<m>@<p>-<q>

Messages of size from <l> to <m> and number of processes from <p> to <q>, inclusive

Description

Set this environment variable to select the desired algorithm(s) for the collective operation <opname> under particular conditions. Each collective operation has its own environment variable and algorithms.

Table 3.4-1 Environment Variables, Collective Operations, and Algorithms

Environment Variable

Collective Operation

Algorithms

I_MPI_ADJUST_ALLGATHER

MPI_Allgather

  1. Recursive doubling
  2. Bruck's
  3. Ring
  4. Topology aware Gatherv + Bcast
  5. Knomial

I_MPI_ADJUST_ALLGATHERV

MPI_Allgatherv

  1. Recursive doubling
  2. Bruck's
  3. Ring
  4. Topology aware Gatherv + Bcast

I_MPI_ADJUST_ALLREDUCE

MPI_Allreduce

  1. Recursive doubling
  2. Rabenseifner's
  3. Reduce + Bcast
  4. Topology aware Reduce + Bcast
  5. Binomial gather + scatter
  6. Topology aware binominal gather + scatter
  7. Shumilin's ring
  8. Ring
  9. Knomial

I_MPI_ADJUST_ALLTOALL

MPI_Alltoall

  1. Bruck's
  2. Isend/Irecv + waitall
  3. Pair wise exchange
  4. Plum's

I_MPI_ADJUST_ALLTOALLV

MPI_Alltoallv

  1. Isend/Irecv + waitall
  2. Plum's

I_MPI_ADJUST_ALLTOALLW

MPI_Alltoallw

Isend/Irecv + waitall

I_MPI_ADJUST_BARRIER

MPI_Barrier

  1. Dissemination
  2. Recursive doubling
  3. Topology aware dissemination
  4. Topology aware recursive doubling
  5. Binominal gather + scatter
  6. Topology aware binominal gather + scatter

I_MPI_ADJUST_BCAST

MPI_Bcast

  1. Binomial
  2. Recursive doubling
  3. Ring
  4. Topology aware binomial
  5. Topology aware recursive doubling
  6. Topology aware ring
  7. Shumilin's
  8. Knomial

I_MPI_ADJUST_EXSCAN

MPI_Exscan

  1. Partial results gathering
  2. Partial results gathering regarding layout of processes

I_MPI_ADJUST_GATHER

MPI_Gather

  1. Binomial
  2. Topology aware binomial
  3. Shumilin's

I_MPI_ADJUST_GATHERV

MPI_Gatherv

  1. Linear
  2. Topology aware linear
  3. Knomial

I_MPI_ADJUST_REDUCE_SCATTER

MPI_Reduce_scatter

  1. Recursive halving
  2. Pair wise exchange
  3. Recursive doubling
  4. Reduce + Scatterv
  5. Topology aware Reduce + Scatterv

I_MPI_ADJUST_REDUCE

MPI_Reduce

  1. Shumilin's
  2. Binomial
  3. Topology aware Shumilin's
  4. Topology aware binomial
  5. Rabenseifner's
  6. Topology aware Rabenseifner's
  7. Knomial

I_MPI_ADJUST_SCAN

MPI_Scan

  1. Partial results gathering
  2. Topology aware partial results gathering

I_MPI_ADJUST_SCATTER

MPI_Scatter

  1. Binomial
  2. Topology aware binomial
  3. Shumilin's

I_MPI_ADJUST_SCATTERV

MPI_Scatterv

  1. Linear
  2. Topology aware linear

I_MPI_ADJUST_IALLGATHER

MPI_Iallgather

  1. Recursive doubling
  2. Bruck’s
  3. Ring

I_MPI_ADJUST_IALLGATHERV

MPI_Iallgatherv

  1. Recursive doubling
  2. Bruck’s
  3. Ring

I_MPI_ADJUST_IALLREDUCE

MPI_Iallreduce

  1. Recursive doubling
  2. Rabenseifner’s
  3. Reduce + Bcast
  4. Ring (patarasuk)
  5. Knomial
  6. Binomial

I_MPI_ADJUST_IALLTOALL

MPI_Ialltoall

  1. Bruck’s
  2. Isend/Irecv + Waitall
  3. Pairwise exchange

I_MPI_ADJUST_IALLTOALLV

MPI_Ialltoallv

Isend/Irecv + Waitall

I_MPI_ADJUST_IALLTOALLW

MPI_Ialltoallw

Isend/Irecv + Waitall

I_MPI_ADJUST_IBARRIER

MPI_Ibarrier

Dissemination

I_MPI_ADJUST_IBCAST

MPI_Ibcast

  1. Binomial
  2. Recursive doubling
  3. Ring
  4. Knomial

I_MPI_ADJUST_IEXSCAN

MPI_Iexscan

Recursive doubling

I_MPI_ADJUST_IGATHER

MPI_Igather

  1. Binomial
  2. Knomial

I_MPI_ADJUST_IGATHERV

MPI_Igatherv

Linear

I_MPI_ADJUST_IREDUCE_SCATTER

MPI_Ireduce_scatter

  1. Recursive halving
  2. Pairwise
  3. Recursive doubling

I_MPI_ADJUST_IREDUCE

MPI_Ireduce

  1. Rabenseifner’s
  2. Binomial
  3. Knomial

I_MPI_ADJUST_ISCAN

MPI_Iscan

Recursive Doubling

I_MPI_ADJUST_ISCATTER

MPI_Iscatter

  1. Binomial
  2. Knomial

I_MPI_ADJUST_ISCATTERV

MPI_Iscatterv

Linear

The message size calculation rules for the collective operations are described in the table. In the following table, "n/a" means that the corresponding interval <l>-<m> should be omitted.

Table 3.4-2 Message Collective Functions

Collective Function

Message Size Formula

MPI_Allgather

recv_count*recv_type_size

MPI_Allgatherv

total_recv_count*recv_type_size

MPI_Allreduce

count*type_size

MPI_Alltoall

send_count*send_type_size

MPI_Alltoallv

n/a

MPI_Alltoallw

n/a

MPI_Barrier

n/a

MPI_Bcast

count*type_size

MPI_Exscan

count*type_size

MPI_Gather

recv_count*recv_type_size if MPI_IN_PLACE is used, otherwise send_count*send_type_size

MPI_Gatherv

n/a

MPI_Reduce_scatter

total_recv_count*type_size

MPI_Reduce

count*type_size

MPI_Scan

count*type_size

MPI_Scatter

send_count*send_type_size if MPI_IN_PLACE is used, otherwise recv_count*recv_type_size

MPI_Scatterv

n/a

Examples

Use the following settings to select the second algorithm for MPI_Reduce operation:
I_MPI_ADJUST_REDUCE=2

Use the following settings to define the algorithms for MPI_Reduce_scatter operation:
I_MPI_ADJUST_REDUCE_SCATTER="4:0-100,5001-10000;1:101-3200,2:3201-5000;3"

In this case. algorithm 4 is used for the message sizes between 0 and 100 bytes and from 5001 and 10000 bytes, algorithm 1 is used for the message sizes between 101 and 3200 bytes, algorithm 2 is used for the message sizes between 3201 and 5000 bytes, and algorithm 3 is used for all other messages.

I_MPI_ADJUST_REDUCE_SEGMENT

Syntax

I_MPI_ADJUST_REDUCE_SEGMENT=<block_size>|<algid>:<block_size>[,<algid>:<block_size>[...]]

Arguments

<algid>

Algorithm identifier

1

Shumilin’s algorithm

3

Topology aware Shumilin’s algorithm

<block_size>

Size of a message segment in bytes

> 0

The default value is 14000

Description

Set an internal block size to control MPI_Reduce message segmentation for the specified algorithm. If the <algid> value is not set, the <block_size> value is applied for all the algorithms, where it is relevant.

Note

This environment variable is relevant for Shumilin’s and topology aware Shumilin’s algorithms only (algorithm N1 and algorithm N3 correspondingly).

I_MPI_ADJUST_BCAST_SEGMENT

Syntax

I_MPI_ADJUST_BCAST_SEGMENT=<block_size>|<algid>:<block_size>[,<algid>:<block_size>[...]]

Arguments

<algid>

Algorithm identifier

1

Binomial

4

Topology aware binomial

7

Shumilin's

8

Knomial

<block_size>

Size of a message segment in bytes

> 0

The default value is 12288

Description

Set an internal block size to control MPI_Bcast message segmentation for the specified algorithm. If the <algid> value is not set, the <block_size> value is applied for all the algorithms, where it is relevant.

Note

This environment variable is relevant only for Binomial, Topology-aware binomial, Shumilin’s and Knomial algorithms.

I_MPI_ADJUST_ALLGATHER_KN_RADIX

Syntax

I_MPI_ADJUST_ALLGATHER_KN_RADIX=<radix>

Arguments

<radix>

An integer that specifies a radix used by the Knomial MPI_Allgather algorithm to build a knomial communication tree

> 1

The default value is 2

Description

Set this environment variable together with I_MPI_ADJUST_ALLGATHER=5 to select the knomial tree radix for the corresponding MPI_Allgather algorithm.

I_MPI_ADJUST_BCAST_KN_RADIX

Syntax

I_MPI_ADJUST_BCAST_KN_RADIX=<radix>

Arguments

<radix>

An integer that specifies a radix used by the Knomial MPI_Bcast algorithm to build a knomial communication tree

> 1

The default value is 4

Description

Set this environment variable together with I_MPI_ADJUST_BCAST=8 to select the knomial tree radix for the corresponding MPI_Bcast algorithm.

I_MPI_ADJUST_ALLREDUCE_KN_RADIX

Syntax

I_MPI_ADJUST_ALLREDUCE_KN_RADIX=<radix>

Arguments

<radix>

An integer that specifies a radix used by the Knomial MPI_Allreduce algorithm to build a knomial communication tree

> 1

The default value is 4

Description

Set this environment variable together with I_MPI_ADJUST_ALLREDUCE=9 to select the knomial tree radix for the corresponding MPI_Allreduce algorithm.

I_MPI_ADJUST_REDUCE_KN_RADIX

Syntax

I_MPI_ADJUST_REDUCE_KN_RADIX=<radix>

Arguments

<radix>

An integer that specifies a radix used by the Knomial MPI_Reduce algorithm to build a knomial communication tree

> 1

The default value is 4

Description

Set this environment variable together with I_MPI_ADJUST_REDUCE=7 to select the knomial tree radix for the corresponding MPI_Reduce algorithm.

I_MPI_ADJUST_GATHERV_KN_RADIX

Syntax

I_MPI_ADJUST_GATHERV_KN_RADIX=<radix>

Arguments

<radix>

An integer that specifies a radix used by the Knomial MPI_Gatherv algorithm to build a knomial communication tree

> 1

The default value is 2

Description

Set this environment variable together with I_MPI_ADJUST_GATHERV=3 to select the knomial tree radix for the corresponding MPI_Gatherv algorithm.

I_MPI_ADJUST_IALLREDUCE_KN_RADIX

Syntax

I_MPI_ADJUST_IALLREDUCE_KN_RADIX=<radix>

Arguments

<radix> 

An integer that specifies a radix used by the Knomial MPI_Iallreduce algorithm to build a knomial communication tree

> 1

The default value is 4

Description

Set this environment variable together with I_MPI_ADJUST_IALLREDUCE=5 to select the knomial tree radix for the corresponding MPI_Iallreduce algorithm.

I_MPI_ADJUST_IBCAST_KN_RADIX

Syntax

I_MPI_ADJUST_IBCAST_KN_RADIX=<radix>

Arguments

<radix> 

An integer that specifies a radix used by the Knomial MPI_Ibcast algorithm to build a knomial communication tree

> 1

The default value is 4

Description

Set this environment variable together with I_MPI_ADJUST_IBCAST=4 to select the knomial tree radix for the corresponding MPI_Ibcast algorithm.

I_MPI_ADJUST_IREDUCE_KN_RADIX

Syntax

I_MPI_ADJUST_IREDUCE_KN_RADIX=<radix>

Arguments

<radix> 

An integer that specifies a radix used by the Knomial MPI_Ireduce algorithm to build a knomial communication tree

> 1

The default value is 4

Description

Set this environment variable together with I_MPI_ADJUST_IREDUCE=3 to select the knomial tree radix for the corresponding MPI_Ireduce algorithm.

I_MPI_ADJUST_IGATHER_KN_RADIX

Syntax

I_MPI_ADJUST_IGATHER_KN_RADIX=<radix>

Arguments

<radix> 

An integer that specifies a radix used by the Knomial MPI_Igather algorithm to build a knomial communication tree

> 1

The default value is 4

Description

Set this environment variable together with I_MPI_ADJUST_IGATHER=2 to select the knomial tree radix for the corresponding MPI_Igather algorithm.

I_MPI_ADJUST_ISCATTER_KN_RADIX

Syntax

I_MPI_ADJUST_ISCATTER_KN_RADIX=<radix>

Arguments

<radix> 

An integer that specifies a radix used by the Knomial MPI_Iscatter algorithm to build a knomial communication tree

> 1

The default value is 4

Description

Set this environment variable together with I_MPI_ADJUST_ISCATTER=2 to select the knomial tree radix for the corresponding MPI_Iscatter algorithm.