OFA*-capable Network Fabrics Control

I_MPI_OFA_NUM_ADAPTERS

Set the number of connection adapters.

Syntax

I_MPI_OFA_NUM_ADAPTERS=<arg>

Arguments

<arg>

Define the maximum number of connection adapters used

>0

Use the specified number of adapters. The default value is 1

Description

Set the number of the adapters that are used. If the number is greater than the available number of adapters, all the available adaptors are used.

I_MPI_OFA_ADAPTER_NAME

Set the name of adapter that is used.

Syntax

I_MPI_OFA_ADAPTER_NAME=<arg>

Arguments

<arg>

Define the name of adapter

Name

Use the specified adapter. By default, any adapter can be used

Description

Set the name of adaptor to be used. If the adapter with specified name does not exist, the library returns an error. This has effect only if I_MPI_OFA_NUM_ADAPTERS=1.

I_MPI_OFA_NUM_PORTS

Set the number of used ports on each adapter.

Syntax

I_MPI_OFA_NUM_PORTS=<arg>

Arguments

<arg>

Define the number of ports that are used on each adapter

> 0

Use the specified number of ports. The default value is 1

Description

Set the number of used ports on each adaptor. If the number is greater than the available number of ports, all the available ports are used.

I_MPI_OFA_NUM_RDMA_CONNECTIONS

Set the maximum number of connections that can use the rdma exchange protocol.

Syntax

I_MPI_OFA_NUM_RDMA_CONNECTIONS=<num_conn>

Arguments

<num_conn>

Define the maximum number of connections that can use the rdma exchange protocol

>= 0

Create the specified number of connections that use the rdma exchange protocol. All other processes use the send/ receive exchange protocol

-1

Create log2(number of processes) rdma connections

>= number of processes

Create rdma connections for all processes. This is the default value

Description

There are two exchange protocols between two processes: send/receive and rdma. This environment variable specifies the maximum amount of connections that use rdma protocol.

RDMA protocol is faster but requires more resources. For a large application, you can limit the number of connections that use the rdma protocol so that only processes that actively exchange data use the rdma protocol.

I_MPI_OFA_SWITCHING_TO_RDMA

Set the number of messages that a process should receive before switching this connection to RDMA exchange protocol.

Syntax

I_MPI_OFA_SWITCHING_TO_RDMA=<number>

Arguments

<number>

Define the number of messages that the process receives before switching to use the rdma protocol

>= 0

If this process receives <number> of messages, start using the rdma protocol

Description

Count the number of messages received from the specific process. The connection achieved the specified number tries to switch to rdma protocol for exchanging with that process. The connection will not switch to rdma protocol if the maximum number of connections that use the rdma exchange protocol defined in I_MPI_OFA_NUM_RDMA_CONNECTIONS has been reached.

I_MPI_OFA_RAIL_SCHEDULER

Set the method of choosing rails for short messages.

Syntax

I_MPI_OFA_RAIL_SCHEDULER=<arg>

Arguments

<arg>

Mode selector

ROUND_ROBIN

Next time use next rail

FIRST_RAIL

Always use the first rail for short messages

PROCESS_BIND

Always use the rail specific for process

Description

Set the method of choosing rails for short messages. The algorithms are selected according to the following scheme:

I_MPI_OFA_TRANSLATION_CACHE

Turn on/off the memory registration cache.

Syntax

I_MPI_OFA_TRANSLATION_CACHE=<arg>

Arguments

<arg>

Binary indicator

enable | yes | on | 1

Turn on the memory registration cache. This is the default

disable | no | off | 0

Turn off the memory registration cache

Description

Set this environment variable to turn on/off the memory registration cache.

The cache substantially increases performance, but may lead to correctness issues in certain situations. See product Release Notes for further details.

I_MPI_OFA_TRANSLATION_CACHE_AVL_TREE

Enable/disable the AVL tree* based implementation of the RDMA translation cache.

Syntax

I_MPI_OFA_TRANSLATION_CACHE_AVL_TREE=<arg>

Arguments

<arg>

Binary indicator

enable | yes | on | 1

Turn on the AVL tree based RDMA translation cache

disable | no | off | 0

Turn off the AVL tree based RDMA translation cache. This is the default value

Description

Set this environment variable to enable the AVL tree based implementation of RDMA translation cache in the OFA path. When the search in RDMA translation cache handles over 10,000 elements, the AVL tree based RDMA translation cache is faster than the default implementation.

I_MPI_OFA_DYNAMIC_QPS

Control the library to create queue pairs (QPs) dynamically.

Syntax

I_MPI_OFA_DYNAMIC_QPS=<arg>

Arguments

<arg>

Binary indicator

enable | yes | on | 1

Create QPs dynamically. This is the default value if the number of processes is larger than or equal 2,000

disable | no | off | 0

Create all QPs during the initial stage. This is the default value if the number of processes is less than 2,000

Description

Set this environment variable to turn on dynamic creation of QPs.

I_MPI_OFA_PACKET_SIZE

Set the size of the packet used for sending.

Syntax

I_MPI_OFA_PACKET_SIZE=<arg>

Arguments

<arg>

Define the size of packet in bytes

> 0

Use the specified packet size. The default value is 8192

Description

Set the packet size in bytes. If the number is negative, the size is set to 8.

I_MPI_OFA_LIBRARY

Set the name of the used OFA library.

Syntax

I_MPI_OFA_LIBRARY=<arg>

Arguments

<arg>

Define the name of the OFA library

Name

Use the specified library. By default, the name is libibverbs.so

Description

Set the name of the InfiniBand* (IB*) library. If the library with the specified name does not exist, an error is returned.

I_MPI_OFA_NONSWITCH_CONF

Define the nonstandard template for port connections.

Syntax

I_MPI_OFA_NONSWITCH_CONF=<arg>

Arguments

<arg>

Define the template for port connections

Name

Use the specified template

Description

The nodes in clusters are normally connected so that porti of a node can access porti of all other nodes. Use this environment variable if ports are not connected in this way. The following example is the template format:

host1@port11#port12#...#host2@port21#port22....

Portij defines the port used to send from hosti to hostj

For example:

node1@1#1#2#node2@2#1#1#node3@1#2#1#

This sample specifies the following configuration:

Port1 is always used to communicate with itself (loopback).