Improve optimization potential for process mapping to hardware topology

[cniethammer]

Problem

The mapping of MPI processes to the physical hardware is crucial for the performance of MPI communication.
Achieving a good mapping is currently left mainly to users via MPI library dependent mpirun arguments. The alternative virtual topology interface provides some potential to pass information about communication paths to the MPI library - and allow reordering of processes. However, the existing interfaces seem to lack at the moment. MPI_Cart_create and the MPI_*Graph interface target this but due to lack of information, e.g., as there might be other communication paths being used or it is unclear how provided edge weights have to be interpreted - implementations did not take action in many cases.

Proposal

As a first step, introduce an info key for MPI_DIstgraph_create which asserts that only P2P communication will be performed only over the specified edges.

Changes to the Text

Add info key "assert_p2p_on_edges_only"

Effect on implementations

TBD

#Effect on users
TBD