c++ - MPI_Comm_split explanation

Question

Can someone explain and tell me more about MPI_Comm_split communicator?

MPI_Comm_split(MPI_COMM_WORLD, my_row, my_rank,&my_row_comm);

This is just example i met by reading some basic documentations. Maybe someone could tell me how this communicator is working?

question from:https://stackoverflow.com/questions/65661524/how-is-a-new-communicator-created-in-mpi

Answer 1

Just to begin with, let's have a look at the man page:

MPI_Comm_split(3)                     MPI                    MPI_Comm_split(3)

NAME
       MPI_Comm_split -  Creates new communicators based on colors and keys

SYNOPSIS
       int MPI_Comm_split(MPI_Comm comm, int color, int key, MPI_Comm *newcomm)

INPUT PARAMETERS
       comm   - communicator (handle)
       color  - control of subset assignment (nonnegative integer).  Processes
              with the same color are in the same new communicator
       key    - control of rank assignment (integer)

OUTPUT PARAMETERS
       newcomm
              - new communicator (handle)

So what does that do?
Well, as the name suggests, it will split the communicator comm into disjoint sub-communicators newcomm. Each process of comm will be into one unique of these sub-communicators, hence the fact that the output newcomm is only one single communicator (for the current process). However, globally speaking, you have to understand that the many versions of newcomm are different sub-communicators, partitioning the input comm.

So that is what the function does. But how does it do it?
Well, that's where the two parameters color and key come into play:

• color is an integer value that permits to decide in which of the sub-communicators the current process will fall. More specifically, all processes of comm for which color will have the same numerical value will be part of the same sub-communicator newcomm. For example, if you were to define color = rank%2; (with rank the rank of the process in comm), then you would create (globally) two new communicators: one for the processes of odd ranks, and one for the processes of even ranks. However, keep in mind that each processes will only be seeing the one of these new communicators they are part of... So in summary, color permits to tell apart the various "teams" you will create, like the colour of the jersey football teams will wear to distinguish themselves during a match (hence the naming I presume).
• key will just permit to optionally decide how the processes will be ranked into the new communicators they are part of. For example, if you set key = rank;, then the order of ranking (not the ranking itself) in each new communicators newcomm will follow the order of ranking in the original communicator comm. But if you don't care about the ordering, you can as well set key=0; and the ranking in each of the new communicators will be whatever the library decides...

Finally, two trivial examples:

• MPI_Comm_split(comm, 0, rank, &newcomm) will just duplicate comm into newcomm (just as MPI_Comm_dup())
• MPI_Comm_split(comm, rank, rank, &newcomm) will just return an equivalent of MPI_COMM_SELF for each of the processes