MPI is a library used to send messages between processes in a distributed memory model.
These messages can contain all types of data such as string or integers but actual objects also.
It's not a programming language, it's a programming model that is mostly used for parallel programming in a cluster.
In this cluster, there is lot of nodes, but only one is the "chief", also known as master. Others are just workers nodes.
The primary advantage of using MPI is that it allow programmers to split a task and to distribute it to all or some specific workers nodes and allow each node to work on its own task simultaneously.
The Communicator is primordial to MPI projects, it allows to different processes to communicate with eachother. Of course, there can be several communicators in one MPI program, they will have theirs own processes. It can be useful if you want to split processes such that only certain processes receive messages to and from each other.
Actually, MPI allows two major types of communication :
Point to Point
Collectives
It's the most basic type of communication, which is when processes communicate on one to one basis. For example, P0 talks to P1 and P1 talks to P0.
Also, it can be blocking or unblocking. Obviously, blocking state means that the process will not continue until it has received its message. Otherwise, unblocking state means that it can keep executing steps while waiting for its message.
Point to Point work on two functions : Send and Receive.
Send function can work on multiple types like : Primitives Types, Serializable Classes and Structures. But sending Serializable Classes can affect performance and sending will be slower than sending primitives type or structures.
Receive function : When you receive a primitive type you must match it directly but not with classes.
For example, you can receive a poodle as a dog.
The other type of communication is Collective, all processes can communicate with each other.
It's a better type of communication than Point to Point because it would get messy pretty quickly.
There are two primary advantages of using Collective Communication :
Its easier for programmers to read and maintain code
Faster than Point to Point communications.
For example, if you want to send a message to every processes it will use only one simple call, whereas it would require N^2 Point to Point communications, so it would consumes a lot of resources.
There are the majors communication's way that MPI implements :
It just blocks all processes in a point, like a real barrier, and this barrier will open when all processes would be at that point.
It's when one process is asking all the other processes.
It's when one process is sending a message to all the other processes.
It's when all processes are sending messages to each other.
It is used to combine data from all other processes and return it to the process who requested it.
MPI is used, for example, in Parallel Computing and Monte Carlo Simulations.
