Fault-Aware Group-Collective Communication Creation and Repair in MPI

The increasing size of HPC systems indicates that executions involve more nodes and processes, making the faults' presence a more frequent eventuality. This issue becomes especially relevant since MPI, the de-facto standard for inter-process communication, lacks proper fault management functionalities. Past efforts produced extensions to the MPI standard enabling fault management, including ULFM. While providing powerful tools to handle faults, ULFM still faces limitations like the collectiveness of the repair procedure. With this paper, we overcome those limitations and achieve fault-aware group-collective communicator creation and repair. We integrate our solution into an existing fault-resiliency framework and measure the overhead in the application code. The experimental campaign shows that our solution is scalable and introduces a limited overhead, and the group-collective repair is a viable opportunity for ULFM-based applications.