Performance Models for Communication in Collective I/O Operations

Many large scale scientific applications spend a significant amount of time in file I/O operations. Collective I/O APIs provide higher level abstractions of I/O across a group of processes. They often reduce the time spent in file I/O by reorganizing data across processes to match the layout of the data on the file system. In this paper we present performance models for the communication occurring in collective write operations as a first step towards developing a full and accurate mode of collective I/O operations. The models derived in this paper take both the application data decomposition and the file domain partitioning strategies used by the I/O library into account. We discuss properties of our performance models and demonstrate using LogGP parameters derived on multiple platforms their impact on the performance of collective I/O operations. The paper further provides comparison to actual measurements performed on an InfiniBand cluster. Our results indicate a good overall match between predicted and observed behavior.