Distributed Deep Learning Optimization of Heat Equation Inverse Problem Solvers

The inversion problem of partial differential equation plays a crucial role in cyber-physical systems applications. This article presents a novel deep learning optimization approach to constructing a solver of heat equation inversion. To improve the computational efficiency in large-scale industrial applications, data and model parallelisms are incorporated on a platform of multiple GPUs. The advanced Ring-AllReduce architecture is harnessed to achieve an acceleration ratio of 3.46. Then, a new multi-GPUs distributed optimization method GradReduce is proposed based on Ring-AllReduce architecture. This method optimizes the original data communication mechanism based on mechanical time and frequency by introducing the gradient transmission scheme solved by linear programming. The experimental results show that the proposed method can achieve an acceleration ratio of 3.84 on a heterogeneous system platform with two CPUs and four GPUs.