Effective Running of End-to-end HPC Workflows on Emerging Heterogeneous Architectures

In high-performance computing (HPC), end-to-end workflows are typically utilized to gain insights from scientific simulations. An end-to-end workflow consists of scientific simulation and data analysis, and can be executed in-situ, in-transit, and offline. Existing studies on end-to-end workflows have largely focused on the high-performance execution approaches. However, the emerging heterogeneous architectures and energy concerns lead to the rethinking of workflow execution approaches. As a guide to the rethinking, this paper evaluates how to run end-to-end HPC workflows efficiently in terms of performance, energy, and error resilience. The evaluation covers emerging heterogeneous processor architectures, processor power capping techniques, and heterogeneous-reliability memory.