The Flexible Preconditions Model for Macro-Dataflow Execution

In this paper, we propose the flexible preconditions model for macro-dataflow execution. Our approach unifies two current approaches for managing task dependences, eager execution vs. strict preconditions. When one of the two outperforms the other, flexible preconditions can always attain, and possibly surpass, the performance of the better approach. This work focuses on the performance of parallel programming models based on macro-dataflow, in which applications are composed of tasks and inter-task dependences. Data-flow models usually make a choice between specifying the task dependences before task creation (as strict preconditions), or during task execution, when they are actually needed (eager execution). This paper shows how the choice between eager execution and strict preconditions affects the performance, memory consumption and expressiveness of macro-dataflow applications. The flexible preconditions model is sufficiently flexible to support both eager execution and strict preconditions, as well as hybrid combinations thereof. This capability enables programmers and future auto-tuning systems to pick the precondition combination that yields the best performance for a given application. The experimental evaluation was performed on a 32-core SMP, and is based on a new macro-dataflow implementation, QtCnC, that supports eager execution, strict preconditions and flexible preconditions in a single framework. (QtCnC is an implementation of the CnC model on the QThreads library.) For applications where all dependences are known ahead of time, flexible and strict preconditions execute up to 56% faster than eager execution (for the benchmarks and platform used in our study). On the other hand, for applications where the complete set of per-task dependences is determined after the tasks are spawned, flexible preconditions and eager execution perform up to 38% better than strict preconditions.