Learning Optimal Decision Trees Under Memory Constraints

Existing algorithms for learning optimal decision trees can be put into two categories: algorithms based on the use of Mixed Integer Programming (MIP) solvers and algorithms based on dynamic programming (DP) on itemsets. While the algorithms based on DP are the fastest, their main disadvantage compared to MIP-based approaches is that the amount of memory these algorithms may require to find an optimal solution is not bounded. Consequently, for some datasets these algorithms can only be executed on machines with large amounts of memory. In this paper, we propose the first DP-based algorithm for learning optimal decision trees that operates under memory constraints. Core contributions of this work include: (1) strategies for freeing memory when too much memory is used by the algorithm; (2) an effective approach for recovering the optimal decision tree when parts of the memory are freed. Our experiments demonstrate a favorable trade-off between memory constraints and the run times of our algorithm.