Evolving Deep Forest with Automatic Feature Extraction for Image Classification Using Genetic Programming

Deep forest is an alternative to deep neural networks to use multiple layers of random forests without back-propagation for solving various problems. In this study, we propose a genetic programming-based approach to automatically and simultaneously evolving effective structures of deep forest connections and extracting informative features for image classification. First, in the new approach we define two types of modules: forest modules and feature extraction modules. Second, an encoding strategy is developed to integrate forest modules and feature extraction modules into a tree and the search strategy is introduced to search for the best solution. With these designs, the proposed approach can automatically extract image features and find forests with effective structures simultaneously for image classification. The parameters in the forest can be dynamically determined during the learning process of the new approach. The results show that the new approach can achieve better performance on the datasets having a small number of training instances and competitive performance on the datasets having a large number of training instances. The analysis of evolved solutions shows that the proposed approach uses a smaller number of random forests over the deep forest method.