CONSTRAINED BAYESIAN OPTIMIZATION METHODS USING REGRESSION AND CLASSIFICATION GAUSSIAN PROCESSES AS CONSTRAINTS

In Bayesian optimization, which is used for optimizing computationally expensive black-box problems, constraints need to be considered to arrive at solutions that are both optimal and feasible. Techniques to deal with black-box constraints have been studied extensively, often exploiting the constraint model to sample points that have a high probability of being feasible. These constraints are usually modeled using regression models if the output is a known value on a continuous real scale. However, if the constraint or set of constraints is not modeled explicitly, but rather it is only known whether a design vector is either feasible or infeasible, then we treat the constraints as a categorical constraint. Because of this, these constraints should be modeled using classification models rather than regression methods, which have also been studied. Because of the variety of approaches to handling constraints, there is a need to compare methods for handling both classification constraints as well as continuous constraints modeled with individual regression models. This paper explores and compares four main methods, with two additional ones specifically for classification constraints; these methods handle black-box constraints in Bayesian optimization by modeling the constraints with both regression and classification Gaussian processes. We found that the problem type and number of constraints influence the effectiveness of different approaches with statistical differences in convergence. Regression models can be advantageous in terms of model fit time; however, this is also a function of the total number of constraints. Generally, regression constraints outperformed classification surrogates in terms of minimizing computational time, but the latter was still effective. Overall, this study provides valuable insights into the performance and implementation of different constrained Bayesian optimization techniques. This can help inform engineers on which method is most suitable for their problem, what issues they may encounter during implementation, and give a general understanding of the differences between using regression and classification Gaussian processes as constraints.