Parallel Predictive Entropy Search for Multi-objective Bayesian Optimization with Constraints

Real-world problems often involve the optimization of several objectives\nunder multiple constraints. An example is the hyper-parameter tuning problem of\nmachine learning algorithms. In particular, the minimization of the estimation\nof the generalization error of a deep neural network and at the same time the\nminimization of its prediction time. We may also consider as a constraint that\nthe deep neural network must be implemented in a chip with an area below some\nsize. Here, both the objectives and the constraint are black boxes, i.e.,\nfunctions whose analytical expressions are unknown and are expensive to\nevaluate. Bayesian optimization (BO) methodologies have given state-of-the-art\nresults for the optimization of black-boxes. Nevertheless, most BO methods are\nsequential and evaluate the objectives and the constraints at just one input\nlocation, iteratively. Sometimes, however, we may have resources to evaluate\nseveral configurations in parallel. Notwithstanding, no parallel BO method has\nbeen proposed to deal with the optimization of multiple objectives under\nseveral constraints. If the expensive evaluations can be carried out in\nparallel (as when a cluster of computers is available), sequential evaluations\nresult in a waste of resources. This article introduces PPESMOC, Parallel\nPredictive Entropy Search for Multi-objective Bayesian Optimization with\nConstraints, an information-based batch method for the simultaneous\noptimization of multiple expensive-to-evaluate black-box functions under the\npresence of several constraints. Iteratively, PPESMOC selects a batch of input\nlocations at which to evaluate the black-boxes so as to maximally reduce the\nentropy of the Pareto set of the optimization problem. We present empirical\nevidence in the form of synthetic, benchmark and real-world experiments that\nillustrate the effectiveness of PPESMOC.\n