Neural Architecture Search with an Efficient Multiobjective Evolutionary Framework

Deep learning methods have become very successful at solving many complex\ntasks such as image classification and segmentation, speech recognition and\nmachine translation. Nevertheless, manually designing a neural network for a\nspecific problem is very difficult and time-consuming due to the massive\nhyperparameter search space, long training times, and lack of technical\nguidelines for the hyperparameter selection. Moreover, most networks are highly\ncomplex, task specific and over-parametrized. Recently, multiobjective neural\narchitecture search (NAS) methods have been proposed to automate the design of\naccurate and efficient architectures. However, they only optimize either the\nmacro- or micro-structure of the architecture requiring the unset\nhyperparameters to be manually defined, and do not use the information produced\nduring the optimization process to increase the efficiency of the search. In\nthis work, we propose EMONAS, an Efficient MultiObjective Neural Architecture\nSearch framework for the automatic design of neural architectures while\noptimizing the network's accuracy and size. EMONAS is composed of a search\nspace that considers both the macro- and micro-structure of the architecture,\nand a surrogate-assisted multiobjective evolutionary based algorithm that\nefficiently searches for the best hyperparameters using a Random Forest\nsurrogate and guiding selection probabilities. EMONAS is evaluated on the task\nof 3D cardiac segmentation from the MICCAI ACDC challenge, which is crucial for\ndisease diagnosis, risk evaluation, and therapy decision. The architecture\nfound with EMONAS is ranked within the top 10 submissions of the challenge in\nall evaluation metrics, performing better or comparable to other approaches\nwhile reducing the search time by more than 50% and having considerably fewer\nnumber of parameters.\n