Interpretable Neural Architecture Search via Bayesian Optimisation with Weisfeiler-Lehman Kernels

Current neural architecture search (NAS) strategies focus only on finding a\nsingle, good, architecture. They offer little insight into why a specific\nnetwork is performing well, or how we should modify the architecture if we want\nfurther improvements. We propose a Bayesian optimisation (BO) approach for NAS\nthat combines the Weisfeiler-Lehman graph kernel with a Gaussian process\nsurrogate. Our method optimises the architecture in a highly data-efficient\nmanner: it is capable of capturing the topological structures of the\narchitectures and is scalable to large graphs, thus making the high-dimensional\nand graph-like search spaces amenable to BO. More importantly, our method\naffords interpretability by discovering useful network features and their\ncorresponding impact on the network performance. Indeed, we demonstrate\nempirically that our surrogate model is capable of identifying useful motifs\nwhich can guide the generation of new architectures. We finally show that our\nmethod outperforms existing NAS approaches to achieve the state of the art on\nboth closed- and open-domain search spaces.\n