$μ$NAS: Constrained Neural Architecture Search for Microcontrollers

IoT devices are powered by microcontroller units (MCUs) which are extremely\nresource-scarce: a typical MCU may have an underpowered processor and around 64\nKB of memory and persistent storage, which is orders of magnitude fewer\ncomputational resources than is typically required for deep learning. Designing\nneural networks for such a platform requires an intricate balance between\nkeeping high predictive performance (accuracy) while achieving low memory and\nstorage usage and inference latency. This is extremely challenging to achieve\nmanually, so in this work, we build a neural architecture search (NAS) system,\ncalled $\\mu$NAS, to automate the design of such small-yet-powerful MCU-level\nnetworks. $\\mu$NAS explicitly targets the three primary aspects of resource\nscarcity of MCUs: the size of RAM, persistent storage and processor speed.\n$\\mu$NAS represents a significant advance in resource-efficient models,\nespecially for "mid-tier" MCUs with memory requirements ranging from 0.5 KB to\n64 KB. We show that on a variety of image classification datasets $\\mu$NAS is\nable to (a) improve top-1 classification accuracy by up to 4.8%, or (b) reduce\nmemory footprint by 4--13x, or (c) reduce the number of multiply-accumulate\noperations by at least 2x, compared to existing MCU specialist literature and\nresource-efficient models.\n