A comparative analysis of neural architecture search techniques in deep learning

Neural Architecture Search (NAS) is an innovative automated machine learning technique that aims to discover optimal network structures, thereby enhancing the overall performance of the model. This method significantly accelerates the efficiency of network architecture development, as it alleviates the need for professional knowledge of the neural network and time-consuming manual efforts. Moreover, NAS enables a more systematic exploration of the network structure, allowing researchers to discover novel architectures that may not be designed manually. NAS encompasses three fundamental aspects: search space, search strategy, and performance evaluation. This paper primarily focuses on the introduction of mainstream search strategies. They are elaborated by representative works of different categories of NAS. Additionally, this work offers a comparative analysis of the quantitative performance of each search strategy. It is helpful for providing insights into the advantages of each kind of solution and the future development trends of NAS, thereby paving the way for further advancements in this exciting field.