Chameleon: Adaptive Code Optimization for Expedited Deep Neural Network Compilation

Achieving faster execution with shorter compilation time can foster further\ndiversity and innovation in neural networks. However, the current paradigm of\nexecuting neural networks either relies on hand-optimized libraries,\ntraditional compilation heuristics, or very recently genetic algorithms and\nother stochastic methods. These methods suffer from frequent costly hardware\nmeasurements rendering them not only too time consuming but also suboptimal. As\nsuch, we devise a solution that can learn to quickly adapt to a previously\nunseen design space for code optimization, both accelerating the search and\nimproving the output performance. This solution dubbed Chameleon leverages\nreinforcement learning whose solution takes fewer steps to converge, and\ndevelops an adaptive sampling algorithm that not only focuses on the costly\nsamples (real hardware measurements) on representative points but also uses a\ndomain-knowledge inspired logic to improve the samples itself. Experimentation\nwith real hardware shows that Chameleon provides 4.45x speed up in optimization\ntime over AutoTVM, while also improving inference time of the modern deep\nnetworks by 5.6%.\n