David and Goliath: An Empirical Evaluation of Attacks and Defenses for QNNs at the Deep Edge

Machine learning (ML) is shifting from the cloud to the edge. Edge computing reduces the surface exposing private data and enables reliable throughput guarantees in real-time applications. Of the panoply of devices deployed at the edge, resource-constrained microcontrollers (MCUs), e.g., Arm Cortex-M, are more prevalent, orders of magnitude cheaper, and less power-hungry than application processors (APUs) or graphical processing units (GPUs). Thus, enabling intelligence at the so-called deep/extreme edge is the zeitgeist, with researchers focusing on unveiling novel approaches to deploy artificial neural networks (ANN) on these constrained devices. Quantization is a well-established technique that has proved effective, i.e., negligible impact on accuracy, in enabling the deployment of neural networks on MCUs; however, it is still an open question to understand the robustness of quantized neural networks (QNNs) in the face of well-known adversarial examples. To fill this gap, we empirically evaluate the effectiveness of attacks and defenses from (full-precision) ANNs on (constrained) QNNs. Our evaluation suite includes three QNNs targeting TinyML applications, ten attacks, and six defenses. With this study, we draw a set of interesting findings. First, quantization increases the point distance to the decision boundary and leads the gradient estimated by some attacks to explode or vanish. Second, quantization can act as a noise attenuator or amplifier, depending on the noise magnitude, and causes gradient misalignment. Regarding adversarial defenses, we conclude that input pre-processing defenses show impressive results on small perturbations; however, they fall short as the perturbation increases. At the same time, train-based defenses increase adversarial robustness by increasing the average point distance to the decision boundary, which holds even after quantization. However, we argue that train-based defenses still need to smooth the quantization-shift and gradient misalignment phenomenons to counteract adversarial example transferability to QNNs. All artifacts are open-sourced to enable independent validation of results and encourage further exploration of the robustness of QNNs.