Jacks of All Trades, Masters Of None: Addressing Distributional Shift and Obtrusiveness via Transparent Patch Attacks

We focus on the development of effective adversarial patch attacks and -- for\nthe first time -- jointly address the antagonistic objectives of attack success\nand obtrusiveness via the design of novel semi-transparent patches. This work\nis motivated by our pursuit of a systematic performance analysis of patch\nattack robustness with regard to geometric transformations. Specifically, we\nfirst elucidate a) key factors underpinning patch attack success and b) the\nimpact of distributional shift between training and testing/deployment when\ncast under the Expectation over Transformation (EoT) formalism. By focusing our\nanalysis on three principal classes of transformations (rotation, scale, and\nlocation), our findings provide quantifiable insights into the design of\neffective patch attacks and demonstrate that scale, among all factors,\nsignificantly impacts patch attack success. Working from these findings, we\nthen focus on addressing how to overcome the principal limitations of scale for\nthe deployment of attacks in real physical settings: namely the obtrusiveness\nof large patches. Our strategy is to turn to the novel design of\nirregularly-shaped, semi-transparent partial patches which we construct via a\nnew optimization process that jointly addresses the antagonistic goals of\nmitigating obtrusiveness and maximizing effectiveness. Our study -- we hope --\nwill help encourage more focus in the community on the issues of obtrusiveness,\nscale, and success in patch attacks.\n