Weaknesses
1. The proposed method, TARO, has two components but their individual contributions are not studied. TARO improves over the positive-only unsupervised attack by i. changing it to a targeted attack and ii. by selecting the target using a score function. However, there are indications in the prior work and in the results in this paper that the majority of the improvement comes from i. and not ii. For example, (Petrov and Kiwatkowska, 2022) also compare targeted and untargeted adversarial self supervision and show that targeted attacks across the whole batch result in more certifiably robust models than the untargeted ones. The fact that the improvements in Table 4 in this paper are significantly lower than the improvements in Table 3 is further evidence that it might be i. that drives the improvement rather than ii. Hence, the authors should consider adding an additional baseline to all their results where instead of using their score function, they select a random sample from the batch. This would also be computationally easier, as the paper reports a 5% overhead for the target selection.
2. The paper does not address the fact that the training loss and the loss used for computing the adversarial samples need not be the same. In fact, a number of prior works have this precise setting, e.g., (Alayrac et al., 2019), (Nguyen et al., 2022). Hence, it is not particularly novel that the loss used for training a positive-only model can be different from the one used for the attacks.
3. Only RoCL and ACL are compared against the novel method, but a number of other unsupervised adversarial training methods have been proposed as listed above. In order to claim that the target selection via a score function is better than them, the authors should compare with a larger range of methods. In my view, it is especially important to compare against methods that generate adversarial examples using the whole batch, rather than a single target as these methods might be stronger than the proposed TARO. Some examples of such prior work are (Ho and Vasconcelos, 2020), (Fan et al., 2021), (Petrov and Kwiatkowska, 2022) and the Adversarial-to-Adversarial and Dual Stream methods by (Jiang et al., 2020).
4. In the “Visualization of embedding space” paragraph and Fig. 3, it is actually not that clear whether the targeted attacks generate more samples on the boundary than the untargeted attacks. Visually the two plots seem quite similar.
References:
Jean-Baptiste Alayrac, Jonathan Uesato, Po-Sen Huang, Alhussein Fawzi, Robert Stanforth, and Pushmeet Kohli. Are labels required for improving adversarial robustness?, 2019
Chih-Hui Ho and Nuno Vasconcelos. Contrastive learning with adversarial examples, 2020
Ziyu Jiang, Tianlong Chen, Ting Chen, and Zhangyang Wang. Robust pre-training by adversarial contrastive learning, 2020
Lijie Fan, Sijia Liu, Pin-Yu Chen, Gaoyuan Zhang, and Chuang Gan. When does contrastive learning preserve adversarial robustness from pretraining to finetuning?, 2021
A. Tuan Nguyen, Ser Nam Lim, and Philip Torr. Task-agnostic robust representation learning, 2022
Aleksandar Petrov and Marta Kwiatkowska. Robustness of Unsupervised Representation Learning without Labels, 2022
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.