OSLO: One-Shot Label-Only Membership Inference Attacks

Summary

In this work, the authors propose a novel label-only membership inference attack. This method involves adding an adversarial perturbation to compel a model, trained without the target data point, to misclassify the image. This approach requires only a single query to the model and achieves SOTA performance for label-only attacks.

Strengths

- The paper is well-written, with a strong and compelling motivation. - The results are very promising.

Weaknesses

The main concern for me is that this paper lacks important related works, like [1] and [2]. In particular, [1] uses a similar strategy to conduct a one-shot label-only inference attack. While the method proposed in this paper is quite similar to [1], it seems to improve the transferability of the adversarial perturbation by checking holdout models. However, the overall contribution is not clear to me. I am not sure if the contribution is significant enough for NeurIPS, but I would appreciate hearing the authors' perspectives on this. [1] WU, YUTONG et al. You Only Query Once: An Efficient Label-Only Membership Inference Attack. In The Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=7WsivwyHrS&referrer=%5Bthe%20profile%20of%20Jiwei%20Li%5D(%2Fprofile%3Fid%3D~Jiwei_Li1) [2] Wen, Yuxin et al. Canary in a Coalmine: Better Membership Inference with Ensembled Adversarial Queries. ArXiv abs/2210.10750 (2022): n. pag.

Questions

- What if the target model has a different training algorithm, like SGD? Would this make the attack less effective? - How to apply this attack to a new target data point? Does the attacker have to retrain some of the models?

Rating

5

Confidence

3

Soundness

2

Presentation

3

Contribution

3

Limitations

The authors mention limitations at the end of the paper which I appreciate.

Summary

The paper proposes a new label-only membership inference attack that only needs a single query to the target model to perform the attack. The general idea of the proposed attack is based on the observation that member samples need more adversarial perturbation to force a misclassification than non-member samples. The adversarial perturbation leads to non-members being misclassified while members are more robust and are still correctly classified. Transferable adversarial attacks are used to perform the attack in a single query using several surrogate models. The proposed One-Shot Label Only (OSLO) attack has three phases: 1) surrogate model training 2) creating the transferable adversarial examples on the surrogate models and 3) performing the membership inference on the target model. In contrast to other label-only membership inference attacks, OSLO has a different perturbation budget for each sample. The proposed attack is evaluated on three datasets (CIFAR-10, CIFAR-100, and SVHN), and in an ablation study, it is shown that the attack is model agnostic and also works with other transferable adversarial attack methods. The paper is technically sound, and the results show that OSLO outperforms existing attacks by a large margin.

Strengths

- The paper is well-written and easy to understand - The paper is well organized, and even a reader who is not an expert in privacy attacks can follow the paper easily - The proposed approach is novel - The approach is quite simple yet apparently highly effective

Weaknesses

- A recent paper also proposes a label-only attack that needs only a single query [1]. However, OSLO is not mentioned and, more importantly, not compared to YOQO. - Equation 2 is presented without introducing any of the variables. It is easier to understand equation 2 if it is mentioned in the text what $h_y(x;\theta)$ and $h_j(x;\theta)$ is. [1] You Only Query Once: An Efficient Label-Only Membership Inference Attack, Wu et al., ICLR 2024 Misc: - I think the H_0 and H_1 hypotheses have to be switched in equation 1 - There seems to be a typo in the heading of 6.1. I think it should be "Why OSLO outperform[s] previous approaches".

Questions

*Q1:* Why is the generation of the adversarial example stopped when the confidence is below a certain threshold $\tau$? In my understanding, it would make more sense to stop the generation as soon as the label flips. *Q2:* In reality, the attacker has to choose the threshold $\tau$. How would the attacker choose the threshold in a realistic setting? *Q3:* The perturbation budget is chosen differently for each sample. On the other hand, the threshold $\tau$ is the same for all the samples. Wouldn't it make sense to choose a different threshold for different samples since a lower threshold means more perturbation is added to the samples? *Q4:* In 6.2, it is mentioned that "three target models using different hyper-parameters" were trained. However, the hyperparameters were never mentioned. Which model architectures were trained here, what were the hyper-parameters for training these models, and what parameters were altered?

Rating

7

Confidence

5

Soundness

3

Presentation

3

Contribution

3

Limitations

The limitations are addressed.

Summary

The paper investigates the vulnerability of deep learning models to membership inference attacks (MIAs), focusing on label-only settings where only the predicted hard label is available. It introduces the One-Shot Label-Only (OSLO) MIA, which infers a sample's membership in the training set with a single query. OSLO leverages transfer-based black-box adversarial attacks, exploiting the characteristic that member samples are more resistant to adversarial perturbations. The paper presents extensive comparisons against state-of-the-art label-only attacks, showing that OSLO significantly outperforms them in terms of precision and true positive rate (TPR) under low false positive rates (FPR). For instance, OSLO achieves a TPR that is 7x to 28x stronger than previous methods under a 0.1% FPR on CIFAR10 for a ResNet model, while maintaining over 95% precision across multiple datasets. Additionally, the paper evaluates various defenses against OSLO, demonstrating its robustness and effectiveness.

Strengths

- The research question is important and well-motivated. - The paper is clearly written and easy to understand. - The proposed OSLO method significantly outperforms previous approaches.

Weaknesses

- **Mitigation Strategies**: The discussion on potential countermeasures against OSLO is somewhat limited. A deeper exploration of feasible strategies to mitigate the risks posed by such attacks would enhance the paper's practical utility. - **Computational Costs**: While OSLO does not rely on uniform perturbations like some existing methods, it requires higher computational resources to calculate specific perturbations for each sample. This could be a significant drawback in scenarios with limited computational capacity.

Questions

- How would methods aimed at enhancing model robustness, such as adversarial training, impact the performance of label-only inference attacks?

Rating

6

Confidence

2

Soundness

2

Presentation

3

Contribution

2

Limitations

The authors have adequately summarized the limitations at the end of the paper.

Summary

This paper presents a new label-only membership inference attack, which requires only a single query to the target model. Specifically, it leverages transfer-based black-box attacks to generate an adversarial perturbation. This perturbation is then added to the sample and input into the target model, and its prediction is used to judge whether it is a member or non-member. The experiments demonstrate the effectiveness of the proposed methods.

Strengths

- The comparison with previous label-only attacks is clear, making it easy to pinpoint the main contributions of this paper. - The experiments are comprehensive, covering many different datasets and models. - It is nice to see that the paper evaluates many defense strategies and uses Figure 9 to clearly show the ineffective handling of the model accuracy-attack mitigation trade-off in existing defenses.

Weaknesses

- While the paper is motivated by improving the attack costs compared to existing works, this method requires training multiple source models and validation models, which could be quite costly. This is especially true when applying it to relatively large-scale models, such as diffusion models, which makes the proposed attacks less practical. - It is unclear whether the proposed MIA was tuned on the validation data independent of the test set. Given that this method requires tuning threshold $\tau$, there is no mention of how this tuning is done and whether using the validation data. If test data was used to tune the method instead of validation data, the results reported might be overly optimistic and not reflect the proposed MIA's performance on "true" test (held-out) data. - The experimental setting is unclear. Specifically, it remains uncertain how the data split is implemented. Given that 25,000 samples are used to train the target model, it is important to clarify how many datasets are used to train the source models and validation models. More importantly, how many source models and validation models are trained for the experiments? As this is closely tied to the attack costs. - It is better to conduct an ablation study. The two main contributions are using transfer-based attacks to improve the query costs and using the validation models to adjust the scale of perturbations added to the adversarial example. The latter design is interesting as it resembles calibration to the sample difficulty in existing membership inference literature. To make each contribution clearer, it is better to conduct an ablation study to investigate the importance of each design. Additionally, an ablation study should investigate the impacts of the number of source models and validation models on the attack performance.

Questions

- How does your proposed approach perform when tuning the method's components on validation data that is disjoint from the test data on which final performance is reported? - How many datasets are used to train the source models and validation models? - How many source models and validation models are trained for the experiments?

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

Please see my above comments.

Thank you

Thank the authors for the clarification. I do see the distinction between [1, 2] and the novelty of the paper now. I really appreciate it. Therefore, I increase my score accordingly. Meanwhile, I have one question: how do you control the FPR accurately? I see you can control the threshold $\tau$ to change FPR, but how do you get the exact value, such as FPR = 0.1%?

Thanks for the response

Thank you for the response. The findings on the influence of adversarial training are indeed intriguing, and I appreciate your work on this. I have one additional question: when the victim model has undergone adversarial training, should the surrogate models used by the attacker also be adversarially trained? If so, does this imply that the attacker must first determine whether the victim model is adversarially trained before mounting the MIA? I understand that the inclusion of adversarial training complicates the scenario, as the adversarial examples used during training could also be considered member instances.

Response to Rebuttal

Thank you very much for your detailed rebuttal and the additional insights! My questions have been appropriately addressed and I will retain my current score.