Scalable Membership Inference Attacks via Quantile Regression

Summary

This paper introduces a new class of membership inference attacks based on performing quantile regression on the distribution of confidence scores induced by the model under attack on points that are not used in training. The approach is computationally efficient and does not require knowledge of the model's architecture, making it truly "black-box". The paper discusses the efficacy of this approach through extensive experiments on various datasets and model architectures. The experiments show that this approach is competitive with (and sometimes more effective than) much more computationally expensive shadow model approaches. Overall, the paper presents a more scalable and efficient membership inference attack.

Strengths

- interesting topic - novel method - well-written paper

Weaknesses

- more evaluation metrics needed - compare with different attacks

Questions

- I appreciate it if the authors could also compare the proposed method to the model-based attacks (https://arxiv.org/abs/1610.05820) and metric-based attacks. (https://arxiv.org/abs/2003.10595) - Why the proposed method performs worse on C-10 and C-100 but better on IN-1k? I would suggest the authors elaborate more on it.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

- Besides accessing the performance, it is also important to report the time/computational cost of different methods to further demonstrate the efficiency of the proposed method.

Summary

This work presents a novel membership inference attack that offers computational efficiency compared to state-of-the-art (SOTA) approaches. While the paper addresses a topical issue and provides an alternative attack method, there are several weaknesses that need to be addressed for a more comprehensive and convincing study, including the lack of evaluation of TPR@lowFPR, concerning empirical results, insufficient support for computational efficiency claims, and a lack of empirical evidence for the theoretical results. Addressing these weaknesses is essential to enhance the paper's credibility and provide a more comprehensive evaluation of the proposed attack method. Finally, I would be happy to increase my score if the authors could alleviate some of my concerns.

Strengths

**Topical and relevant**: The paper addresses a relevant topic in the field of membership inference attacks, which is of interest to conference readers. The proposed attack method introduces a novel approach that offers computational efficiency compared to existing SOTA methods. **Competitive performance without training multiple shadow models**: The authors suggest an attack method that competes with state-of-the-art approaches while being computationally more efficient. By eliminating the need to train several shadow models, their method offers a practical advantage, which is a significant contribution to the field.

Weaknesses

Regarding the theoretical results: **Lack of TPR@lowFPR evaluation**: The theoretical results presented in the paper focus on controlling the false positive rate (FPR). However, what truly matters in membership inference attacks (see [1]) is the true positive rate (TPR) at low FPR. The paper lacks analysis of TPR@lowFPR, which is crucial for assessing the attack's effectiveness. (This recent paper [2] could help the authors with this kind of analysis). Regarding the empirical results: **Concerning empirical results** - The quantile attack outperforms the Likelihood Ratio Test-based Inference Attack (LIRA) in terms of TPR@lowFPR on ImageNet. This is surprising, as the Neyman-Pearson Lemma suggests that the LRT is the most powerful test. The discrepancy may be due to the limited computational resources used in training shadow models or differences in training setups when training the underlying imagenet classifier as batch size, # epochs, etc impact the succeptibility to membership inference attacks (e.g., see [1]). - The replicated LIRA attacks with 8 and 4 shadow models outperform the read-off LIRA attack with 64 shadow models [2]. This observation questions the authors' conclusion that their attack is more favorable. It suggests that if the authors had trained 64 shadow models on ImageNet, the LIRA attack would perform better, as indicated by the evaluation on tabular datasets. **Computational intensity claims**: The authors argue that their method is computationally more efficient by eliminating the need for training shadow models. However, there is no empirical evidence supporting this claim. It is crucial to provide comparative plots showing the tradeoff between computational cost and TPR@fixed FPR to substantiate this argument. **Lack of empirical support for theoretical results**: The theoretical results regarding FPR control lack empirical evaluation. Conducting simulation studies or additional experiments would provide empirical evidence to support these claims. ----- **References** [1] Membership Inference Attacks From First Principles, https://arxiv.org/abs/2112.03570 [2] Gaussian Membership Inference Privacy, https://arxiv.org/abs/2306.07273

Questions

Could the authors please elaborate on why the read-off attack results from [2] that use 64 shadow models is performing worse than the authors' implemented attacks with 4 and 8 shadow models? There must be some discrepancies in the underlying models that have been trained!?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

This point should be ok.

Summary

The paper proposes a new membership inference attack based on training an attack model with the pinball loss. This avoids the use of shadow models, a common technique for membership inference, while often outperforming attacks which do use shadow models. They evaluate their attacks on a variety of image and tabular data, and show it is competitive with or outperforms the state of the art LiRA attack on these datasets.

Strengths

The proposed approach only requires the training time of a single shadow model. The proposed attack often reaches very high precision at low false positive rates. Precision is often competitive with attacks that require many shadow models. The evaluation considers several datasets, including both image and tabular data. The paper is easy to read.

Weaknesses

Some magic happens in the experiment section in the paragraph starting at line 290 (page 7). Here, the authors appear to introduce a different objective for their attack model, to directly learn a sample’s mean and deviation. I would like to see some ablation of the choices made in this paragraph. The proposed attack has a higher “online” computation cost than the offline variant of LiRA. LiRA can train shadow models independently of the target model, while the quantile attack requires the target model to train its attack model. From plots, it looks like the attack performs poorly at higher false positive rates. In settings where attack accuracy or attacker advantage is preferred to TPR and FPR, this attack might end up worse than existing attacks. Results are only reported for a single target model, rather than averaging over multiple target models. The attack model requires hyperparameter tuning, which can be amortized over multiple shadow models when running LiRA. Small comment: The definition of precision should have TPR in the numerator

Questions

Is the pinball loss used at all when attacking the CIFAR models? If it is, could you try to explain in more detail how it is used? Do you have thoughts on why the attacks start to perform worse than other attacks at higher FPR?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The paper does not discuss limitations. It has minor overclaiming in some places, such as when saying the attack requires only a single model, when all results are presented after hyperparameter tuning. Another place is in claiming existing shadow model attacks require knowledge of the target model architecture, which is untrue — the LiRA paper evaluates with different architectures.

Summary

The main focus of this paper is about the membership inference attack problem, i.e., determining whether a particular example was used in training or not. Most existing such attacks estimate the distribution of some test statistics, which are usually computationally expensive. In contrast to existing approaches, this paper proposes to perform quantile regression on the distribution of confidence scores induced by the model under attack on points that are not used in training. For implementation, the authors first collect a dataset known to be never used in training. Given a well trained model $f$, the corresponding confidence scores on collected data can be obtained. Thereafter, a quantile regression model $q$ is trained to predict the target quantile of the data-label pair.

Strengths

Reducing many (around hundred) shadow models to a single one is interesting. Experiment studies on different image datasets, including CIFAR-10, CIFAR-100, ImageNet-1K.

Weaknesses

-- Paper writing still needs some improvements. It requires efforts to follow. -- The parameter $\alpha$ is important yet hard to adjust manually. -- Currently, I have not observed any significant influence of the ongoing work on industry-level applications. Despite this, it is important to note that through additional analysis and exploration, there may be a possibility of uncovering substantial potential for impact in the future. As of now, the ongoing work does not appear to have made any noticeable waves in the industry. Nevertheless, by delving deeper into its intricacies and exploring its possibilities, we might unveil its capacity to bring about significant changes that could shape the industry landscape in the coming years. -- The authors fail to provide a comprehensive explanation of the significance of membership inference attacks (MIAs). It remains unclear why it is crucial to prevent the disclosure of specific data points that were used in training a model. If we assume that we have already inferred the utilization of certain data during the training process, what kind of significant problems could arise as a result? While consulting the references cited in the submission may shed some light on these matters, the paper itself should strive to be self-contained and inclusive, providing a thorough understanding of the subject matter without relying heavily on external sources. -- In order to facilitate better comprehension, it is necessary to reorganize the related work more effectively. By improving the organization of the related work, we can enhance its accessibility and ensure that readers can easily grasp its significance. A well-structured presentation of the related work will contribute to a clearer understanding of the subject matter.

Questions

All the current MIAs, including this submission, are based on the observation that models tend to overfit their training sets. Will it still be true for current big models trained on big data? My understanding is that the assumption under MIAs is mainly caused by the generalisation ability of most models. Will the conclusion still be true in other domains, e.g., natural language and audio?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

Please refer to the weaknesses shown above. Moreover, the contribution of this work appears to be rather limited in scope. To put it simply, the essence of this study involves training a regression model using only a subset of the available training data. It therefore makes that the significance and potential impact of this work may be somewhat restricted. It would be better to show experiments on other image models, such as based on VIT, SwinT, BEIT, image encoder in CLIP, .etc.

Summary

The authors propose a novel class of membership inference attacks based on quantile regression applied to confidence score distributions. The proposed 'black-box' algorithm does not require knowledge of the model's architecture and performs competitively with state-of-the-art shadow model attacks while being computationally more efficient. The paper presents several experiments showcasing the effectiveness of the approach across various datasets.

Strengths

1. The paper is well-written, with a clear and understandable motivation. 2. The technical aspects are solid and coherent.

Weaknesses

1. In my opinion, the main contribution of this paper, which involves using the quantile regression model q to predict the quantiles of the score s(x,y), is not a novel idea within quantile regression. [a] has previously utilized pinball loss in conformal prediction for estimating quantiles of output. Combining quantile regression with membership inference might not present a significant technical challenge and may seem trivial. 2. The claim that the method does not require any knowledge of the model's architecture appears overstated. The method's performance heavily depends on the quantile regression model q. If q is not well-trained, the method will fail. The paper only employs the ConvNext-Tiny model in the main paper, and the sensitivity to different model architectures remains unknown. 3. I find the theoretical part confusing. In my understanding, Theorems 1 and 2 do not establish the theoretical superiority of the proposed method compared to the baseline method using constant quantiles. The paper introduces group validity, which has been extensively studied in the papers mentioned in Line 253, resulting in a lack of theoretical novelty. Additionally, even if the quantile regression model q is learned from a richer set of models, there is no concept of a 'regression model q' for the baseline method. Therefore, it is unclear how the authors concluded that the proposed method outperforms the baseline attack. It seems more intuitive to conclude that a richer model complexity for q may benefit the proposed method. [a] Romano et al., Conformalized Quantile Regression, NeurIPS 2019

Questions

1. Could you provide a sensitivity study regarding the architecture of q? Is it an important factor? Will the method perform better when q uses the same architecture as f? Will it lead to more accurate quantile estimation? 2. I mentioned my concerns about the theoretical part above. 3. The experimental results do not perform as well on small datasets. While the computation of LIRA when n=2 seems acceptable and achieves much better performance than your method, could you explain the observed gap?

Rating

3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

I did not find the potential negative societal impact.

Summary

This paper studies the question of membership inference attack (MIA), which can be formalized as a hypothesis testing (HT) problem. The main contribution of this paper is introducing a new class of MIA. The authors claim that the proposed method is competitive with SOTA MIA methods while being more computationally efficient. The authors explain the motivation behind their method and theoretically prove the method's effectiveness (in terms of false positive rate). This paper also conducts experiments on various datasets to evaluate their method.

Strengths

1. This paper explains the relationship between MIA and HT, which is friendly to readers unfamiliar with MIA. 2. This paper compares the proposed method with those methods in Yeom et al [2018] and Carlini et al [2022]. The proposed methods seem to be a direct generalization of Yeom et al [2018].

Weaknesses

My major concerns lie in the theoretical part of this paper: 1. Theorems 1 and 2 are established at the population level. The approximation error of $q$ is not discussed. 2. This paper does not include any non-asymptotic results. 2. The theoretical part of this paper only compares the proposed method with the baseline attack (Yeom et al [2018]). The difference between the proposed method and Carlini et al [2022] is not discussed.

Questions

1. It seems unintuitive that the proposed method outperforms Carlini et al [2022]. Could you please explain this phenomenon? 2. Theorem 1 assumes that $\mathcal{H}$ is closed under additive shifts. Are there any examples of such hypothesis classes?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

No

thank you for the reply

I'm happy to keep my score! Re single target model: Generally when evaluating LiRA, it's common to train multiple models on different subsets of the training data, and verify that MI is successful for all of these. So this should capture more randomness than just SGD randomness. Re different ways to get low pinball loss: I think it's interesting that you may need quite different approaches to get small pinball loss on different datasets. This could be a limitation in practice, though, since this could end up with training a bunch of different models to see what works best. Re Fig 11 in LiRA paper: My reading of this figure is that the shadow model architecture (11a) doesn't matter that much, but I suppose that's open to interpretation.

Response to Rebuttal

Thank you for your response and for your clarifications. I still think that the results on FPR control are not really interesting and that they do not add much insight to the paper. As the authors have partly addressed my concern, I increase my score accordingly under the assumption that the authors will include i) computational resource stats (compute + clock time as the Carlini et al attack can be parallelized and then clock time is a less reasonable measure) to all their existing results and that ii) Figure 1 from the main paper will be modified so that the readout results with $n=64$ shadow models from Carlini et al are not featured in the Figure as they seem highly misleading (authors admited that 'the description in LiRA in the original paper is not completely specified' and so comparing authors' results to those by Carlini et al in this fashion is likely to be incorrect and misleading).

Thanks for the response, I am keeping my score

Thank you for your response. However, I'm not inclined to raise my score as my concerns remain unresolved. Although this paper doesn't center on conformal prediction, I've found that the techniques it presents align closely with the conformal prediction research line, both technically and theoretically, yielding limited novelty. 1. While I acknowledge that the paper doesn't introduce novel quantile regression techniques, I still perceive the amalgamation of quantile regression and the membership inference attack in this paper as straightforward. Creating a quantile regression model is fundamental in conformal prediction. Blending it with a new context doesn't inherently yield high novelty; further customization of techniques with the membership inference attack is needed. 2. In my view, group-conditional guarantees [1-3] have already received extensive exploration. Particularly, [1] delves into adversarial settings. The theoretical outcomes lack novelty and do not appear as solid as previous works. The theorem essentially merges the membership inference attack setting with group-conditional guarantees. 3. I understand that the original model and quantile regression model can belong to entirely unrelated classes. My point is that the lack of exploration into the model complexity of the quantile regression model is not justified. As you mentioned in the rebuttal, "large image datasets use ConvNext or ResNet; smaller, tabular datasets and models were attackable with gradient-boosting trees." What underlies this distinction? A sensitivity study is notably absent in both the main paper and rebuttal. The experimental results are heavily contingent on the suitable structure of the quantile regression model, demanding a strong inductive bias for its selection. 4. The concern about low performance on small datasets remains unresolved. LIRA's shadow models exhibit lower pinball loss than the one trained directly. What if a different quantile regression model were employed? This aligns with the issue raised in point 3. Given the reasons above, I'm inclined to recommend rejection for the current version of the paper. [1] Practical Adversarial Multivalid Conformal Prediction, NIPS 2022\ [2] Batch Multivalid Conformal Prediction, ICLR 2023\ [3] Online Multivalid Learning: Means, Moments, and Prediction Intervals, ITCS 2022