Summary
The paper proposes Trap-MID, a novel defense method against model inversion attacks, drawing inspiration from backdoor attacks and shortcut-based defenses against adversarial examples. The core idea involves adding poisoned data samples to the target model's training data. This data poisoning is based on the Blended backdoor attack, which calculates a linear combination of the training sample and a noise trigger pattern. To reduce the visibility of the trigger, a discriminator is trained in parallel, akin to a GAN training setup. This discriminator is then used to align the trigger patterns with the distribution of the clean training samples. The proposed defense method is empirically evaluated against common white-box attacks (GMI, KED-MI, LOMMA, PLG-MI) and defense methods (MID, BiDO, NegLS) using the standard 64x64 CelebA training samples.
Strengths
- The paper proposes an intriguing direction for model inversion defenses by utilizing backdoor attacks to induce shortcuts in the model. These shortcuts are then exploited by the attack, resulting in misleading attack outcomes. This approach is both clever and conceptually straightforward, which is advantageous.
- The paper is well-written, with all components and sections clearly described. The experimental evaluation is also well-defined, adhering to the standard procedures in model inversion literature.
- The results presented in the evaluation section are convincing. The experiments demonstrate the method's favorable privacy-utility trade-off and its effective defense capabilities. Additionally, the paper investigates adaptive attacks, a critical aspect of assessing a method's effectiveness. Although the adaptive attack still achieves good results against the proposed defense, this does not diminish the contribution's value. The Appendix further provides an extensive ablation and sensitivity analysis, essential for understanding the impact of hyperparameter selection and individual design choices.
Weaknesses
- Although the evaluation is conducted on four attack algorithms, two important aspects are missing in my opinion. First, the proposed defense should also be tested against black-box/label-only attacks. In particular, testing the defense method against the label-only BREP-MI [1] attack would be very interesting since this method aims to reconstruct samples by maximizing their distance to the decision boundaries. Given that the proposed Trap-MID leverages a model's ability to exploit shortcuts in its predictions, I am curious if the poisoned shortcut samples are differently placed in a model's embedding space. It might be the case that these samples are actually closer to the decision boundary compared to clean training samples. If this is true, the defense might fail. So even if white-box attacks are usually considered stronger, running some label-only attacks could add an intriguing perspective to the paper.
- Similarly, the defense is only evaluated in a low-resolution setting. However, it is important to also investigate a high-resolution setting, which can be considered more practical. In this context, the method should also be tested against PPA [2] to provide a comprehensive evaluation and see if the promised defense effect holds in this setting. I want to emphasize that I am not requesting additional experiments merely as a form of criticism, but because investigating these two additional attacks would add significant value to the paper, helping to support the claims.
- The evaluation heavily relies on attack accuracy and KNN Distance, both computed on an evaluation model trained on the target dataset. However, I think these metrics are limited as they tell us little about the actual visual similarity between reconstructed results and the training samples. For example, in Fig. 2, the images reconstructed from the NegLS model look unrealistic and reveal only limited information about the target identity. Still, the Attack Accuracy in Tab. 1 is high for PLG-MI and the KNN-Dist is low. Both metrics seem very susceptible to adversarial noise and misguided attack results. Similarly, the FID score only assesses the image quality of the reconstructed samples and compares it to the training data. But this metric has no meaning regarding privacy leakage: generating samples of the same style as the target training data leads to a low FID, even if the images reveal no private attributes. Conversely, images following a substantially different distribution can still reveal private features, even if the FID score is high. Additional metrics are therefore necessary to assess the actual effectiveness. From literature, one could use a model like CLIP or for the face recognition setting, a FaceNet model to assess the identity similarity between attack results and training samples (see, e.g., [2]). Another option would be the knowledge extraction score introduced in [4], which measures the information content included in the attack results.
- While I appreciate that the paper includes a theoretical analysis of the method and its effectiveness, I have some doubts about the formal proof. Particularly, I do not think that the KL divergence is a valid measure here to assess the trapdoor visibility. The problem is that the KL divergence compares two distributions. However, similar distributions might not necessarily mean that the triggers are invisible, and vice versa. For example, one could use triggers that are clearly visible but follow the clean data distribution. For example, using physical triggers instead of noise, but also noise patterns could be designed to follow the clean data distribution. This limits the expressiveness of the theorem since it relies on the KL divergence. In my view, the analysis should use a sample-wise measurement that compares a clean sample directly with its poisoned counterpart to provide a reliable measurement of trigger visibility.
- The work of [6] investigates a similar direction (using surrogate samples to misguide the attack). While their approach is partly limited in the number of classes to defend, I think the approach should at least be discussed in the related work section since the underlying idea of providing false guidance for the attack is conceptually similar.
Small Remarks:
- L118: "Given access to the target classifier f, the adversary aims to recover the private data of class y by estimating the posterior distribution p(X|y)..." -> While this definition is not wrong, I think model inversion attacks do not necessarily require the adversary to recover the whole posterior distribution; reconstructing a single sample can be enough for a serious privacy breach. While there exist attacks directly aiming to reconstruct the posterior, e.g., VMI [3], this might not be true for all attacks.
- There is another recent defense that might be included in the related work section [5]. The paper was very recently accepted at CVPR and only became available close to the NeurIPS deadline. So, I do not expect the paper to include this work in its evaluation. Just wanted to highlight this related work.
- L517: I think the URL is the wrong one here, as it leads to the KED paper.
- L525: If I have not missed it, the paper never actually defines the evaluation model architecture. Defining this architecture once helps the reproducibility of the paper.
-> I encourage the authors to participate in the rebuttal and will increase my scores if the weaknesses mentioned are addressed adequately.
References:
[1] Kahla et al., Label-Only Model Inversion Attacks via Boundary Repulsion. CVPR 2022
[2] Struppek et al., Plug & Play Attacks: Towards Robust and Flexible Model Inversion Attacks. ICML 2022
[3] Wang et al., Variational model inversion attacks. NeurIPS 2021
[4] Struppek et al., Be Careful What You Smooth For: Label Smoothing Can Be a Privacy Shield but Also a Catalyst for Model Inversion Attacks. ICLR 2024
[5] Ho et al., Model Inversion Robustness: Can Transfer Learning Help? CVPR 2024
[6] Chen et al. Data-Centric Defense: Shaping Loss Landscape with Augmentations to Counter Model Inversion. ICML Workshop 2023