Bucks for Buckets (B4B): Active Defenses Against Stealing Encoders

Summary

This paper studies active defenses against model stealing attacks, with a specific focus on SSL encoders. The authors propose B4B, a framework including three modules: 1) a coverage estimation to distinguish legitimate users and adversaries; 2) a cost function that maps the coverage to a penalty to prevent stealing; 3) a per-user transformation function to prevent sybil attacks. The results show that B4B can prevent stealing and also preserve the utility of representations for legitimate users.

Strengths

- Trendy topic - Extensive ablation studies

Weaknesses

- Absence of comparison with previous work. - Impractical assumption - Design intuition lacks persuasiveness. - Lack of experiments

Questions

see limitations

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

This paper focuses on a trendy topic: defenses against model stealing attacks. To tackle this problem, the authors propose an active defense framework, namely B4B. They evaluate its performance on two SSL vision encoders, i.e., SimSiam and DINO, and consider four downstream datasets, i.e., CIFAR10, STL10, SVHN, and FashionMNIST. I appreciate the authors' efforts in conducting extensive ablation studies to understand the impact of hyperparameters. These experiments provide valuable insights and guidelines for readers interested in implementing B4B in real-world applications. However, there are still some concerns: - Absence of comparison with previous work. The authors claim B4B is the first active defense that prevents stealing without degrading representation quality for legitimate API users. However, it is important to note that [13] has demonstrated active defenses, such as adding random noise, can already achieve this goal. Additionally, other research works have proposed multiple active defenses against encoder stealing attacks, such as [26] (top-k, feature rounding, feature poisoning) and [32] (noise, top-k, rounding). I would suggest the authors reassess their claims and compare B4B with existing defense methods to elaborate on the advantages and unique contributions of B4B. - Impractical assumption. The authors declare that "B4B is independent of the protected encoder’s architecture and does not rely on any assumption about the adversary’s data and query strategy" (line 148-149). However, the design of B4B's cost function suggests that it primarily works for users who query data from a small distribution, while users with queries from a larger distribution are considered adversaries. This assumption is strong and impractical for API providers. I recommend that the authors discuss this limitation of B4B. Furthermore, it would be valuable to analyze the impact of queried distribution and the corresponding utility to demonstrate the trade-offs of B4B in real-world scenarios. - Design intuition lacks persuasiveness. The design of B4B is based on an observation: the representations returned to adversaries cover a significantly larger fraction of the embedding space than representations of legitimate users. However, a previous study [32] has shown that a small surrogate dataset is sufficient to steal the encoder. This raises concerns about the effectiveness of B4B, as attackers are able to employ small datasets to bypass it. Furthermore, in Figures 2 and Figure 9 in the Appendix, the authors visualize downstream datasets in the encoder’s embedding space. It is confusing to observe that STL10 and CIFAR10, two datasets that shared 90% classes, appear to be separated in both figures. Ideally, these datasets should be located in a similar space. This further weakens the design intuition’s persuasiveness. I recommend that the authors review their statements and provide explanations for the two points. - Lack of experiments. As a defense method, it is important to show its effectiveness against different encoder stealing attacks [13, 26, 32]. However, the authors only consider [14] in this paper, which limits the utility of their work. I recommend the authors expand their experimental evaluation to include different types of stealing attacks and report the performance of B4B. This will provide a more comprehensive assessment of B4B. Furthermore, the authors propose five transformations, namely Affine, Pad, Add, Shuffle, and Binary. However, there is a lack of experimental analysis regarding the impact of these transformations on B4B’s effectiveness. Additionally, in Table 1, the authors should explicitly mention the specific transformations applied to the legitimate users, attackers, and sybil attacks. This will enhance the clarity and reproducibility of their experiments.

Summary

The authors proposed B4B, which aims at actively preventing model stealing while preserving high-utility representations for 133 legitimate users. The paper is overall well-organized and easy to follow. The proposed three blocks are clear and effective, namely the embedding space estimation, cost increasing, and per-user representation transformations. This topic also makes sense in practical to protect the commercial APIs. Experiments are well designed.

Strengths

- The paper is easy to follow. - The motivation is clear enough, and the study is of value for protecting commercial APIs especially for those foundation models. - The proposed three blocks are simple yet effective.

Weaknesses

- Is there any defense baselines in Table 1? - Have you tried an ablation study on "SYBIL" in Table1? For example, maybe the user has 5+ sybil accounts with 20k queries per account. I think it could be interesting to explore if there is a convergency. - I would like to suggest more experiments on existing public APIs. As emphasized in paper, it should be a valuable contribution to a safer sharing and democratization of high-utility encoders over public APIs. It could be better to conduct experiments to directly support this claim.

Questions

refer to weakness

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

refer to weakness

Summary

The paper propose a defense over model stealing on-the-fly by assessing dataset distributions, and directly train on it. The paper is well-written and easy to read, while proposing a novel type of active defense over model stealing, without harming the performance of benign users.

Strengths

The authors propose an active defense over model stealing, which is based on the assumption that malicious queries will cover larger fraction of representation space, while benign queries are limited to a smaller fraction of space. The authors assign adversaries with noisy representations and benign users with clean representations. They also provide an additional transformation to avoid adversaries aggregate the stolen representation via creating multiple accounts.

Weaknesses

I have particular concerns on the assumptions and experimental setup of this paper. * The paper is based on the assumption that individual queries in a much narrower distribution, while adversary queries in a broader one. However, I wonder if this is the case in practice. For large tech firms, they might have requests from diverse users and query a specific API for many times, which also results in diverse usage. For a model trained for specific downstream task (eg, to identify cancer), there can be only one domain and the queries and adversaries can be similar (both include images containing/not containing cancer). Can B4B defend over this? Otherwise, authors should quality the potential usage of their work. Indeed, this assumption is difficult to prove to be true or to be false, so at least discussions should be included. * I question the experiment setting of Table. 1, which follows the assumption that adversary query is inherently different from benign ones. However, this also means adding a simple classifier will also have high defend accuracy, since the distribution can be drastically different for ImageNet and CIFAR 10. While training on image classifiers is nowhere near to real world applications, I believe authors should test on a wider range or threats possible. * In Figure 3, it turns out ImageNet naturally occurs for more precentage of buckets (covers larger proportion of representation space), while other dataset covers smaller fraction. Thus, I am again skeptical of the result given in Table. 1. It turns out that, ImageNet naturally occurs for larger fraction of representation space, thus naturally occupies more buckets. In this way, what if the legit query is ImageNet, while attack query is CIFAR-10? Will the proposed defense crash, since the legit query occupies larger fraction of representation space?

Questions

As stated in "Weakness" section. I'm particularly interested if the authors can clarify the threat model and potential applications of the proposed method.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

The authors do not discuss potential negative social applications. However, I do not find any potentially negative impact in this paper.

Summary

This paper proposes B4B as a defense against model stealing attacks for pretrained encoders. The main assumption of the defense is that attack queries generally have a much broader coverage of the embedding space than queries from normal users. Based on this assumption, B4B utilizes Local Sensitive Hash and separates the embedding space into a number of 2$^12$ buckets. B4B then calculates the fraction of buckets covered by the user queries, then adds Gaussian noise to the subsequent queries when the coverage is large, where this cost function exponentially increases. To defend against sybil attacks, different transformations are applied to the returned embedding vectors of different users, so that legitimate users can obtain high-quality models with the transformed embedding vectors, while attackers who aim to combine training data from different accounts cannot get performance boost.

Strengths

1. Designing an active defense for model stealing attacks is a good topic. The proposed approach is interesting, and seems to be generally applicable to different models and tasks. 2. The results show that B4B does not degrade the response quality for legitimate users, while significantly decreases the model performance trained by attackers.

Weaknesses

1. In general, I wonder what is the valid condition to consider a user as an attacker. In the current evaluation setup, all legitimate users are querying with images drawn from different data distributions. However, even if the users directly query the model using ImageNet samples, if the user queries are only limited to a small subset of labels instead of the full label set, then I also would not consider it an attack. Have you evaluated this setting? Would the input coverage in this case still be high? 2. Another question is whether the attack queries should be images included in the pretrained data, or does the defense also work for queries drawn from a similar distribution? For example, if you use STL-10 unlabeled data for pretraining and STL-10 labeled data to query, or if you use CIFAR-100 data for pretraining and CIFAR-10 to attack, does the defense work? 3. The full defense includes a couple of hyperparameters that seem critical, i.e., the number of buckets, the cost function, and the transformations. Is the same set of hyperparameters used for all tasks and models in the experiments? How much work is required to tune the implementation for new tasks and model architectures? 4. I do not fully understand the process of per-user representation transformation. How is it done for a new user? Does the defense randomly select/construct a transformation from all possible choices?

Questions

1. In the case when the users query the model using ImageNet samples from a small subset of the label set, would the input coverage still be high and thus the user is identified as an attacker? 2. Another question is whether the attack queries should be images included in the pretrained data, or does the defense also work for queries drawn from a similar distribution? For example, if you use STL-10 unlabeled data for pretraining and STL-10 labeled data to query, or if you use CIFAR-100 data for pretraining and CIFAR-10 to attack, does the defense work? 3. The full defense includes a couple of hyperparameters that seem critical, i.e., the number of buckets, the cost function, and the transformations. Is the same set of hyperparameters used for all tasks and models in the experiments? How much work is required to tune the implementation for new tasks and model architectures? 4. I do not fully understand the process of per-user representation transformation. How is it done for a new user? Does the defense randomly select/construct a transformation from possible choices?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors have adequately addressed the limitations and potential negative societal impact of their work.

Experiments: A range of stealing attack methodologies targeting self-supervised learning models

We would like to clarify that in our work, we tested against the SSL extraction attacks proposed in [13] and also included in [14], using the published code from those papers [13,14]. The code for the attacks proposed in StolenEncoder [26] and Cont-Steal [32] v2 has not been publicly released by the authors, so we were unable to run experiments using those attack implementations for comparison in our submission. We regret any confusion caused by not explicitly stating that the StolenEncoder [26] and Cont-Steal [32] attacks were not tested due to lack of available code. Please let us know if you need any additional details on the attacks and codebases we did evaluate.

Analysis of Encoder Stealing Against Our B4B Defense Using Varying Numbers of Sybil Accounts

To further demonstrate how the number of Sybils impacts encoder extraction effectiveness, we conducted experiments with 2, 3, 4, 5, and 6 fake accounts. Our additional results demonstrate a tradeoff between encoder extraction effectiveness and the overhead of using multiple Sybil accounts. Stealing with 2 Sybils yields the best (average) downstream task accuracy, indicating that it is the most effective extraction approach (2 Sybils were presented in Table 1 in the original submission). Using a single account is far less effective, though avoids the need to remap representations between accounts. More than 2 Sybils further reduces performance as remapping complications accumulate. With 10 Sybils, remapping leaves no more usable data for training the stolen encoder. This highlights our method's advantage: increasing the Sybil accounts makes encoder extraction impractical due to the growing remapping overhead. Our B4B method effectively defends against malicious users who attempt to steal encoders using Sybil accounts. The experiments in the table below utilize the same setup as in Table 1 of our submission, stealing from the victim encoder using the ImageNet dataset. |Attack|Defense|Number of Queries|CIFAR10|STL10|SVHN|F-MNIST| |-------------|---------|-------------------|-------------|------------|-------------|------------| |Single User|None|100k|68.1±0.03|63.1±0.01|61.5±0.01|89.0±0.07| |Single User|B4B|100k|12.01±0.07|13.94±0.05|19.96±0.03|69.63±0.07| |2 Sybils|B4B|2*50k|39.56±0.06|38.50±0.04|23.41±0.02|77.01±0.08| |3 Sybils|B4B|3*33333|33.87±0.05|38.57±0.06|21.16±0.01|72.95±0.05| |4 Sybils|B4B|4*25k|33.98±0.04|34.52±0.08|21.21±0.02|70.71±0.05| |5 Sybils|B4B|5*20k|32.65±0.05|32.45±0.05|29.63±0.01|70.12±0.08| |6 Sybils|B4B|6*16666|26.62±0.04|26.85±0.05|24.316±0.02|70.51±0.04|

W3 & Q3: Further Clarification on B4B (Hyper)Parameter Tuning for the Added CIFAR10 Encoder

The optimized B4B parameters are as follows: $\alpha=2$, $\beta=0.45$ and $\lambda=10^{-7}$. At a high level, this introduces substantial noise when query embeddings cover over 45% of the representation space, hindering model extraction. The $\beta$ threshold of 45% exceeds the coverage for the expected most complex downstream task (SVHN in this case). Further, the $\alpha$ value adds enough random noise to decrease downstream task accuracy by at least 10% and setting $\lambda=10^{-7}$ creates a flat cost curve near the origin, mapping small coverage fractions to small costs. Together, these parameters allow minimal query answering utility loss until an adversary's coverage reaches the extraction-hindering 45% threshold.

Thank you

Thank you for your answer. We are glad that we have addressed the reviewer’s concerns. Would the reviewer consider raising the rating for the submission?