Beyond Slow Signs in High-fidelity Model Extraction

Deep neural networks, costly to train and rich in intellectual property value, are increasingly threatened by model extraction attacks that compromise their confidentiality. Previous attacks have succeeded in reverse-engineering model parameters up to a precision of float64 for models trained on random data with at most three hidden layers using cryptanalytical techniques. However, the process was identified to be very time consuming and not feasible for larger and deeper models trained on standard benchmarks. Our study evaluates the feasibility of parameter extraction methods of Carlini et al. [1] further enhanced by Canales-Mart\'inez et al. [2] for models trained on standard benchmarks. We introduce a unified codebase that integrates previous methods and reveal that computational tools can significantly influence performance. We develop further optimisations to the end-to-end attack and improve the efficiency of extracting weight signs by up to 14.8 times compared to former methods through the identification of easier and harder to extract neurons. Contrary to prior assumptions, we identify extraction of weights, not extraction of weight signs, as the critical bottleneck. With our improvements, a 16,721 parameter model with 2 hidden layers trained on MNIST is extracted within only 98 minutes compared to at least 150 minutes previously. Finally, addressing methodological deficiencies observed in previous studies, we propose new ways of robust benchmarking for future model extraction attacks.

Paper

Similar papers

Peer review

Reviewer aZHM6/10 · confidence 2/52024-06-26

Summary

This paper proposes a unified approach to model parameter extraction by combining two prior works and integrating efficiency optimization. The authors find out that the neuron wiggle sign extraction method proposed in the previous work addresses the bottleneck identified by Calini's paper. Furthermore, the authors present methods to identify hard-to-extract neurons and parallelize the extraction process. Empirical evaluation of different benchmarks shows that the proposed unified model extraction method achieves speed up compared to prior works.

Strengths

This paper has the following strengths: + The authors make an important observation that the neural wiggle method alleviates the sign extraction bottleneck in the prior work. + The authors show a unified model extraction method that combines sign extraction and model signature extraction built on top of two previous works. + The authors propose additional optimization methods to speed up the unified model extraction process.

Weaknesses

This paper has the following weak points: - There is no explicit description of the attack model/threat model used in this work; - While the proposed unified model extraction method shows speedup compared to the prior works and the results are successful on standardly trained ML models, the benchmarks used in this paper are still small-scale. This makes the intellectual property value of the evaluated models questionable since they might be too simple and do not carry financial value.

Questions

Please see the weak points above.

Rating

6

Confidence

2

Soundness

3

Presentation

2

Contribution

3

Limitations

Please see the weak points above.

Reviewer yp6m7/10 · confidence 4/52024-07-09

Summary

This paper explores advanced techniques for high-fidelity model extraction that go beyond simply observing "slow signs" like model outputs or gradients. The authors evaluate and enhance existing parameter extraction methods, particularly those developed by Carlini et al. and further improved by Canales-Martínez et al., applying them to models trained on standard benchmarks. Key contributions include: 1. A unified codebase integrating previous methods 2. Optimizations to improve efficiency in extracting weight signs 3. Identification of weight extraction, rather than weight sign extraction, as the critical bottleneck 4. Significant improvements in extraction speed (e.g., extracting a 16,721 parameter MNIST model in 98 minutes vs. 150+ minutes previously) 5. Proposals for more robust benchmarking of model extraction attacks

Strengths

- Comprehensive analysis and improvement of existing techniques - Practical advancements in extraction efficiency - Important insights into the relative difficulty of extracting different model components - Contribution to standardizing evaluation methods in the field

Weaknesses

- The scalability of the approach to larger, more complex models is not fully explored - Potential ethical implications of improving model extraction techniques are not thoroughly discussed - The paper could benefit from more discussion on potential defenses against these improved extraction methods

Questions

1. Ethical Considerations: Given the potential misuse of advanced model extraction techniques, could you elaborate on the ethical implications of your work? What safeguards or guidelines do you propose to mitigate potential negative impacts? Defense Mechanisms: While your paper focuses on improving extraction techniques, have you explored any potential defenses against these advanced model extraction attacks? If so, what were your findings, and if not, what are your thoughts on possible defensive strategies? 2. Defense Mechanisms: While your paper focuses on improving extraction techniques, have you explored any potential defenses against these advanced model extraction attacks? If so, what were your findings, and if not, what are your thoughts on possible defensive strategies?

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The experiments are only conducted the ViT-B/32 CLIP architecture. It should be at least mentioned that in future it makes sense to justify on more architectures. The authors lowered this limitation by doing evaluation on several datasets.

Reviewer PoZg5/10 · confidence 2/52024-07-24

Summary

This paper proposes to perform model extraction attacks against deep neural networks. First, the authors combine two previous proposed methods into a uniformed code-base. Then, they optimize the sign extraction strategy to achieve a speed up in model extraction. Their proposed method can be used for larger models (e.g., a model with 16721 parameters and two hidden layers).

Strengths

- An important problem. - The results show some efficiency of the proposed method.

Weaknesses

- This paper is hard to understand. The main novelty and the insight behind it are unclear to me. - If Canales-Martinez’s work already eliminates the sign extraction bottleneck (Line 47), why the authors still propose to optimize this process? - What is the design motivation? Why the authors could do better? - Carlini’s work is not the only crypt-analytical work, why not compare to other works (e.g., reference 7)? - Too much background in the Methodology part. For example, the whole Sec. 3.1 is about some background. - Section 3.4 is hard to follow. - The cost of parallel computing is exponential.

Questions

See above.

Rating

5

Confidence

2

Soundness

2

Presentation

2

Contribution

2

Limitations

See above.

Reviewer r46i7/10 · confidence 3/52024-07-24

Summary

This paper continues a line of work on cryptanalytically extracting network parameters from (input, logit) pairs. It includes a concise explanation of relevant prior work in the area, a codebase that unifies two key prior works in the area to enable standardized comparisons, and several improvements to these prior methods that enable faster extraction of weight signs and signatures.

Strengths

- This is an interesting research area that deserves more attention from the broader community - This paper not only makes progress in this area, but it also enables future work with its codebase - The paper is very well-written and does a great job of summarizing the fairly complex attacks that it builds on. Honestly this paper is one of the best I've ever seen for how well it values the reader's time and presents the relevant background and contributions. - The performance improvements are substantial - The discussion includes some interesting analysis

Weaknesses

Moderate issues: - The main reason I'm not giving this paper a higher score is that it seems to make solid incremental improvements to two prior techniques. It doesn't introduce fundamentally new methods as far as I can tell. However, I think it still adds considerable value to the area, passes the quality bar, and could foster useful discussion at NeurIPS. Minor issues / suggestions: - My first question coming into this paper (not having read the prior work) was, "How on earth do people extract the layers one by one when all they have are the inputs and logits?" It took some time to understand this, and I think this is possibly something that could be improved with a well-designed figure. - Line 85: "In a study where the adversary is assumed to have complete access to both the training data and hyperparameters, 93.4% was the maximum fidelity reached by the replicated model." What dataset is this on? Is 93.4% lower than expected? By how much? Typos: - Missing a space in the Figure 3 caption: "(a)The"

Questions

Line 48 of the paper states, "Further improving on sign extraction we speed the process up by up to 14.8 times, so that sign extraction only takes up as little as 0.002% of the whole extraction time for larger models." A 14.8x speedup is nice, but doesn't this mean that sign extraction only took ~0.02% of the whole extraction time before? If so, then it was already negligible. Maybe this is part of your point about Canales-Martinez incorrectly estimated the difficulty of full extraction relative to sign extraction alone. Which of the analyses in Section 3.1 are novel, and which have been touched on before in prior work? Is there any chance that attacks like this could work for networks using smoother nonlinearities like GELU?

Rating

7

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

Adequately addressed

Authorsrebuttal2024-08-12

Dear Reviewer PoZg, Thank you so much for your insightful feedback on our paper! We hope you found our responses useful. As the discussion period is coming to a close, please feel free to ask any remaining questions you may have. We're happy to provide further clarification!

Reviewer PoZg2024-08-14

Thanks for the response

1. Section 3.1's majority of contents are still backgrounds (about two pages, which occupies about half of the methodology section). 2. Section 3.4 is unclear about the acceleration improvements that have been made. This part seems to focus on engineering efforts rather than systematic methodologies. 3. According to the introduction and the methodology, the paper focuses most of its effort on sign extraction. Based on the rebuttal, I understand this paper tries to convince others that sign extraction is not a bottleneck. Is that correct? If so, why do the authors still spend effort on improving sign extraction (see abstract lines 12-14)? 4. The authors stated "Hence, the paper is written in a way to highlight that the previously identified bottleneck is no longer relevant and suggests that future improvements need to be made to the weight extraction process." **However, according to the rebuttal, the paper's main contribution is to optimize the sign extraction attack instead of the weight extraction process.** "We identified that some neurons are harder to extract than others and performing the neuron wiggle method on them for more iterations does not help extract more correct signs. So, overall we minimized the neuron wiggle extraction time for easy to extract neurons and performed sign extraction for hard to extract neurons later in the pipeline." This is contradictory. Thanks again for the authors's response. However, my questions are not well-addressed. I decide to keep my score.

Authorsrebuttal2024-08-14

Thank you for your response

1. The first part of Section 3.1 is indeed about one and a half pages of explaining previous works’ methodologies but they were necessary to make our contribution understandable. From reviewer r46i we received positive comments regarding this: “The paper is very well-written and does a great job of summarizing the fairly complex attacks that it builds on. Honestly this paper is one of the best I've ever seen for how well it values the reader's time and presents the relevant background and contributions.” 2. Section 3.4 is a mix of methodological improvements and engineering efforts. We could not go into detail about it in the paper due to limited space but have a full description of what these entail in Appendix A. Especially our insights on how precision improvement is not necessarily needed are not trivial, since this was not possible yet in Carlini et al.’s paper and Canales-Martinez et al. never connected signature and sign extraction. 3. It was not clear at first that sign extraction was not the bottleneck anymore until we unified all methodologies and metrics. In Figure 1 (a), we show with Sign CM original and unified compared to Signature Carlini that sign extraction was more inefficient before but after unifying codebases it is taking a little less time than the signature extraction. However, this does not mean sign extraction time became trivial just using Canales-Martinez et al.’s methods. Only with our improvements in sign extraction, the sign extraction actually becomes trivial compared to the signature extraction. Further, in the full pipeline our improvements in the sign extraction significantly improve the overall extraction time as shown in abstract lines 15-17. 4. We are not too sure what the reviewer means by contradictory, we followed prior work and improved over the sign extraction performance. By the time our improvements became significant we decided to evaluate the system, including evaluation of the system as a whole, not separately as Canales Martinez. This is where we realized that sign extraction time can become trivial compared to signature extraction, and future efforts should be directed towards signature extraction improvement. Note that everyone working in this field from reading all of the work assumed that sign extraction is the bottleneck and hence iterated over it.

Reviewer PoZg2024-08-14

Thanks again for the response.

1. In my opinion, the background part should be included in the paper but go to the related work or background part. More space should be spent on technical contributions (e.g., Section 3.4). 2. I acknowledge the authors' effort in justifying that sign extraction is no longer a bottleneck; this is indeed important. However, this is not so clear in the paper. I think the author would like to make them clearer in the paper.

Authorsrebuttal2024-08-14

Thank you for your prompt response.

Thank you very much for a prompt response!! 1. We will revise the structure to focus more on our technical contributions. We will put the background methodology in a specific background methodology subsection and will move some of it into the appendix. We will instead move some of our technical descriptions for Section 3.4 into the main paper methodologies. 2. We will work on clarifying this further.

Reviewer yp6m2024-08-14

It is convenient in the ML community to put the code after the abstract or introduction.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC