In privacy-preserving machine learning, differentially private stochastic gradient descent (DP-SGD) performs worse than SGD due to per-sample gradient clipping and noise addition. A recent focus in private learning research is improving the performance of DP-SGD on private data by incorporating priors that are learned on real-world public data. In this work, we explore how we can improve the privacy-utility tradeoff of DP-SGD by learning priors from images generated by random processes and transferring these priors to private data. We propose DP-RandP, a three-phase approach. We attain new state-of-the-art accuracy when training from scratch on CIFAR10, CIFAR100, MedMNIST and ImageNet for a range of privacy budgets $\varepsilon \in [1, 8]$. In particular, we improve the previous best reported accuracy on CIFAR10 from $60.6 \%$ to $72.3 \%$ for $\varepsilon=1$.
Paper
Similar papers
Peer review
Summary
This paper presents a method for synthesizing pre-training data for private fine-tuning without requiring access to any real data. The data generation process utilizes some fundamental properties of natural images that have been previously studied in fields outside of private learning. For example, natural images often exhibit a high degree of sparsity and contain oriented structures. After pre-training the model on the synthesized data, the authors employ a two-stage private learning framework. The first stage involves tuning only the linear classifier, while the second stage tunes all parameters.
Strengths
1. The concept of generating 'free' pre-training data is intriguing, and the methods employed in this paper appear reasonable. 2. Experiments conducted on several vision datasets demonstrate that such synthesized data is important in private learning, particularly when the privacy budget is small. 3. The authors introduce a two-stage approach for private learning, which is new to the private learning community and results in some improvements. 4. The results are well-organized and easy to follow.
Weaknesses
1. As shown in Figure 6, some improvements of the proposed algorithm are from the two-stage approach (Phase-II and Phase-III) for private learning. Can the two-stage approach improve the baseline? E.g., can the method in Hölzl et al., [27] get better results if it also uses Phase-II and Phase-III for private learning and set the hyperparameters accordingly? 2. Kumar et al., 2022 [35] propose to first tune the classification head and then tune all parameters. The authors may want to discuss how their motivation can also explain the benefit of the two-stage approach, as a complementary to the discussion in Line 102 – Line 114. I am willing to increase my score if the authors can address my concerns.
Questions
1. Is it easy to generate ‘free’ pre-training data for language or speech tasks? Language or speech data also exhibits some natural structures, e.g., the subject–verb–object word order. However, I’m not sure whether there are off-the-shelf algorithms that use such properties to generate data. 2. I see the authors mark 'Reproducibility' as yes but the code is not uploaded. Is there a plan to make the code that reproduces all the main observations publicly available?
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
2 fair
Presentation
4 excellent
Contribution
3 good
Limitations
Limitations are discussed adequately.
Summary
The paper focuses on the problem of improving the privacy-utility trade-offs for the foundational technique of DP-SGD. The authors explore gains from learning priors from images generated by random processes and transferring the priors to private data. They do this via designing DP-RandP, a three-phase approach that first learns priors, then training a head classifier on top of frozen features, and then training both the feature extractor and classifier. Through an empirical evaluation on CIFAR10, CIFAR100, and MedMNIST, they show that their method is able to achieve new SOTA accuracies when training from scratch.
Strengths
1. The paper focuses on the important problem of improving the privacy-utility trade-offs for the foundational technique of DP-SGD. 2. The paper is well-written, and the novel ideas are easy to understand.
Weaknesses
Minor point: 1. I think it might be cleaner to mention the gains in privacy cost in terms of the absolute gains in the parameter used to measure privacy. E.g., in Fig 2 caption, since $\epsilon$ goes from 6 to 3, it might be easier to understand a factor of 2 gain? Stating a factor of 20 (which technically applies to the change in the outcome probabilities used in the defn. of DP) might be somewhat misleading (and overstated). Same comment for the gains stated in line 188.
Questions
Since the novel technique has 2 new phases compared to prior works (e.g., De et al. [13]), it might be useful to a reader to understand what gains the first stage provides vs. gains provided by the second phase by itself? Although this is somewhat studied in section 3.3, I feel it might be useful to understand in the evaluation in section 3.2 (so DP-RandP can be compared with existing baselines, and with specific phases provided to those baselines).
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
4 excellent
Presentation
3 good
Contribution
3 good
Limitations
Yes.
Summary
The authors propose using synthetic data to reduce the utility degradation induced by DP-SGD when training on image classification datasets. They suggest a 3 steps procedure: 1) pre-training on synthetic data, 2) DP training the linear head, 3) DP fine-tuning the full body. The procedure seems effective on the considered benchmarks and at various levels of $\epsilon$.
Strengths
- Although the novelty is limited (the synthetic data was already designed by other works and the idea of pre-training to facilitate convergence in DP-SGD training has been explored), the observation this strategy is particularly effective in DP training is interesting and will be surely extremely useful for future research in DP training for image classification. - The paper is clearly written, the experiments are well explained. - The analyses carried out are interesting, and provide practitioners insights on how to adapt the proposed training procedure to their setup
Weaknesses
- The considered datasets are relatively simple. The method does not seem to scale well as the number of classes increases (see CIFAR-100 and ImageNet experiments). While this holds true for most DP training procedures, the ImageNet experiment might suggest the methodology has little or marginal utility in this case. - While the comparison with ScatterNet and Equivariant-CNNs is particularly lengthy in the Related Works section, a comparison with methodologies leveraging public non-synthetic data (e.g. [1,2,3,4]) is missing. While such a comparison is unfair, and will show inferior performance of the proposed methodology, this does not diminish its utility. The purpose of such a comparison would be to show the gap that exists between using synthetically generated data (both data-driven or not) and real data, and would set the goal of bridging this gap with purely synthetic techniques in future works. Minor: - The formatting of the captioning of when multiple subfigures are present on the same line does not render very well, I'd recommend improving it. [1] How to make semi-private learning more effective, https://openreview.net/pdf?id=vVXRNcltT6 [2] Differentially Private Image Classification from Features, https://arxiv.org/pdf/2211.13403.pdf [3] Large Scale Transfer Learning for Differentially Private Image Classification, https://arxiv.org/pdf/2205.02973.pdf [4] Differentially private diffusion models generate useful synthetic images, https://arxiv.org/pdf/2302.13861.pdf
Questions
- Did the authors consider the idea of augmenting their training set with fractals (e.g. [5], and not only black-and-white ones, but also coloured ones) - Did the authors consider applying other contrastive or self-supervised pre-training strategies to better leverage the synthetic data? It may be possible that different strategies can differently impact the accuracy of the linear probe (e.g. as shown in [4]) and may attain higher utility in Phase 3 of the training procedure. - Could you add error bars to the plots? (e.g. $+/- 2\sigma$) [5] FractalDB, https://github.com/hirokatsukataoka16/FractalDB-Pretrained-ResNet-PyTorch
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
Properly Addressed.
Summary
The paper considers the problem of training ML models in a privacy-preserving way, focusing primarily on image classification. The main observation in this paper is that synthetic data learned from random processes can be used to learn useful visual priors which can be extremely useful for private training. Building on this, the authors propose a three phase approach for privately training ML models: 1. Learn a prior (feature extractor) from synthetic images generated by random processes, 2. Train the head classifier with frozen features with privacy parameter epsilon_1, and 3. End-to-end training using the remaining privacy budget epsilon_2. The authors provide an extensive empirical evaluation of this method over three different datasets for a wide range of privacy parameters, demonstrating that their methods significantly improve over state-of-the-art results for all privacy regimes. For example, for CIFAR10 with epsilon = 1, they obtain 72% accuracy, significantly improving over the SOTA (60%).
Strengths
As written above, the paper has some clear and strong empirical improvements in the performance of private training for image classification. Moreover, the idea of using priors learned from synthetic data is interesting and turns out to be extremely useful for private training as demonstrated in the paper. This techniques could also be useful for private training in other domains.
Weaknesses
In general, the paper has strong results, and aside from some minor comments (see below), my main critique is that the paper explains the algorithm too quickly without too much details, and starts the empirical evaluations too early. I would recommend the authors to spend more time in explaining their algorithms, and giving more background for phase 1 which is basically the main component of the algorithm. Also, the authors should give enough details for future readers to be able to replicate the results; for example, which version of DP-SGD is used? What is the batch size? Does it use momentum? More comments: 1. While the methods in the paper improve over SOTA for private training, they don’t achieve the same performance as private training with public data. I would suggest that the authors also compare their methods with methods that use a public dataset for pretaining (the authors currently do this only for epsilon = 1). 2. The idea of having both phase 2 and 3 is interesting and it would be useful to see the improvements of that when combined with pretraining using public data. 3. It would be useful to provide further evidence for the importance of combining both the second and third approached in the algorithm: one possibility is to compare the 3-phased approach with an approach that uses phase 1+2 or phase 1+3. This has been done in separate tables in the paper but providing the performance of all three approaches in the same plot would help better illustrate the advantages of all three phases. 4. Privacy splitting: - The privacy splitting strategy used in the paper is not completely clear and should be explained in more detail. - The paper only shows how privacy splitting affects the performance for small values of eps=1 in Figure 6. It would be useful to provide a similar figure for larger values of epsilon, to show the advantages of phase 3 over phase 2 for such epsilons, and to show how robust is this method to privacy splitting. 5. Can this approach be applied beyond image classification?
Questions
See above.
Rating
8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
3 good
Contribution
4 excellent
Limitations
See above.
Please use this reference list for the referred paper numbers in the rebuttals.
References: [1] M. Nasr, S. Mahloujifar, X. Tang, P. Mittal, and A. Houmansadr. Effectively using public data in privacy preserving machine learning. ICML 2023. [2] H. Mehta, A. G. Thakurta, A. Kurakin, and A. Cutkosky. Towards large scale transfer learning for differentially private image classification. TMLR 2023. [3] H. Mehta, W. Krichene, A. G. Thakurta, A. Kurakin, and A. Cutkosky. Differentially private image classification from features. TMLR 2023. [4] F. Pinto, Y. Hu, F. Yang, A. Sanyal, How to make semi-private learning more effective, TrustML-(un)Limited, ICLR 2023. [5] S. Ghalebikesabi, L. Berrada, S. Gowal, I. Ktena, R. Stanforth, J. Hayes, S.De, S. Smith, O. Wiles and B. Balle. Differentially private diffusion models generate useful synthetic images, arXiv preprint arXiv:2302.13861. [6] M. Baradad, J. Wulff, T. Wang, P. Isola, and A. Torralba. Learning to see by looking at noise. NeurIPS 2021. [7] C.Chiu, J. Qin, Y. Zhang, J. Yu, Yonghui Wu. Self-Supervised Learning with Random-Projection Quantizer for Speech Recognition. ICML 2022. [8] J. Devlin, M. Chang, K. Lee, K. Toutanova, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACL 2019. [9] Y. Liu, M. Ot, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, V. Stoyanov. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv preprint arXiv:1907.11692. [10] W. Shi , R. Shea , S. Chen, C. Zhang , R. Jia, Z. Yu. Just Fine-tune Twice: Selective Differential Privacy for Large Language Models. EMNLP 2022. [11] M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pages 308–318. ACM, 2016. [12] T. Sander, P. Stock, and A. Sablayrolles. Tan without a burn: Scaling laws of dp-sgd, ICML 2023. [13] S. De, L. Berrada, J. Hayes, S. L. Smith, and B. Balle. Unlocking high-accuracy differentially private image classification through scale. arXiv preprint arXiv:2204.13650. [14] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale, ICLR 2021. [15] D. Yu, H. Zhang, W. Chen, and T.-Y. Liu. Do not let privacy overbill utility: Gradient embedding perturbation for private learning. ICLR 2021. [16] A. Panda, X. Tang, V. Sehwag, S. Mahloujifar, and P. Mittal. Dp-raft: A differentially private recipe for accelerated fine-tuning. arXiv preprint arXiv:2212.04486. [17] H. Kataoka, K. Okayasu, A. Matsumoto, E. Yamagata, R. Yamada, N. Inoue, A. Nakamura, and Y. Satoh. Pre-training without natural images, IJCV 2021. [18] T. Wang and P. Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. ICML 2020. [19] K. He, H. Fan, Y. Wu, S. Xie, R. Girshick. Momentum Contrast for Unsupervised Visual Representation Learning, CVPR 2020. [20] F. A. Hölzl, D. Rueckert, and G. Kaissis. Equivariant differentially private deep learning, arXiv preprint arXiv:2301.13104. [21] A. Kumar, A. Raghunathan, R. M. Jones, T. Ma, and P. Liang. Fine-tuning can distort pretrained features and underperform out-of-distribution, ICLR 2022. [22] S. Levine, Chelsea Finn, Trevor Darrell, and P. Abbeel. End-to-end training of deep visuomotor policies, JMLR 2016. [23] F. Kanavati and M. Tsuneki. Partial transfusion: on the expressive influence of trainable batch norm parameters for transfer learning. In Medical Imaging with Deep Learning, 2021 [24] S. Gopi, Y. T. Lee, and L. Wutschitz. Numerical composition of differential privacy. NeurIPS 2021. [25] M. Baradad, R. Chen, J. Wulff, T. Wang, R. Feris, A. Torralba, and P. Isola. Procedural image programs for representation learning. NeurIPS 2022.
Thank you for your rebuttal
I have read the rebuttal, and I find it addresses most of my concerns. - On performance: Although the change to get the improvement for ImageNet is significant, I think it is acceptable as it does not change the key concept of the paper. Obviously, the used $\epsilon$ values are quite high, and tight $\epsilon$ would be preferred. However, this is acceptable as I understand it would be difficult to show improvements in extremely tight privacy regimes. Please include all the details in the updated draft and the source code. - On the comparison with semi-private methods: including a discussion in the updated draft could be useful. - About fractals and other contrastive learning techniques: Thanks for the clarification. Since these are natural questions that may arise in a reader, it could be useful to include some of the responses in the updated draft.
Thank you
We are grateful our rebuttal answered most of the concerns in the review. As recommended by the reviewer, we will include these points in the updated draft, as we cannot update the OpenReview draft during the review period.
We thank you again for your review and will include all these clarifications in the camera ready. As the discussion period is drawing to a close, we would like to ask if you have any remaining concerns that we may address. If you feel that our rebuttal has answered your concerns, we would be grateful if you might consider increasing your score.
Thank you for your response. It addresses most of my concerns. There is one remaining question, and I am happy to increase my score if the authors can address it. 1. Can the two-stage approach improve the baseline? There seems to be a misunderstanding of my question. What I mean is the following: 1) skip Phase-I, i.e., do not pre-train the model. 2) use Phase-II and Phase-III training anyway. For example, the authors can take the code of De et al. [13], and first train the head classifier on top of random layers. After training the head classifier, the authors can continue the Phase-III training. I would be happy to increase my score if the authors are willing to conduct this experiment and tune how the privacy budget is allocated (as in Figure 6).
Two-stage approach for baseline
We thank reviewer for the insightful feedback. Here we provide our Phase II+Phase III results on CIFAR10 at $\varepsilon=1$. Each reported result is averaged across five independent runs and we also report the stand derivation. As noted in Table 11, we use $\sigma=9.3$ for $\varepsilon=1$ while De et al. use $\sigma=10$ for the same number of 875 steps (equals to 73 epochs). This is because we only round $\sigma$ up to 0.1 and we ensure the $\sigma$ we use in Table 11 will not exceed the designed privacy bound. As we add less noise compared to De et al., the baseline result is $57.46_{0.48}$. Epochs for Phase II |$0$ |$1$ | $2$| $4$ | $8$ | $12$ | $16$ | $20$ | $50$ | $73$ --- |---|---|----|-----|-----|------|------|------|------|----- Accuracy |$57.46_{0.48}$|$57.01_{0.69}$|$56.27_{0.57}$|$55.04_{0.90}$|$54.45_{0.80}$|$54.09_{0.41}$|$53.15_{0.91}$|$52.26_{0.97}$|$47.00_{1.53}$|$25.80_{2.08}$ This result shows that when the feature extractor is randomly generated, if we train the head classifier with more steps while keeping the same total steps, the performance will decrease compared to directly finetuning the whole network. Our analysis of this is that, the feature extractor is not encoded with any prior and it is better to update the full network in the whole training procedure to learn more information. We will add this result and the discussion in the draft.
Thank you for the experiment. It addresses my concern. I have changed my recommendation to Accept.
Response
I'm happy with the authors' response and still recommend to accept the paper.
Thank you for the rebuttal
I have read the authors' rebuttal, and thank them for answering the questions in my review. It would be nice to see the details provided in the response to the 2nd point above included somewhere in the paper (even if in the Appendix) so practitioners can be more aware of the interplay of the different phases of the proposed technique as well as the synergies between them. I would like to maintain my positive recommendation for the paper.
Thank you for your comment. To followup on your question on what gains are provided by the second phase by itself, here we provide our Phase II+Phase III results on CIFAR10 at $\varepsilon=1$. Each reported result is averaged across five independent runs and we also report the standard derivation. By excluding Phase I in this experiment, we can answer the question of whether there are any gains provided by the second phase by itself. As noted in Table 11, we use $\sigma=9.3$ for $\varepsilon=1$ while De et al. use $\sigma=10$ for the same number of 875 steps (equivalent to 73 epochs). This is because we only round $\sigma$ up to 0.1 and we ensure the $\sigma$ we use in Table 11 will not exceed the designed privacy bound. As we add less noise compared to De et al., the baseline result is $57.46_{0.48}$. Epochs for Phase II |$0$ |$1$ | $2$| $4$ | $8$ | $12$ | $16$ | $20$ | $50$ | $73$ --- |---|---|----|-----|-----|------|------|------|------|----- Accuracy |$57.46_{0.48}$|$57.01_{0.69}$|$56.27_{0.57}$|$55.04_{0.90}$|$54.45_{0.80}$|$54.09_{0.41}$|$53.15_{0.91}$|$52.26_{0.97}$|$47.00_{1.53}$|$25.80_{2.08}$ This result shows that when the feature extractor is randomly generated, if we train the head classifier with more steps while keeping the same total steps, the performance will decrease compared to directly finetuning the whole network. Our analysis of this is that, the feature extractor is not encoded with any prior and it is more optimal to allocate privacy budget towards updating the full network in the whole training procedure to learn more information. We will add this result and the discussion in the draft. As a final point, because the discussion period is drawing to a close, we are interested in hearing any further concerns you may have.
Decision
Accept (spotlight)