Noise-Aware Differentially Private Regression via Meta-Learning

Summary

The paper proposes a meta learning gaussian dp algorithm. It uses the same framework as ConvCNP but replaces the encoder during train and meta test with a noisy encoder obtained by adding gaussian noise. The paper is not particularly well written and I have trouble following the notation in the main paper and appendix. I have significant doubts about the correctness of the method due to conflation between continuous results (the integral of a gaussian kernel) with discrete results (the sum of the kernel at discrete points). Note: after feedback, the authors addressed my concerns about correctness.

Strengths

The paper considers the use of meta learning to avoid an unfortunate trend in the dp literature which assumes the existence of large scale non private data with a similar distribution to a private dataset. Meta learning is a convenient possible alternative if one can create a good enough synthetic dataset. Assuming everything is correct, the technique outperforms some prior work on simple synthetic and real datasets.

Weaknesses

The experiments consider alternatives that use similar methods (GP or DP-SGD) but not alternatives that use different methods for similar problems (regression/kernel density estimates, clustering, etc.). I believe the technique would outperform the alternatives anyway, but their omission is noteworthy. The experimental datasets are very simple and not comparable to the ones where public datasets are often assumed (images and NLP). Since the technical contribution lies in replacing r with a noisy r, there should be more challenging experiments and discussions on how to create simulated data for them. A few typos: - line 94, r should be theta - line 130, missing word after predictive - line 224 references eq 6 instead of 7 The delta value of 10^-3 is large compared to the literature. Setting it at 1/N puts it into the well-known privacy violating regime where one can return a random record with no noise, violating the privacy of someone with probability 1. The paper seems to be taking too much credit for minor things, like using Gaussian DP as a drop-in replacement in the work of Hall et al. (literally taking a theorem that was meant as a drop-in replacement of prior formulas and using it as a drop-in replacement for those formulas). That is not a contribution. The most important issue for last. I am not convinced that the algorithm is differentially private at the claimed parameter levels. The paper is missing a detailed pseudocode that puts everything together, but my understanding is that nothing changes except that wherever r is needed, a noisy version is used instead. The privacy properties should be provable from first principles (without relying on abstract kernel properties or an RKHS). Plug in the form of the kernel you are using and directly compute the sensitivity accounting for all of the discretized points being used. The reason I am asking for this is because the sensitivity calculation detailed in the appendix appears to be incorrect (it is not very well written and the notation is not explained well, so I am not 100% sure). The key to the proof seems to be that the gaussian kernel integrates to 1, but this is used to replace a discrete summation (i.e., \int_{-\infty}^\infty exp(-(x-y)^2))~dy is very different from \sum_{i=1}^n exp(-(x-y_i)^2)). The summation can diverge depending on what y_i are chosen. I would like to see a detailed privacy proof (with carefully explained notation) that explicitly uses the discretized point set (explaining exactly how the discretized points are chosen) and an explicitly defined covariance matrix.

Questions

I would like to see a detailed privacy proof (with carefully explained notation) that explicitly uses the discretized point set (explaining exactly how the discretized points are chosen) and an explicitly defined covariance matrix. For more challenging problems, such as text and images, how would one create simulated data so that DPConvCNP would be competitive?

Rating

6

Confidence

4

Soundness

3

Presentation

2

Contribution

3

Limitations

The authors addressed limitations.

Summary

This paper equips meta learning framework with DP guarantees. Specifically, datasets are split into context and target subsets respectively, where an encoder learns good representations from abundant context data, and is able to generalize to a limited amount of target data that may be sensitive. Since the output embedding (function) of the encoder is a sum of kernel functions indexed by the context dataset, so functional DP mechanisms are considered to protect the individual privacy in the dataset. Particularly, the authors extend the classical functional DP mechanism to Gaussian DP (GDP), which has theoretical benefits compared to alternatives. Experiments in both synthetic data and real data show its effectiveness.

Strengths

1. Theoretically, the authors extend Gaussian DP to functional outputs, which may pave the way for follow-up works in related fields. 2. Empirically, the experiments are complete and extensive, i.e. on both synthetic and real tasks with comparison to existing works. 3. The paper is well-organized and easy to follow. Figure illustrations are clear and informative.

Weaknesses

I am not familiar with the meta-learning framework (and related works), so it's likely I misunderstood some parts and had confusion about the meta-testing part. Normally, a DP classifier, regressor, or generative model only introduces noise perturbation in the training (e.g. DP-SGD). Once the training is complete, the trained DP model can be applied to test data without worrying about breaching DP guarantees, as ensured by the post-processing theorem. In this work, however, the noise is injected in both training and test stages, where the authors explained that it accounts for mismatch between training and test. This paradigm does not look optimal to me. So here I have a few questions: 1. Can you explain why adding noise to the test stage is necessary? For example, compared to pretraining on the synthetic data then DP fine-tuning on private data, what is the benefit of the current method? 2. It looks to me that if we need to add noise in the test stage, we need to accumulate the total privacy budget $\epsilon$ as more test data are coming in. Is it true? If so, how do you determine the $\epsilon$ in the experiment (e.g. eps=1 or 3 in Fig 5)?

Questions

1. I am not familiar with the neural process either. By reading the paragraphs starting at line 118, I don't see how it is different from a normal neural network. Particularly, how it is able to produce well-calibrated prediction, and how this is evaluated in the experimental results? 2. For the conditional NP, I don't see where is the conditional input, or what does the conditional mean here? 3. In Figure 4, I wonder how these lines are drawn. Do you use any analytical forms to calculate, or some libraries/packages to compute? Also, are they converted into the same $(\epsilon, \delta)$-DP notion? Because the $\epsilon$ in different DP notions are not the same (although they share the same notation).

Rating

7

Confidence

3

Soundness

4

Presentation

3

Contribution

4

Limitations

NA

Summary

Authors propose DPConvCNP, a meta-learning model with a functional DP mechanism. This model is a modification to the SetConv procedure, applying clipping and work from [Hall et al. 2013] with a tighter Gaussian mech. analysis from [Dong et. al 2022] to conduct a sensitivity analysis and privatize the algorithm. The authors then demonstrate empirically that DPConvCNP provides both performance and efficiency boosts relative to the baseline of applying DP-SGD to a neural GP process on both some synthetic tasks and the !Kung dataset.

Strengths

Overall: The paper follows a standard format for a contribution in differential privacy for machine learning: adapt an existing ML approach (SetConv) in a DP setting through sensitivity analysis and the application of a DP mechanism (in this case, an adapted version of the functional mechanism given by Hall et. al). Then, they show that this approach outperforms a Naive application of DP to some other standard method (in this case, applying DP-SGD to a standard neural implementation of a GP, which would be an obvious first attempt at the problem). In the Appendix, I reviewed the proofs and lemmas for Theorems 4.1 and 4.2, they checked out and the analysis is well presented. I did not spend time checking Proposition B.1; it seems like the main trick there is noticing that you can apply linearity of expectation over the grid. It would be good if another reviewer has reviewed this. S1. I find the quality of this submission’s presentation, from the intro to the supplementary appendix/proofs, to be exemplary. I commend the authors for the clarity of writing, notation, proofs and figures - it is refreshing, and worth highlighting. This is minus my minor nitpicks below, which will be easy to address. S2. I find the approach intuitive, and the analysis sound. The experimental results are compelling, and this method is clearly extensible, as the authors hint at in their limitations section. S3. The authors application of the [Dong et al.] result to the [Hall et al.] functional mechanism is indeed a nice contribution, and should be useful for future work.

Weaknesses

I am deferring more substantial weaknesses to the questions section - broadly, I would like to raise my score, given adequate answers to my questions. Nit: translation equivariant abbreviation TE should be introduced in the section where it is used heavily e.g. Section 3.2, not in the intro, along with a citation, which is confusing and gets lost for the reader. Nit: in Section 5, line 259, “we make the GP variational…” is maybe missing a word or two. Nit: In paragraph “Gradient based vs amortized…” in Section 3, the “on one hand…on the other hand” construction is difficult to follow, please restructure.

Questions

Q1 : My main question/concern with the work is *how* important is leveraging a powerful hyperparameter tuning library (like BayesOpt from Optuna) to adjust hyperparameters for the empirical performance of the proposed method DPConvSet? Having worked with DP-SGD variants (like Opacus’s private engine for optimizers), it’s clear to me that minor fluctuations in hyperparameters affect performance drastically. Recent work [Papernot et. al https://arxiv.org/pdf/2110.03620], [Mohapatra et. al, https://arxiv.org/pdf/2111.04906], and [Koskela et. al, https://arxiv.org/pdf/2301.11989] touches on the importance of private tuning. Can the authors discuss the delta between untuned and tuned versions of their algorithms, to be more “honest” about the effects of hyperparameter tuning on both their baseline and their method? An experiment (even anecdotal) would be nice, although isn’t necessarily required. Q1.5 : Related in Q1, in section 4.3, you include privacy parameters in the meta training of DPConvCNP (as far as I can tell) by re-parameterizing. Can you explain how this maintains the ($\epsilon,\delta$)-DP guarantee if the iterative meta-learning procedure is conditioned on prior trainings? Q2 : It seems like a missed opportunity to discuss the shortcomings of standard supervised learners on small datasets, even a simple standard private regression (many open source implementations available, for example https://diffprivlib.readthedocs.io/en/latest/modules/models.html#linear-regression). Perhaps comparisons between standard learners and meta learners is not worth it in the data scenarios you explore? It’d be helpful for me if you could discuss this (a light experiment if appropriate). Q3 : Can the authors justify their choice of privacy hyperparameters in the text? I acknowledge that $\epsilon$ of 1.0+ is definitely reported on in the literature and used practically, but it’s good to contrast this with $\epsilon < 1.0$, as this is the more theoretically comfortable private regime, in a strict sense. Adding results for this wider range of privacy parameters would strengthen the contribution.

Rating

7

Confidence

4

Soundness

3

Presentation

4

Contribution

2

Limitations

The authors do a very nice job of highlighting the limitations of their work, alongside broader impacts. This is greatly appreciated!