Gaussian Differential Privacy on Riemannian Manifolds

We develop an advanced approach for extending Gaussian Differential Privacy (GDP) to general Riemannian manifolds. The concept of GDP stands out as a prominent privacy definition that strongly warrants extension to manifold settings, due to its central limit properties. By harnessing the power of the renowned Bishop-Gromov theorem in geometric analysis, we propose a Riemannian Gaussian distribution that integrates the Riemannian distance, allowing us to achieve GDP in Riemannian manifolds with bounded Ricci curvature. To the best of our knowledge, this work marks the first instance of extending the GDP framework to accommodate general Riemannian manifolds, encompassing curved spaces, and circumventing the reliance on tangent space summaries. We provide a simple algorithm to evaluate the privacy budget $μ$ on any one-dimensional manifold and introduce a versatile Markov Chain Monte Carlo (MCMC)-based algorithm to calculate $μ$ on any Riemannian manifold with constant curvature. Through simulations on one of the most prevalent manifolds in statistics, the unit sphere $S^d$, we demonstrate the superior utility of our Riemannian Gaussian mechanism in comparison to the previously proposed Riemannian Laplace mechanism for implementing GDP.

Paper

Similar papers

Peer review

Reviewer vrR57/10 · confidence 4/52023-06-30

Summary

The paper extends definition of Gaussian DP to General Riemannian Manifolds and proposed Riemannian Gaussian mechanism that shows to achieve GaussianDP. Paper also develops MCMC procedure for Riemannian Manifolds with constant curvature.

Strengths

* Extending GaussianDP to Riemannian Manifolds * MCMC procedure for Riemannian manifolds of constant curvature.

Weaknesses

The experimental setup is extremely limited. * **Baselines** The experiment compares the $\epsilon$-differential privacy mechanism (Riemannian Laplace mechanism) with the $(\epsilon, \delta)$-differential privacy mechanism (GDP mechanism), but it fails to include baselines that use $(\epsilon, \delta)$-mechanisms. It is necessary to include comparison with the following: * DP-Riemannian Optimization ($\epsilon, \delta$)-differential privacy mechanism (source: https://arxiv.org/abs/2205.09494) Additionally, it is recommended to compare with * Riemannian K-Norm Gradient mechanism ($\epsilon$-DP) (source: https://arxiv.org/pdf/2209.12667.pdf). * **Experiments used only one manifold and of max dimension 3 for n=10 samples:** Experiments were solely carried out on 2-dimensional ($S^1$) and 3-dimensional ($S^2$) manifolds. It is essential to extend the experimentation to include at least one additional manifold, such as a hyperbolic manifold, which represents another constant curvature manifold. Furthermore, it is advisable to introduce variations in the parameters $n$ and $d$ during the experiments. Related work : Also related work on DP-Optimization Improved Differentially Private Riemannian Optimization: Fast Sampling and Variance Reduction , TMLR 2023, (https://openreview.net/pdf?id=paguBNtqiO)

Questions

NA

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

NA

Reviewer 3t867/10 · confidence 4/52023-06-30

Summary

The authors consider the problem of achieving Gaussian Differential Privacy (GDP) on Riemannian manifolds. After defining what this concept means, they provide a mechanism capable of achieving GDP via a generalization of the Gaussian to manifolds. The major challenge in implementing the approach is in finding a suitable scaling parameter, $\sigma$, for the noise. The authors define several equations that must be solved to find $\sigma$, each which become more tractable as additional structural assumptions on the manifold are made.

Strengths

This is a timely paper as there has been an increased interest in manifolds in statistics/machine learning. There have also been several recent works that consider privacy within these contexts as well. The Gaussian mechanism is one of the most popular tools for achieving privacy, so it is quite natural to consider it in the context of manifolds. The extension of the Gaussian mechanism to manifolds is much more challenging than the Laplace.

Weaknesses

Implementing the mechanism seems quite challenging. Just to find the scaling parameter $\sigma$ is difficult in general, plus one must do something like MCMC to actually sample from the mechanism. This limits the practical impact of the paper.

Questions

Def 3.1: I would emphasize that this is a known equivalence from the Dong et al paper. I think the authors should also explain a bit more why this definition is easier to work with on manifolds than the original gaussian dp. In general, I’m not convinced that it truly is. The GDP definition is simply a statement about the tradeoff between type 1 and type 2 errors, it really has nothing to do with the underlying space. So the central challenge isn’t defining GDP for manifolds, but defining a mechanism that achieves it. “In contrast to the task of releasing manifold-valued private summary, Han et al. [2022] considers a different scenario where the private summary resides on the tangent bundle of Riemannian manifolds and extends (ε, δ)-DP and its Gaussian mechanism to general manifolds." This seems a bit inaccurate since their intention was not to privately release the gradient but rather use the private gradient for the private summary which resides on the manifold. I think the wording here should be clarified. Minor— I don’t think the authors ever actually define $\Phi$.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

NA

Reviewer 12f96/10 · confidence 3/52023-07-04

Summary

To obtain manifold-value statistic with the Gaussian differential privacy (GDP), the authors extend the GDP from Euclidean spaces to general Riemannian manifolds. Then they proved that their proposed Riemannian Gaussian distribution make GDP tractable. Furthermore, they proposed an efficient numerical algorithm to compute the privacy budget μ on the manifolds of one dimension, especially on the manifolds with constant curvature. At last, they provided some simulation experiments to illustrate performance of their algorithms. Due to the importance of GDP among the variants of DP, it is meaningful to generalize GDP to Riemannian manifolds. They solved this question partially. Considering the importance of this question, the paper is worth to be accepted. Also, this paper is organized clearly except some typos. I have checked carefully some important details of proofs, so the results should be reliable.

Strengths

1. They research the extension of an important variant of DP—GDP and prove that they can achieve GDP on Riemannian manifolds by using Riemannian Gaussian distribution. 2. Following previous works, their results make sense.

Weaknesses

Some typos influence the readability of this paper. For example, ‘compliant’, not ‘compliment’ in subsection Main Contributions; and $N_m(\eta,\sigma^2)$ is correct, not $N_m(\mu,\sigma^2)$. 2. For the case of manifolds of dimension $d>1$, they make a too strong assumption that the manifold with constant curvature to design an efficient numerical algorithm. To the best of my knowledge, this assumption can’t encompass enough manifolds (I can only think of Euclidean space, the sphere and Lobachevskij space). The motivation of this assumption is not strong in practice.

Questions

1. This paper mentioned all hypothesis testing-based privacy definitions converge to the guarantees of GDP in the limit of composition and when the dimension of the privatized data approaches infinity, a large class of noise addition private mechanisms is shown to be asymptotically GDP. Are these results valid on Riemannian manifolds? 2. It is hard to solving the case of manifolds of dimension $d>1$, which make us use numerical methods to compute the integral. This work uses MCMC to approximate the integral, how about other numerical integration methods? 3. They claimed that their work is the extension of an important variant of DP, so this work may be valuable in practice. Thus they should explain the point clearly in the introduction and provide some real world numerical examples.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

See "weakness".

Reviewer j3sD4/10 · confidence 4/52023-07-05

Summary

This work studies an extension of a Gaussian privacy mechanism on Riemannian manifolds, so as to extend the notion of mu-GDP for estimators with given global sensitivity. The idea is to sample a random point on the manifold with a Gibbs density (wrt to the Riemannian volume) whose potential is proportional to the squared distance to the given estimator (that would simply be a Gaussian density on a Euclidean space). It is well known that in order for such a distribution to be well defined, it is enough to assume that the sectional (or, equivalenty, Ricci) curvature of the manifold is bounded from below. The main challenges, then, reside in : (1) tuning the temperature parameter of the Gibbs distribution so as to obtain mu-GDP, and (2) sample from such a distribution. While giving general formulae for the choice of the temperature parameter, the authors study the case of homogeneous manifolds, as well as model manifolds with constant curvature, also providing simulation studies. Differential privacy on non-linear spaces, in particular Riemannian manifolds, has not been studied a lot yet, and this paper aims at providing new, simple insights.

Strengths

• The question tackled in this paper is important, given the increasing need of dealing with non-linearity in data. • The paper is written clearly and the main text is easy to follow.

Weaknesses

• While the paper addresses an important question, the execution remains shallow. Laplace and Gaussian mechanisms can be very easily extended to Riemannian manifolds, and the computations that yield to determine the calibration of their parameters are very similar to the Euclidean case. One challenge that is barely addressed here (see the very brief remark just before Definition 4.1) is the understanding of the connexion between these parameters and the geometry (in particular, curvature bounds) of the ambient manifold. Instead, the authors choose to focus on very simple (and perhaps not realistic in most applications) manifolds, where everything becomes much simpler. Hence, I think that a big, if not essential, part of the picture is missing in this work. • The presence of Definition 3.1 (mu-GDP mechanisms on manifolds) is obscure to me: A definition is already given in Definition 2.2, and I do not see why it should be different on manifolds. Should Definition 3.1 actually be a theorem? I mean that one should check that M is mu-GDP in the sense of Definition 2.2 if and only if it is (eps,delta(eps))-DP for all eps>0, with delta(eps) given by the formula that appears in Definition 3.1 (something similar to Theorem A.1). • Some proofs seem either wrong or incomplete to me, here are some pointers: - One line after Theorem A.1, this theorem is interpreted using the O(eps^2) notation, which completely forgets the dependence in mu. Hence, the whole proof of Theorem 3.1 forgets mu and is, therefore, imprecise/incomplete. - I did not understand the first line of the proof of Theorem 3.1. Also, there, what is x (it does not appear in the formula)? - In Equation 4, I think that the normalizing constant Z_{eta,sigma} is missing in the upper bound. Some minor remarks: • Line 92, there is a typo. • Line 99: « the » should be replaced with « a ». • Line 237: Remove « satisfies » • Every citation should come to a pointer to a specific definition or result, especially when the citation refers to a whole book (e.g., Dudley, 2002)! • In Theorem A.1, what is the ‘t’ in the subscript of mu_t? • Also in Theorem A.1, « privacy profile » has not be defined anywhere.

Questions

See paragraph on weaknesses.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

1 poor

Presentation

3 good

Contribution

1 poor

Limitations

NA

Reviewer 12f92023-08-14

Thanks to your reply. Despite answers have been addressed, the weakness on the constant curvature assumption remains, and this reduces the technical difficulties and interests to applications.

Authorsrebuttal2023-08-14

We appreciate your prompt response. While acknowledging that our approach relies on the constant curvature assumption, we believe it is a substantial advancement over prior research as we explain next. Previous studies have typically followed the following two approaches: 1. Focus on specific manifold and metric: For instance, in the work by [1], emphasis was solely placed on the SPD space utilizing a log-Euclidean metric. 2. Dependence on tangent spaces: As illustrated in [2], the approach revolved around tangent spaces and their utilization in resolving private empirical risk minimization issues through private gradient implementation. In contrast, our research improves upon both of the aforementioned aspects: 1. Our method is more general as the assumption of constant curvature encompasses more than one particular manifold. 2. Our method works directly on the manifold itself without relying on computation on the tangent spaces, and thus it's more flexible since we do not need to restrict to solve problems through the use of private gradient. [1] S. Utpala, P. Vepakomma, and N. Miolane. Differentially Private Fréchet Mean on the Manifold of Symmetric Positive Definite (SPD) Matrices with log-Euclidean Metric. Transactions on Machine Learning Research, 2023. [2] A. Han, B. Mishra, P. Jawanpuria, and J. Gao. Differentially Private Riemannian Optimization. arXiv preprint arXiv:2205.09494, 2022.

Reviewer j3sD2023-08-16

Thank you for the detailed rebuttal. I am willing to increase my score from 3 to 4, but I still believe that this work misses precise quantitative connections between the privacy mechanism and the geometry of the space, which is a major limitation, in my opinion.

Authorsrebuttal2023-08-16

We greatly appreciate your decision to increase the score. While we fully recognize the significance of establishing precise quantitative connections between the privacy mechanism and the geometry of the space, addressing this challenging task deserves more research efforts in future works. To the best of our knowledge, no previous work has attempted to tackle this fundamental problem. Nevertheless, our novel framework paved the way for future investigation. In particular, the first essential step is to extend GDP (approximate DP) and its privacy mechanism to general manifolds. However, previous works have focused on specific manifolds, e.g., the work in [1], therefore making it impossible to study the connection between its geometry and the privacy mechanism. The reason is that we need to observe how the privacy mechanism responds to different geometries (e.g., a change in curvature) in order to make the connection. For example, if the privacy mechanism were extendable to both spherical and hyperbolic spaces, we could then examine how the calibrations of the noise parameter $\sigma$ differ between these two spaces. In contrast, we extend GDP to general manifolds (definition 3.1) and provide a way to extend the Gaussian mechanism to general manifolds (Theorem 3.2). Additionally, we design a simple algorithm to efficiently calibrate the noise parameter $\sigma$ for constant curvature spaces. This is an important first step. That is, once the privacy mechanism has been extended to a wide range of manifolds and its parameter $\sigma$ can be efficiently calibrated, we can then start further investigation of the connection. However, considering that it is beyond the scope of our current work, we intend to tackle this problem in future research. [1] S. Utpala, P. Vepakomma, and N. Miolane. Differentially Private Fréchet Mean on the Manifold of Symmetric Positive Definite (SPD) Matrices with log-Euclidean Metric. Transactions on Machine Learning Research, 2023.

Reviewer 3t862023-08-18

Thank you for the response. I get your point about the hypothesis testing and the underlying geometry. I would acknowledge that it is at least a little ambiguous and your approach makes the application to manifolds cleaner, but it isn't obvious that the underlying space needs to be considered when defining the type of space (the underlying space plays no role in the privacy definition for pure DP, approximate DP, etc). The f-DP framework just says that the function f is tradeoff between type 1 and type 2 errors, and those errors could come from anywhere. Regardless, I still consider this a minor (though interesting) point.

Authorsrebuttal2023-08-19

Thanks for the positive feedback. We understand that it isn't obvious that the underlying space needs to be considered when considering the hypothesis testing approach. To make the use of Definition 3.1 more natural and obvious, we will add more explanations in the revision. For convenience, we will add more explanations here as well to address any lingering concerns you might still hold. Consider a query result that resides on the unit sphere $S^2$. To privatize it in a GDP-compliant way, we have two options outlined below. 1. Commonly referred to as the extrinsic approach, we can simply disregard the unit sphere $S^2$ and consider the query result lies on $\mathbb{R}^3$ instead. The rest is trivial as we can apply the existing framework of GDP on Euclidean space. However, the major shortcoming of this approach is the absence of assurance that the privatized query will still reside on $S^2$. 2. To address this issue, we need to take $S^2$ into consideration. This is referred to as the intrinsic approach. To guarantee that a GDP-compliant mechanism $M$ will produce results within $S^2$, we need to ensure the distribution of its outcome is defined entirely on $S^2$. Therefore, the hypothesis testing approach relates to the problem of distinguishing between $M(\mathcal{D})$ and $M(\mathcal{D}')$, two distributions on $S^2$ instead of $\mathbb{R}^3$. To determine the optimal type 1 and type 2 trade-off $T(M(\mathcal{D}), M(\mathcal{D}'))$ between these distributions is difficult. For instance, computing the ratio between two likelihoods can be quite involved due to the complexity of the normalizing constant (e.g., the dependence of $Z(\eta, \sigma)$ on $\eta$). This is where Definition 3.1 comes in, using the connection with $(\varepsilon, \delta)$-DP, we can establish GDP on $S^2$ purely based on the notion of Borel probability measure.

Authorsrebuttal2023-08-21

Dear Area Chair and Reviewers

After the initial submission, we received many constructive and insightful feedback from the reviewers. We express our gratitude to the reviewers for acknowledging the importance of extending GDP and the Gaussian mechanism to general manifolds, even though it is primarily applicable to a specific subset of manifold structures. As we proceed, we will outline some of the key concerns that have arisen in the reviews. 1. One of the recurring concerns that reviewers is the use of Definition 3.1. We have addressed this by giving a concrete example in our reply to reviewer 3t86. 2. In our response, we have extensively clarified the aspects of confusion raised by Reviewer j3sD regarding the proof of Theorem 3.1, providing additional explanations and background results. We extend our sincere gratitude for reviewing our paper. We value the engaging discourse and invaluable feedback you have provided.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC