Unbounded Differentially Private Quantile and Maximum Estimation

In this work we consider the problem of differentially private computation of quantiles for the data, especially the highest quantiles such as maximum, but with an unbounded range for the dataset. We show that this can be done efficiently through a simple invocation of $\texttt{AboveThreshold}$, a subroutine that is iteratively called in the fundamental Sparse Vector Technique, even when there is no upper bound on the data. In particular, we show that this procedure can give more accurate and robust estimates on the highest quantiles with applications towards clipping that is essential for differentially private sum and mean estimation. In addition, we show how two invocations can handle the fully unbounded data setting. Within our study, we show that an improved analysis of $\texttt{AboveThreshold}$ can improve the privacy guarantees for the widely used Sparse Vector Technique that is of independent interest. We give a more general characterization of privacy loss for $\texttt{AboveThreshold}$ which we immediately apply to our method for improved privacy guarantees. Our algorithm only requires one $O(n)$ pass through the data, which can be unsorted, and each subsequent query takes $O(1)$ time. We empirically compare our unbounded algorithm with the state-of-the-art algorithms in the bounded setting. For inner quantiles, we find that our method often performs better on non-synthetic datasets. For the maximal quantiles, which we apply to differentially private sum computation, we find that our method performs significantly better.

Paper

Similar papers

Peer review

Reviewer QwHj6/10 · confidence 3/52023-07-01

Summary

The paper considers the problem of differentially private computation of quantiles for the data. The core idea is to leverage the ‘AboveThreshold’ algorithm as a subroutine to guess the desired item value such that the rank of the item value is about qn for target quantile q. Authors also discuss the application of this method for mean and sum estimation where the proposed algorithm can help clipping the item values.Experimental results indicate improvement for inner quantiles over Page, Age, Workhours dataset.

Strengths

Quantile estimation is very important problem with real-world applications. This paper presents a straightforward application Abovethreshold differential privacy techniques. For applications, the simplicity in algorithms is greatly valued. The paper is well written and very accessible for the general audience.

Weaknesses

One major concern is the requirement of O(n) passes through the dataset. With a large volume of data, n passes seem to be formidable. The experimental section can be strengthened by using more data points and considering query latency. Related works on quantile approximation are missing. Gillenwater, Jennifer, Matthew Joseph, and Alex Kulesza. "Differentially private quantiles." They present a DP-algorithm with exponential mechanism that estimates all m quantiles in $O(mnlog(n)+m^2n)$ time. Zhao, Fuheng, et al. "Differentially private linear sketches: Efficient implementations and applications." && Pagh, Rasmus, and Mikkel Thorup. "Improved utility analysis of private countsketch." They present a DP-algorithm using count sketch to estimate all quantiles in one pass with gaussian mechanism.

Questions

1) Can the algorithm improve to O(logN) pass over the data with something like exponential search? 2) 1000 data points in the experiments are very small. Also maybe consider including some results on the query latency. For example the time to find the median.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

No potential negative societal impact.

Reviewer VnsR4/10 · confidence 2/52023-07-04

Summary

The paper tries to improve upon Sparse Vector Techniques for differential privacy to obtain better estimates of the maximum and quantiles.

Strengths

The paper covers an important problem, finding a differentially private estimate of the max. This is often needed in one of the most common aggregations, i.e. sums, to clip and set a sensitivity. It promises less noise for the same privacy through an improvement to the sparse vector technique. I would be interested in reading it again if the proofs were correct. However,...

Weaknesses

I unfortunately cannot trust the results in the paper due to bad proofs. The proof for Lemma B.1 is incorrect though the conclusion may not be. For example, the statement in Line 557 of the supplement is incorrect. The statement says you will get the same answer _with the exact same noise_. You’ve fixed the randomness and there is no remaining randomness for the probabilities. You end up using v’ \neq v and v_k’ \neq v_k later. Furthermore, injectivity of the mapping v, v_k -> v’, v_k’ is not enough to show that the probabilities P(AboveThreshold(x) = O) and P(AboveThreshold(x’) = O) are the same for some output O. For example x -> x/2 is an injective mapping from [0,1]. But for a uniform distribution, the probability of the image is ½, not 1. In this case, the mapping happens to be a shift, so the Jacobian of the map is the identity, which actually is enough since the determinant is 1. But the current proof is too much of a mess to know if I can trust the theoretical claims in the paper. Please fix the reference in Lemma 3.1. The Thm 4, 5 in the VLDB version are show that algorithms are _not_ DP.

Questions

Please fix the proofs! It is hard to tease out your contributions from the contributions section. Most of the contributions section belongs in the introduction. The contributions should be a succinct description of what _you_ contributed, not about where it can be used.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

1 poor

Presentation

2 fair

Contribution

2 fair

Limitations

Yes

Reviewer nQ9t7/10 · confidence 3/52023-07-07

Summary

The authors propose a differentially private algorithm for differentially private quantile estimation that can handle unbounded data setting. The authors revisited the AboveThreshold subroutine in Sparse Vector Technique and proved new privacy guarantee results. The authors can verify the asymptotic runtime complexity and demonstrate the utility through experiments. The method can have successful application to differentially private sum estimation.

Strengths

**Originality** The proposed method for finding private quantile is a creative use of an existing method for the application to a new domain. The authors proved new privacy results for the existing method. **Quality** The method can solve the problem. The theoretical analysis can verify the privacy and complexity of the proposed method. The authors can demonstrate the accuracy of the method through numerical studies. **Clarity** The problem is well-defined and clearly stated. The method can be clearly understood. **Significance** The problem being solved is important and as verified by the experiment the proposed method can improve accuracy for real-world data . The new privacy analysis to the Sparse Vector Technique can be of independent interest.

Weaknesses

The authors do not provide theoretical utility guarantee; though the experiment results look good.

Questions

1. You provide theoretical runtime complexity analysis. It would be of interest to compare the runtime in experiments to verify your asymptotic analysis. 2. From a quick search, there seems to be little literature in quantile estimation that addresses the unbounded setting. It would be great to highlight this by citing some papers that can handle unbounded data in Introduction/Related Work. You briefly mentioned this in your contributions, saying “commonly used algorithms can apply loose bounds to ensure data is contained within” and analyzing their drawbacks, but no citation is included.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

The proposed method works worse than the existing EMQ method for synthetic data. Can you offer any suggestions on how to improve your method’s accuracy for these well-behaving datasets?

Reviewer Qz1m7/10 · confidence 4/52023-07-07

Summary

The paper considers the problem of computing the quantile q of a dataset with differential privacy. The prior standard technique for this problem was the exponential mechanism with the loss of an output T being the difference between the number of examples less than T and qn. An issue with the exponential mechanism (and most mechanisms for quantile estimation) is that it requires the user to bound the range of possible values in the dataset. The authors propose an alternate algorithm which does not require a prior bound, and takes advantage of the AboveThreshold algorithm. They propose using the AboveThreshold on queries of the form "how many examples in the dataset are less than T?", where T is exponentially increasing, and return the value T that AboveThreshold reports has more than qn elements less than T. As part of the analysis, the authors give an improved analysis of AboveThreshold, showing it has improved privacy guarantees for monotonic queries when using Exponential or Gumbel noise; previously, this was shown for Laplace noise. The Exponential distribution has lower variance, so this allows them to get a lower-variance estimate without sacrificing privacy. The authors perform experiments to compare to the exponential mechanism, as well as the aggregate tree method. On synthetic datasets, the exponential mechanism performs slightly better, but otherwise the authors' result is better for quantile estimation. For sum estimation, when using parameters agnostic to the dataset, the authors' algorithm by far outperforms the baselines, due to not relying on a good prior range estimate.

Strengths

Overall I enjoyed reading the paper and feel it makes a substantial contribution to the field. Private quantile selection is an extremely important problem with a number of applications, including any setting where we wish to compute a private sum. The problem of needing a bounded range for these problems has always been a troublesome part of DP research and application, and having a seemingly robust approach that at least partially gets around this issue is thus of significant impact. The main idea in the paper is simple to describe, which makes it more likely to see adoption and further development in follow-up works, but at the same time requires new analyses to achieve competitive privacy-utility tradeoffs. Finally, I felt the paper was overall a pretty easy read, in part due to the elegance of the algorithmic ideas but also the quality and clarity of the writing.

Weaknesses

I think the only major weakness is that one still needs to choose beta appropriately. The authors mention this in the discussion in Appendix D, but this discussion is somewhat incomplete. Still, I believe that beta is a much easier parameter for a practitioner to choose appropriately than the range of the dataset, especially since the experiments show beta is somewhat dataset-agnostic, so this is a minor weakness.

Questions

Intuitively it seems like smaller beta is more likely to underestimate the quantile and larger beta is more likely to overestimate, is this correct? Also, for e.g. a bimodal dataset such as the Gaussian mixture 0.5 * N(0, 1) + 0.5 * N(Delta, 1), Delta >> 1, for estimating the 75th percentile, how would you expect UQE to compare to EMQ? (with a good or bad range estimate)

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

Yes, in the appendix the authors discuss the main limitation/weakness. N/A on negative societal impact.

Reviewer Qz1m2023-08-14

Thank you for the response, and the detailed answers to my questions. As of now I'm still of the opinion the paper deserves acceptance, as it seems the concerns of reviewer VnsR are likely to be addressed during the rebuttal process (although I will of course plan to stay up-to-date on that thread to confirm this)

Authorsrebuttal2023-08-14

We again thank the reviewer for such a detailed review! We'll be more than happy to answer any questions the reviewer may ultimately have on the thread clarifying and updating the proofs. We want to make sure to provide any and all helpful follow-up details so that hopefully all the reviewers are in agreement on the correctness of our theoretical work.

Reviewer nQ9t2023-08-15

Dear authors, Thanks for your responses. I recommend you include your response to the limitation question (and/or) some supporting experiments somewhere in the main text and supplementary of the paper. I would be happy to see if you have any new results that support your discussion. I don’t think you have addressed my concerns in **Weaknesses**. Do you have a utility guarantee similar to Theorem 3.3 of your cited work *Kaplan et al. (2022)* or in whatever error metrics that tells how your estimations differ from the true quantiles? I believe this is needed to make the paper more theoretically complete. Following your discussion with reviewer VnsR, could you please also send an updated proof as a stand-alone pdf? I would be happy to check the correctness and provide some comments if you can provide it before the rebuttal ends. Thanks again

Authorsrebuttal2023-08-15

Thanks again to the reviewer for all the detailed suggestions and feedback! We also greatly appreciate the reviewers willingness to look over the minor clarification and updates to our proof. According to the rules, only figures and tables can be included in the pdf, so we've contacted the AC about making an exception. The proof updates can also be found in our rebuttal but we can definitely understand the pdf formatting is easier to review and we'll hopefully be able to send a pdf of the proof very shortly. We will also be sure to include our limitations discussion in future versions and appreciate the reviewers suggestions. We do not have sufficient theoretical evidence to make concrete claims, but we will certainly add our conjectured intuition and some supporting experiments that may be highly useful to practitioners and future work. $\textbf{Regarding the theoretical utility guarantees}$ We apologize for failing to address the weakness mentioned by the reviewer. We can improve the $(\alpha, \beta)$ guarantees of SVT in Theorem 3.24 of [1] (and cited in [2]) with our new results using Gumbel or Exponential noise. We will be sure to include these improved guarantees in future versions of our work and thank to reviewer for this suggestion. In particular, if we run AboveThreshold with Gumbel noise and $\epsilon_1 = \epsilon_2 = \epsilon/3$ (which gives $\epsilon$-DP for AboveThreshold) then we can improve the guarantees of Theorem 3.24 in [1] to $ \alpha = 3 ( \log(k) + \log(1/\beta)) / \epsilon$. In the theorem and algorithm of [1] it is assumed that the sensitivity $\Delta=1$ which we will also apply for simplicity but can be generalized. $ $ Proof sketch that we'd be happy to elaborate upon: According to Definition 3.9 of [1], we want to show that the probability of halting before query k is at most $\beta$. Let $\nu$ be the noise added to the threshold hold, let $\nu_i$ be the noise added to query $f_i$, and let $\tau = max_{i\leq k} f_i(x) + \nu_i$. We know AboveThreshold will not halt before $k$ if the noisy threshold is above all noisy queries $i \leq k$. So we want to show that $Pr(T + \nu > \tau) \geq 1 - \beta$. It is known that adding Gumbel noise and selecting the greatest noisy value is equivalent to the exponential mechanism (see Lemma 4.2 of [3] for a proof). Therefore, given that our gumbel noise parameter is $3 / \epsilon$ we have $Pr(T + \nu > \tau) = \exp( \epsilon T / 3) / (\exp( \epsilon T / 3) + \sum \exp(\epsilon f_i(x) / 3))$. We then apply the assumption from Theorem 3.24 of [1] that $f_i(x) \leq T - \alpha$ for all $i$ and our goal is to show $\exp( \epsilon T / 3) / (\exp( \epsilon T / 3) + k \exp(\epsilon (T- \alpha) / 3)) \ \geq 1 - \beta$. Some straightforward algebraic manipulation then shows that $ \alpha = 3 ( \log(k) + \log(1/\beta)) / \epsilon$ satisfies this inequality. $ $ The leading constant for this $\alpha$ guarantee can also be improved identically to 2 for the monotonic setting as we can set $\epsilon_1 = \epsilon_2 = \epsilon / 2$ to achieve $\epsilon$-DP. Further we can show the same improvement for adding exponential noise which we would be happy to provide a proof as well. We think that these results could also likely be applied to give theoretical guarantees for the quantile algorithm, but leave such analysis to future work. We would like to note that our empirical analysis was of the exact same datasets and experimental setup as in [4] so as to give fair practical comparison. $ $ [1] "The Algorithmic Foundations of Differential Privacy", Dwork and Roth [2] "Understanding the Sparse Vector Technique for Differential Privacy" Lyu, Su, and Li [3] "Practical Differentially Private Top-k Selection with Pay-what-you-get Composition" Durfee and Rogers [4] "Differentially Private Approximate Quantiles", Kaplan, Schnapp, Stemmer

Area Chair yBWF2023-08-17

Anonymous link to updated proof.

I have uploaded a PDF of the author's revised proof (discussed in their rebuttal) to the following anonymous link: https://drive.google.com/file/d/1-tG7AwFugTTCRHpObidWzAtCbJdTOSBK/view?usp=sharing Please let me know if you have any trouble accessing the file. Thanks, Your AC

Reviewer QwHj2023-08-18

Thanks a lot for providing more information on the work such as revised proof and detailed responses to reviewers. I slightly raised my score.

Authorsrebuttal2023-08-18

We sincerely thank the reviewer for considering our rebuttals and raising their score! We apologize again for not clearly stating our runtime, which is asymptotically equivalent in the data size to quickselect, the fastest non-private quantile algorithm. We appreciate the reviewer bringing this confusion to our attention and we will be sure to make the efficiency of our algorithm clear in future versions of our work.

Reviewer VnsR2023-08-20

Thank you for your responses. While I feel this is better addressed in another review cycle, I've slightly increased my score, and lowered my confidence. I hope you found the review helpful in improving the paper.

Authorsrebuttal2023-08-20

We are incredibly grateful to the reviewer for pointing out the easily added but important technicality, which has helped improve the clarity of our proofs. We also appreciate the reviewer revisiting their score and confidence!  While we are very embarrassed to have missed this technicality in the first place, we strongly feel that these types of minor technical updates and clarifications are quite common within the rebuttal period, and are in fact the main purpose of the rebuttal period. Our updates are only minor revisions where the underlying logic remained unchanged and have no effect on the main body of the paper. The notion that our proofs were fundamentally flawed entirely stemmed from our not specifying that $\nu_{i<k}$ denoted all $\nu_i$ such that $i<k$. We apologize for this omission, but hope that the entirety of our work will not receive biased judgement that major revisions were needed as a result of such a tiny oversight.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC