Optimal Private and Communication Constraint Distributed Goodness-of-Fit Testing for Discrete Distributions in the Large Sample Regime

We study distributed goodness-of-fit testing for discrete distribution under bandwidth and differential privacy constraints. Information constraint distributed goodness-of-fit testing is a problem that has received considerable attention recently. The important case of discrete distributions is theoretically well understood in the classical case where all data is available in one"central"location. In a federated setting, however, data is distributed across multiple"locations"(e.g. servers) and cannot readily be shared due to e.g. bandwidth or privacy constraints that each server needs to satisfy. We show how recently derived results for goodness-of-fit testing for the mean of a multivariate Gaussian model extend to the discrete distributions, by leveraging Le Cam's theory of statistical equivalence. In doing so, we derive matching minimax upper- and lower-bounds for the goodness-of-fit testing for discrete distributions under bandwidth or privacy constraints in the regime where the number of samples held locally is large.

Paper

Similar papers

Peer review

Reviewer 3Xr96/10 · confidence 3/52024-07-10

Summary

The paper focuses on the minmax rate for goodness-of-fit testing for discrete distributions under bandwidth and differential privacy constraints in a distributed setting, leveraging Le Cam’s theorem. The main distinction from previous literature lies in the consideration of the distributed setting.

Strengths

The paper is well-written, well-organized, and mathematically rigorous, with a clear exposition of the concepts using probability theory. The key contribution is the extension of the optimal rate for goodness-of-fit testing under differential privacy to the distributed case. However, I have concerns regarding this extension, which I detail in Weaknesses.

Weaknesses

I might be wrong, but I didn't see the difference between the distributed formulation in this paper and the central setting in other works (where data is available at one central location). The only exception is that the distributed formulation posits that each server adopts a local protocol (local privacy-mapping with bandwidth constraints). However, given that the raw data $X^{(j)},j=1,\ldots,m$ are i.i.d., wouldn’t the optimal rate be achieved when all local protocols are equivalent? If one server has a better protocol than others, the risk defined after line 190 is not minimized. If this is true, the question is what distinguishes the proposed distributed protocol from the privacy-preserving case in a central setting? For the latter, the min-max rate has already been derived, as seen in: "Local Privacy and Statistical Minimax Rates" "Robust Estimation of Discrete Distributions under Local Differential Privacy" "The Cost of Privacy: Optimal Rates of Convergence Performance Estimation with Differential Privacy" The paper needs to clearly articulate the differences and potential advantages of the distributed approach over these established results in the central setting. Without this clarification, the novelty and implications of the results may be unclear. My score will change based on the authors' response to this question.

Questions

See Weaknesses

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

The authors adequately addressed the limitations.

Reviewer 4qfs6/10 · confidence 2/52024-07-12

Summary

This paper explores distributed goodness-of-fit testing for discrete distributions under bandwidth and differential privacy constraints. The authors extend results from multivariate Gaussian models using Le Cam’s theory of statistical equivalence. They derive matching minimax upper and lower bounds for the goodness-of-fit testing problem when the number of samples held locally is large.

Strengths

The framework presented for extending goodness-of-fit testing from Gaussian models to discrete distributions is novel and addresses practical issues in federated learning scenarios. The derivation of matching minimax upper and lower bounds is rigorous and thorough, leveraging statistical equivalence effectively. The paper addresses key challenges in distributed settings, specifically under bandwidth and privacy constraints, which are crucial for modern applications in federated learning.

Weaknesses

The results rely on the assumption $md\log d/\sqrt{n}=o(1)$, which can be attained when the number of data is large. When $n$ is large, the setting naturally gets close to the Gaussian case, from which some existing tools can be leveraged. In this sense, the analyses presented in this paper are not too surprising. Moreover, the absence of empirical validation or simulations to demonstrate the practical performance of the theoretical results limits the impact of the findings in practice.

Questions

- How practical are the assumption $md\log d/\sqrt{n}=o(1)$ in real-world federated learning scenarios? Can you provide examples or case studies where these conditions hold? - The paper assumes large sample regimes for the derivation of the minimax rates. How would the results change if the sample size was not large? Are there any extensions or modifications of the theory to handle smaller sample sizes? - How does the proposed method compare with existing methods for distributed goodness-of-fit testing in terms of computational efficiency and communication overhead?

Rating

6

Confidence

2

Soundness

3

Presentation

3

Contribution

2

Limitations

The authors adequately addressed the limitations.

Reviewer aGfz7/10 · confidence 3/52024-07-12

Summary

This paper investigates the problem of Goodness-of-fit testing for multinomial distributions in federated learning in the case where the number of samples n per federated agent is large, and under a bandwidth or privacy constraint. Under certain scaling regimes, the authors characterize the number of samples needed for risk (defined as sum of Type I and Type II error) to vanish asymptotically. The authors provide an excellent description of Le Cam theory, and discuss it’s relation to their work.

Strengths

This paper is very well written. It is both thorough and rigorous, while still being approachable and well-written. Section 4 and 5 specifically are very nicely done and explain a complex idea simply. The results are interesting, and novel to my knowledge.

Weaknesses

The paragraphs after Theorem 1 and 2 respectively could be expanded somewhat. It would be interesting to see some discussion about the theorems in context. Small Comments 166: the the sample 303: citation missing

Questions

The title says "Optimal Private *and* Communication Constraint Distributed Goodness-of-Fit Testing". However, it seems that you consider "Optimal Private *or* Communication Constraint", as Theorem 1 and 2 consider these constraints separately. Can you comment on this? What would happen if you consider these jointly?

Rating

7

Confidence

3

Soundness

4

Presentation

4

Contribution

2

Limitations

I think this paper is thorough, and details the exact theoretical setting where the proven results apply.

Reviewer Nun84/10 · confidence 4/52024-07-17

Summary

The paper addresses distributed goodness-of-fit testing problems under user-level communication and local differential privacy (DP) constraints. In this scenario, each of the m users receives n samples, and a central server aims to test whether the underlying discrete distribution is uniform. This classical problem has been extensively studied in similar settings, such as when n = 1 or when the task is to estimate the underlying distribution. The main contribution of this paper is a tight characterization of the separation rates in the large-sample regime (where n is sufficiently large). The primary technical tool employed in the proof is Le Cam's statistical equivalence. By leveraging the equivalence between the Gaussian location model (GLM) and the multinomial model in large local sample regimes, the problem is reduced to the GLM, which has been addressed in previous works. While Le Cam's statistical equivalence is indeed a powerful and elegant method for establishing minimax rates, I find the novelty and contribution of this work to be somewhat limited. The main technical tools are well-established, and the primary theorem applies only to a restricted parameter regime. Additionally, the discussion of related prior works could be more comprehensive. Lastly, the presentation of the paper could be significantly improved. ============== Post rebuttal ========= I have updated my score accordingly, given that the authors claim that the techniques used in this work can be extended to prove lower bounds under interactive models.

Strengths

The main technical tool, the notion of statistical equivalence, seems to be a suitable and powerful method for addressing this class of problems, as it allows for the reduction of one problem to another.

Weaknesses

1. **Limited Contributions**: The main technical contribution of this paper appears to be limited, as the statistical distance between multinomial models and Gaussian location models is derived from prior works. Additionally, distributed testing for GLM under local DP/communication constraints is also well-established. The main theorems only establish the separation rates in a limited regime (i.e., large \(n\)). The proposed reduction, and consequently the statistical equivalence, apply only to the non-interactive setting, and it is well-known that establishing interactive lower bounds is significantly more challenging. 2. **Insufficient Discussion of Prior Works**: There are several relevant works that need further discussion. For instance, the "communication super-efficiency" effect, which has appeared in the "estimation" version of the problem in [1], is not discussed in the current draft. Although the paper is included in the references, I could not find the corresponding citation in the main text. 3. **Presentation**: The organization and presentation of the paper can be significantly improved. For example, the symbols \(\mathcal{Q}\) and \(\mathcal{P}\) are sometimes used to refer to multinomial and Gaussian models (e.g., in the proof of Theorem 1) without being explicitly stated, and at other times they refer to two general statistical models (e.g., in Section A.2), which is confusing. It would be helpful to explicitly specify these notations. There are also many minor typos and unclear notations that reduce readability. Some examples include: - Line 286: "identified" should be "identical"? - Line 303: unclear reference [] - Line 642: missing reference - Line 673: missing reference [1] Archaya et. al., "Distributed estimation with multiple samples per user: Sharp rates and phase transition".

Questions

N/A

Rating

4

Confidence

4

Soundness

3

Presentation

2

Contribution

2

Limitations

N/A

Authorsrebuttal2024-08-12

We would like to thank the the Reviewer (aGfz) again for the time and effort spent on evaluating our work and for their response to our rebuttal.

Reviewer Nun82024-08-12

Reply to the rebuttal

Thank you for your response. I agree that the regime where $n \geq 1 $ is indeed interesting. However, my primary concern remains with the technical novelty of the work. The statistical distance between multinomial models and GLMs has already been established, and distributed testing for GLMs under local DP is known too. As such, the main technical contribution appears to be bridging these two results. While I acknowledge that this involves some non-trivial extension to distributed settings, I still find the overall contribution somewhat limited. Regarding the interactive protocols, I also agree that, as in many other statistical tasks, interaction may not necessarily reduce the error. However, I believe there is currently no established lower bound for the $ n \geq 1 $ regime with sequential or blackboard interaction. In my view, developing such an interactive lower bound would significantly strengthen the work. Nevertheless, based on the current draft and the authors' response, it's unclear whether the framework developed in this paper can be extended to address this scenario. I appreciate the authors' attention to the references and the presentation issues. Please do include a discussion of [1] explaining why their techniques do not apply to the testing problem.

Authorsrebuttal2024-08-12

We thank Reviewer (Nun8) for their response. We are happy to hear that they agree the $n\geq 1$ setting is interesting and that the extension is non-trivial. We also find that there seems to be no known lower bound for blackboard or sequential protocols when $n > 1$ in the case of testing. In [2], the authors show that for $n=1$, there is no benefit to a sequential setup when compared to a shared randomness setup for uniformity testing with discrete data. For the Gaussian location model, we note that the proof of the shared randomness lower bounds of [11] can be extended to obtain the same (rate) results for sequential setups as well. Our (extended) machinery of the revised version then explicitly implies that there is no benefit of a sequential setups (outside of shared randomness) for uniformity testing with discrete data in the regime where $md \log (d) / \sqrt{n} = o(1)$. We will add details of this particular extension to the revised version of our paper. Admittedly, this does not mean that we can conclude anything concerning the benefit of a sequential setup in the regime(s) where $n > 1$ and $md \log (d) / \sqrt{n} \gtrsim 1$. Also, we note that for blackboard protocols, much less is known and we can indeed not exclude to benefit of a blackboard protocol. Proving lower bounds in the testing setting for blackboard protocols specifically is an interesting but difficult problem with many open questions in the literature. We will also include a discussion on why the technique of [1] does not yield an optimal lower bound in the testing setting in our revised version. We agree that including such a discussion is important to highlight the contribution of our work. We briefly sketch the reasons that this (and certain other) estimation techniques do not extend to the testing setting below. Let us start with describing a similarity: for both the estimation and the testing problem, lower bounds are typically proven by bounding a divergence measure between probability distributions, such as the chi-square divergence, mutual information or total variation [1,3,7,11,12,13,18,19]. For estimation problems such as the one considered in [1], or those of the examples considered in [3], it suffices to essentially "tensorize" the divergence, which, loosely speaking, breaks the problem into the ``sum of the local divergences''. This is essentially the role of Theorem C.2 in [1] (see also Theorem 1 and 2 in [3]), which bounds the total variation between elements of a perturbed family of probability distributions by the sum of the local conditional "scores", local conditional variances of the transcript densities or the local mutual information, see (16), (17) and (18) in [1]'s supplement. The loss due to a bandwidth constraints (in [1]) or privacy constraint (in [3]) is then captured by data processing arguments. For estimation, such tensorization bounds turn out to give tight lower bounds. We note that a lot more goes into the proof of [1] (e.g. Poissonisation, sub-Gaussian concentration), but the principle difference with estimation and testing is this kind of tensorization step. Similar tensorization arguments can also be found in other estimation problems such as [4,7], for the Fisher information and mutual information respectively. Such a "tensorization approach" does not yield tight bounds in testing problems. Using mutual information, [19] tries this tensorization approach for the testing problem, but they only recover the optimal testing rates when each server communicates only one bit ($b=1$). Another "estimation" approach tried for a goodness-of-fit testing problem can be found [18], which similarly obtain a lower bound for testing that is only tight for $b=1$, through a direct Taylor expansion of the likelihood (which can also be seen as tensorizing a divergence). The authors of [18] provide a detailed discussion of the shortcomings of the latter approach in Section 4 of their paper. To obtain tight lower bounds for the testing problem, the papers that successfully do so for $b > 1$ and privacy constraints (i.e. [11,12,13]) use techniques that differ greatly from the techniques employed in [1,3]. In [13], the authors use a combinatorial expansion of the likelihood that works specifically for $n=1$ in the multinomial model, which does not generalize to large numbers of observations. [11,12] circumvent the latter issue in the Gaussian setting by employing a Brascamp-Lieb inequality, an inequality from functional analysis. This inequality explicitly uses the Gaussianity of the log-likelihood explicitly. We hope that above additions to our work are satisfactory and we would like to thank you again for your consideration of our work. Additional references: [18] Acharya et. al., "Distributed signal detection under communication constraints" [19] Szabo et. al., "Optimal distributed composite testing in high-dimensional Gaussian models with 1-bit communication"

Reviewer 3Xr92024-08-14

Thanks for your response. My questions have been addressed. I'll raise the score.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC