Contrastive Moments: Unsupervised Halfspace Learning in Polynomial Time

We give a polynomial-time algorithm for learning high-dimensional halfspaces with margins in $d$-dimensional space to within desired TV distance when the ambient distribution is an unknown affine transformation of the $d$-fold product of an (unknown) symmetric one-dimensional logconcave distribution, and the halfspace is introduced by deleting at least an $ε$ fraction of the data in one of the component distributions. Notably, our algorithm does not need labels and establishes the unique (and efficient) identifiability of the hidden halfspace under this distributional assumption. The sample and time complexity of the algorithm are polynomial in the dimension and $1/ε$. The algorithm uses only the first two moments of suitable re-weightings of the empirical distribution, which we call contrastive moments; its analysis uses classical facts about generalized Dirichlet polynomials and relies crucially on a new monotonicity property of the moment ratio of truncations of logconcave distributions. Such algorithms, based only on first and second moments were suggested in earlier work, but hitherto eluded rigorous guarantees. Prior work addressed the special case when the underlying distribution is Gaussian via Non-Gaussian Component Analysis. We improve on this by providing polytime guarantees based on Total Variation (TV) distance, in place of existing moment-bound guarantees that can be super-polynomial. Our work is also the first to go beyond Gaussians in this setting.

Paper

Similar papers

Peer review

Reviewer rhCN8/10 · confidence 3/52023-06-28

Summary

Assuming that the data is generated from a full-rank affine transformation of a $d$-fold product distribution, one of which is divided into two groups by a halfspace. The authors address the problem of learning the halfspace by weighting the isotropized sample so that either the corresponding sample mean or the top eigenvector of the covariance matrix shifts along the normal direction to the halfspace.

Strengths

- Good presentation of theoretical results, with the proof of the main theorem summarized succinctly. - Interesting idea of using Descartes' Rule of Signs to single out a nonzero mean from weighted samples. - Experimental results agree with the theory. - In contrast to prior work which focuses on Gaussian data, the proposed method extends to data generated from logconcave distributions.

Weaknesses

- Real-world applications is still unclear to me. In what situation do we expect the set-up in Definition 1 to arise? The authors list the methods that benefit from the halfspace learning in Line 40-49, but where do these methods apply? - Related to the point above, the authors might want to test their method on real-world data. - The paper seems to have been reorganized but some notations are left unchanged---see questions below.

Questions

- The purpose of Lemma 1 and Lemma 2 is unclear. From a quick search in the supplement, they are not used to prove the main Theorem. And I personally think that Lemma 4 and Lemma 5 are more informative. Do the maximum of the two expectations shows up in the proof of Lemma 4? - What are $\mu_{\alpha_1}$ and $\mu_{\alpha_2}$ in Lemma 4? - What are $a'$ and $b'$ in the Proof of Lemma 1? - Bottom of Page 6: "Since $\hat{q}(x) - \hat{q}(-x) \geq 0$ for $x\geq b'$." looks like an incomplete sentence. - Proof Idea of Lemma 4: I do not understand the statement "$F'(0)$ has at most one root." and "$F"(0)$ has no root." - Do the time and sample complexities also depend on the distribution's tail decay rate?

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

The authors address some challenges of improving the sample complexity bound to linear, and extending the algorithm to more general distributions.

Reviewer YLvd5/10 · confidence 3/52023-07-03

Summary

This paper proposes a non-standard approach to the problem of learning halfspaces, where instead of labelled samples, the learner only receives unlabelled ones as well as strong prior information about the unknown distribution. In particular, the marginal distribution is assumed to correspond to some affinely skewed product distribution, with identical components except from one which is truncated outside some interval. The unsupervised distribution learning problem corresponds to a supervised learning problem when the labels are determined by a halfspace in the direction of the truncation. The main result shows that when the truncation region is wide enough but not too wide and either symmetric or bounded away from being symmetric, then the direction of truncation can be discovered efficiently from unlabelled samples. The algorithm exploits standard ideas from distribution learning (data normalization, PCA and sample re-weighing), but its proof involves careful application of appropriate tools.

Strengths

The paper proposes and studies an interesting connection between unsupervised and supervised learning. Moreover, I appreciated the technical work required to prove guarantees about the main algorithm, even though it is based on simple and clean algorithmic ideas.

Weaknesses

There are several concerns I have regarding the setup, the impact and the results. In particular: 1. While the model demonstrates an interesting connection between supervised and unsupervised learning, if we focus on either of these aspects, it seems less relevant. For supervised learning, the distributional assumptions are too strong (usually one either makes strong assumptions on the marginal distribution or assumes the existence of margin, but not both). For unsupervised learning, the component we want to discover is too specific (instead of a general property, like matching some Gaussian moments, it is assumed that the component is just a truncation of the distribution of the other components). 2. On the algorithmic level, the impact seems to be limited. Variants of the algorithm presented have largely appeared in the literature for similar problems. 3. The main result (Theorem 1) does not seem to be completely accurate. More specifically, there has been no account for the case where the truncation region $[a,b]$ has $|a+b|>0$ but $|a+b|\approx 0$ (e.g., $|a+b| = 2^{-d}$). While Lemma 4 accounts for the case when $|a+b|$ is large enough, Lemma 5 only accounts for the case $|a+b|=0$. It is not clear what happens in between. In Figure 5.2, it seems that Lemma 5 could be generalized to capture cases when $|a+b|$ is positive but bounded. However, this has not been proven. Overall, the paper seems to have important unresolved technical issues and also it could highly benefit by relaxing the distributional assumptions. I would encourage the authors to continue working on the paper, but in its current form, it is below the acceptance threshold. -- The authors fixed an important technical issue in their proof (item 3 above) and I have increased my score (from 3 to 5) to reflect this.

Questions

I would be interested to hear any comments that the authors have regarding the points (1,2 and 3) I made in the weaknesses section. Moreover, I have the following suggestions regarding the presentation. 1. The problem definition (page 2) needs to be clarified further to avoid confusion. In particular, it is not clear what is known and what is unknown. For example, does the learner know $q,a$ and/or $b$? 2. How does the paper [DK'22] compare to your results? On a more general note, I think it would be beneficial to present in section 1.2 some result on NGCA (e.g. [DK'22]) more concretely, as well as some result on supervised learning and emphasize the trade-offs when considering the problem proposed in this paper. [DK'22]: Diakonikolas, Ilias, and Daniel Kane. "Non-gaussian component analysis via lattice basis reduction." Conference on Learning Theory. PMLR, 2022. 3. Some typos: - Line 105: double "the". - Line 106: "follows uses" $\to$ "uses".

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

Yes

Reviewer Ua7t7/10 · confidence 4/52023-07-08

Summary

This work addresses learning a halfspace with margin from unlabeled samples (which is possible due the combination of margin and distributional assumptions). They impose that the ambient distribution is an affine transformation of the d-fold product of some 1D symmetric log-concave distribution, conditioned on one coordinate lying outside of an interval with mass at least $\varepsilon$. They provide an algorithm for recovering this distribution up to TV error $\delta$ with time and sample complexity $poly(1/\varepsilon,1/\delta,d)$, based on analyzing the first two moments of a certain exponential reweighting of the empirical distribution.

Strengths

- The paper was generally well-written and organized. The proposed algorithm is easy to implement, and its analysis requires some clever new techniques. - I appreciate the initial simplified lemmas; they allow the reader to understand main ideas at a high-level without getting bogged down with the full details. - Approaches for this problem based on previous work would seem to require Gaussianity and fail to provide as strong recovery guarantees in polynomial time - Experiments validate their algorithm on Gaussian and non-Gaussian data

Weaknesses

- The notion of margin seems a bit non-standard. Can the authors relate this to the standard notion (or refer to an existing example of this usage)? - The proven time/sample complexity, while polynomial, is quite slow - While the authors suspect that improved guarantees are possible for the same algorithm, it would be preferable if they could support this conjecture with experiments; the current sample sizes are quite large Minor Nits: - Some inconsistency in Sec 3 for whether $\hat{q}$ is isotropized or not (it mostly is, but not in the warm up) - Inconsistency between single bars and double bars for the Euclidean norm (e.g. Lemma 1 uses both) - Inconsistency between $\mu_i$ and $\mu_{\alpha_i}$ in Sec 3 - Line 214, clause beginning with "Since" looks like it should be combined with following sentence - Line 217, should be for $x \geq 0$? - Perhaps the rule of signs should be stated in the integral form for which is applied? Also, it would improve readability to mention the u-substitution used to eliminate the square in the exponential - Line 225, $F'(0)$ is a scalar, doesn't have a root, same with $F''(0)$ (presumably should just be $\alpha$) - Line 226, doesn't $F''(0) \neq 0$ always hold? - Equation after line 240, would match previous notation to use $\hat{q}(x)$?

Questions

- The results are stated for non-singular affine transformations. What happens as the transformation approaches singularity? Given that there appears to be no breakdown in the run time bound in this regime, the non-singularity requirement was surprising to me

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

Yes, authors are clear about assumptions and potential impacts.

Reviewer 6stc6/10 · confidence 4/52023-07-12

Summary

In this work, the authors study the problem of unsupervised learning a halfspace. In their setting they have a continuous distribution (isotropic-log-concave) and they remove $\geq \epsilon$ mass from an interval $u\cdot x\in[a,b]$, and their goal is to find the half space that separates the two regions. Their idea is basically to reweight the samples and use the first two moments to find the unknown vector.

Strengths

Well written and easy to follow. The authors provide experiments a proof of concept. This is a nice problem and it was not studied before. Even if the method of attacking this problem is what someone expect, still I found the results intersting.

Weaknesses

1. The distributional assumptions are strong. They consider a product of the same log-concave distribution and also they assume that this distribution is symmetric. 2. Some steps that are trivial are over-simplified where some other steps that are not easy to follow are not explained, especially in proof of lemma 4. 3. The results are not tight as is needed a high polynomial on $\epsilon$.

Questions

545-546: Is this part of the proof ok? The distribution is no longer symmetric after removing the mass. I think this problem is closed related to learning with truncated samples. Maybe the authors should comment on this/add related work. typos: 550: I guess you mean that the directions are uncorrelated as after removing the mass and rotating this is not a product distribution but it is true that the directions are uncorrelated. 529: there is a missing number.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

no limitations.

Reviewer efHe6/10 · confidence 1/52023-07-13

Summary

This paper studies the unsupervised learning for high-dimensional halfspace under affine transformation. The goal is to estimate the normal vector of the halfspace such that the learned distribution is close to the underlying truth. The paper provides an algorithm using so-called ''contrastive moments'' with sample and time complexity guarantee.

Strengths

- The algorithm provided achieves polynomial dependence in $d, 1/\epsilon, 1/\delta$ for both time and sample complexity. - Existing literature relies on supervised learning, or Gaussianity of the distribution, or NGCA and ICA. While this paper only requires log-concavity. - Techniques introduced, e.g. contrastive moments, might be of independent interest.

Weaknesses

- The product and isotropic distribution with affine transformation assumption seems to be restrictive, can the analysis or algorithm go beyond that? - The presentation of the problem is hard to follow, might need more explicit mathematical definitions. For example, where is $\tilde{P}$ in **Problem** defined.

Questions

- Are coordinates of $P(Q)$ independent with each other before the affine transformation? - Can author(s) provide an real data application example for the problem setup in Definition 1, and specify the the roles of $q,Q,d, P,u$?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

1: Your assessment is an educated guess. The submission is not in your area or the submission was difficult to understand. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

No limitation is discussed.

Reviewer Ua7t2023-08-10

Thanks for these clarifications. I find the answers to my questions satisfactory and maintain my accept decision.

Reviewer YLvd2023-08-11

I thank the authors for responding to my concerns. I have increased my score to reflect that I now believe the paper to be technically sound.

Reviewer efHe2023-08-14

I thank the authors for the response. I have increased the score.

Reviewer rhCN2023-08-17

Response

I thank the authors for thoroughly addressing my concerns. I think providing theoretical results for "hard-margin" problems is an important first step towards more harder and prevalent "soft-margin" problems, so I will keep the score as is. For **Q5**, are we on the same page that "$F'(0)$ has at most one root." and "$F''(0)$ has no root." are typos, since $F'(0)$ and $F''(0)$ are both numbers, and not functions.

Authorsrebuttal2023-08-17

Follow-up Response

Yes, sorry, we should have said explicitly, those are typos in our proof idea description. As you point out, F'(0) and F''(0) are scalars. The corrected proof idea is summarized in our response to Q5.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC