Feature Adaptation for Sparse Linear Regression

Sparse linear regression is a central problem in high-dimensional statistics. We study the correlated random design setting, where the covariates are drawn from a multivariate Gaussian $N(0,\Sigma)$, and we seek an estimator with small excess risk. If the true signal is $t$-sparse, information-theoretically, it is possible to achieve strong recovery guarantees with only $O(t\log n)$ samples. However, computationally efficient algorithms have sample complexity linear in (some variant of) the condition number of $\Sigma$. Classical algorithms such as the Lasso can require significantly more samples than necessary even if there is only a single sparse approximate dependency among the covariates. We provide a polynomial-time algorithm that, given $\Sigma$, automatically adapts the Lasso to tolerate a small number of approximate dependencies. In particular, we achieve near-optimal sample complexity for constant sparsity and if $\Sigma$ has few ``outlier'' eigenvalues. Our algorithm fits into a broader framework of feature adaptation for sparse linear regression with ill-conditioned covariates. With this framework, we additionally provide the first polynomial-factor improvement over brute-force search for constant sparsity $t$ and arbitrary covariance $\Sigma$.

Paper

References (48)

Scroll for more · 36 remaining

Similar papers

Peer review

Reviewer Ahsd8/10 · confidence 4/52023-06-27

Summary

This work studies the problem of sparse linear regression under the statistical model where the examples are drawn as zero-mean Gaussians with covariance matrix $\Sigma$ and each response is a t-sparse linear combination of the examples plus i.i.d. Gaussian noise. While classical results establish that the LASSO can computationally efficiently recover a good solution v* with a nearly optimal number of samples when the covariance matrix $\Sigma$ is well-conditioned, guarantees beyond this simple setting are substantially lacking, with a few notable exceptions. In general, not much is known beyond a brute-force approach of trying all $\binom{n}{t}$ sparsity patterns, which requires $n^t$ time. This work shows that even if $\Sigma$ is ill-conditioned, if Sigma is well-conditioned after removing the top or bottom O(1) eigenvalues (i.e. a notion of “robust” well-condtionedness), then this running time can be improved to roughly $f(t) \cdot n^3$. When there are a small number of tiny eigenvalues, the main problem is the existence of sparse linear dependencies among the variables. The algorithmic approach to address this problem is then to iteratively peel off a small number of these variables at a time (IterativePeeling()). This procedure gives a construction of a small dictionary of vectors such that any t-sparse vector can be written as a linear combination of this dictionary with coefficients bounded in L1 (Lemma 2.9), which in turn implies that the LASSO identifies a good solution v in the sense of bounded excess risk. The authors interpret the technique of applying LASSO to an augmented dictionary as “feature adaptation”.

Strengths

* Sparse linear regression is a widely studied yet notoriously difficult problem, and this work identifies a very natural class of inputs for which positive results can be obtained. * I find the discussion in Section 2.1 on the proposal of (t, alpha) dictionaries as a canonical abstraction of all existing approaches to sparse linear regression to be very interesting and valuable. It helps concentrate research efforts on this problem to this more structured approach, and I believe it may prove to be influential in following works on sparse linear regression.

Weaknesses

* It seems like like there are no bounds on the sparsity of the approximate solution v that is outputted by this algorithm, probably due to the fact that the feature adaptation step augments the feature set with dense linear combinations of the existing variables (please let me know if I have misunderstood). Thus, the measure of the “sparsity” of this algorithm lies in the small sample complexity, rather than its ability to output a sparse solution. Thus, the guarantees of this algorithm are tightly linked to the statistical setting of sparse linear regression, and it may be difficult to adapt these techniques to the problem of outputting a good sparse solution to linear regression. However, I do find this limitation very interesting, and I wonder if there are gaps between the performance of these algorithms if one is required to output a sparse solution. Such separations are known in certain sparse recovery settings (see, e.g., https://arxiv.org/abs/1110.4414).

Questions

n/a

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

The authors have provided adequate discussion of limitations

Reviewer BTho8/10 · confidence 3/52023-06-27

Summary

The paper introduces an algorithm to solve sparse regression when the covariates are generated from a normal distribution with ill-conditioned covariance matrix, i.e. outlayer eigenvalues. The algorithm is based on feature augmentation where, meaning that the covariates are completed with well-chosen vectors, and is designed to provably achieve near optimal sample complexity in the studied framework.

Strengths

Regarding presentation, the paper is clearly written and presented. Each theoretical result is explained and justified, which makes the paper easy to follow. Regarding the content, the idea of augmenting the features by taking the data distribution into account before solving the Lasso seems new. The paper provides interesting theoretical insights on this method, while maintaining a computational perspective, for instance when explaining the design of the dictionary.

Weaknesses

While the main contributions of the paper are theoretical, it would have been interesting to get more experimental details on the algorithm. In particular, numerical illustrations of the behavior with respect to the conditioning of the covariance matrix (number of outlayers, gap between largest and smallest eigenvalues, ...), and of the robustness with respect to standard algorithms like Basis Pursuit (for instance in "phase transitions" between well and ill-conditioned scenarios), may broaden the audience and the impact of the contributions.

Questions

The ideas of experiments below may help provide more information on the practical behavior of the algorithm and put it into perspective with the theoretical results. * Could the authors provide the number of samples necessary to reach low risk with respect to the ratio $\frac{\lambda_{n - d_h}}{\lambda_{d_l+1}}$ and with respect to the number of outlayer eigenvalues when using their algorithm and when using BP ? * Could the authors provide the computation time with respect to the number of samples in addition to the risk ? * Could the authors provide the size of the set $S$ obtained in practice with Iterative Peeling, and compare it to the bounds given in Lemma 2.4 and 2.5, with respect to the sparsity $t$ ? * Could the authors provide a comparison of the performance with and without the knowledge of $\Sigma$ (estimated from samples v. known distribution) ?

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

4 excellent

Limitations

The limitations are discussed by the authors. ----------- After reading the other reviews and the rebuttal, I increased my rating.

Reviewer 8yrN7/10 · confidence 3/52023-07-09

Summary

This paper presents an innovative polynomial-time algorithm for sparse linear regression in the correlated random design setting. The algorithm adapts the Lasso technique to effectively tolerate a limited number of approximate dependencies, resulting in both computational and statistical efficiency for covariance matrices with a few "outlier" eigenvalues. The proposed method is part of a more extensive framework of feature adaptation for sparse linear regression with ill-conditioned covariates and offers the first polynomial-factor improvement over brute-force search for constant sparsity and arbitrary covariance.

Strengths

1. The paper contributes a novel algorithm for sparse linear regression, adeptly adapting the Lasso to accommodate a small number of approximate dependencies. 2. The proposed algorithm exhibits both computational and statistical efficiency for covariance matrices with a few "outlier" eigenvalues, providing a substantial advancement over existing methods in this context.

Weaknesses

1. The paper assumes constant sparsity for simplicity, but it is essential to explore the impact of this assumption on the main results. 2. To more convincingly demonstrate the algorithm's computational and statistical efficiency, the paper would benefit from the inclusion of supplementary numerical simulations.

Questions

1. The paper assumes constant sparsity $t$ for simplicity. It is unclear whether this assumption is necessary to establish the main results of the paper? In particular, it would be interesting to examine the case where the sparsity $t$ takes the order of $\log n$. 2. It is important to consider the more general case where $t$ is treated as a variable instead of a constant. I wonder whether the main results in the paper remain valid in this case.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

3 good

Contribution

4 excellent

Limitations

Yes

Reviewer EHBK5/10 · confidence 4/52023-07-09

Summary

This paper studies the correlated random design setting, where the covariates are drawn from a multivariate Gaussian, and seeks an estimator with small excess risk. This work provides a polynomial-time algorithm that, given Σ, automatically adapts the Lasso to tolerate a small number of approximate dependencies, and achieves near-optimal sample complexity for constant sparsity and if Σ has few “outlier” eigenvalues.

Strengths

Sparse linear regression is a fundamental problem in high-dimensional statistics. This paper studies a polynomial-time algorithm that automatically adapts the Lasso to tolerate a small number of approximate dependencies. In theoretical analysis, this work achieves near-optimal sample complexity for constant sparsity. The proposed algorithm fits into a broader framework of feature adaptation for sparse linear regression with ill-conditioned covariates.

Weaknesses

1.If this work can provide experimental verification of the superiority of the proposed algorithm, it will be more convincing. For example, compare with the related work to verify the performance of the proposed algorithm in terms of time complexity and accuracy. 2.In figure 1, when the number of samples is greater than 100, is the standard deviation of adapted BP algorithm zero? 3.The presentation of references is not standardized, such as: [38] Sara Van De Geer. On tight bounds for the lasso. Journal of Machine Learning Research, 19:46, 2018. [22] Jonathan Kelner, Frederic Koehler, Raghu Meka, and Ankur Moitra. Learning some popular gaussian graphical models without condition number bounds. In Proceedings of Neural Information Processing Systems (NeurIPS), 2020. 4.The organization and presentation of this paper can be further improved.

Questions

See “Weaknesses”.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

1 poor

Contribution

2 fair

Limitations

N/A

Reviewer BTho2023-08-14

Thanks for the answer

Thanks to the authors for their answer and insights on practical details. After reading the other reviews and the rebuttal, I will increase my rating.

Program Chairsdecision2023-09-21

Decision

Accept (spotlight)

© 2026 NYSGPT2525 LLC