Generalization Bound and Learning Methods for Data-Driven Projections in Linear Programming

How to solve high-dimensional linear programs (LPs) efficiently is a fundamental question. Recently, there has been a surge of interest in reducing LP sizes using random projections, which can accelerate solving LPs independently of improving LP solvers. This paper explores a new direction of data-driven projections, which use projection matrices learned from data instead of random projection matrices. Given training data of $n$-dimensional LPs, we learn an $n\times k$ projection matrix with $n > k$. When addressing a future LP instance, we reduce its dimensionality from $n$ to $k$ via the learned projection matrix, solve the resulting LP to obtain a $k$-dimensional solution, and apply the learned matrix to it to recover an $n$-dimensional solution. On the theoretical side, a natural question is: how much data is sufficient to ensure the quality of recovered solutions? We address this question based on the framework of data-driven algorithm design, which connects the amount of data sufficient for establishing generalization bounds to the pseudo-dimension of performance metrics. We obtain an $\tilde{\mathrm{O}}(nk^2)$ upper bound on the pseudo-dimension, where $\tilde{\mathrm{O}}$ compresses logarithmic factors. We also provide an $Ω(nk)$ lower bound, implying our result is tight up to an $\tilde{\mathrm{O}}(k)$ factor. On the practical side, we explore two simple methods for learning projection matrices: PCA- and gradient-based methods. While the former is relatively efficient, the latter can sometimes achieve better solution quality. Experiments demonstrate that learning projection matrices from data is indeed beneficial: it leads to significantly higher solution quality than the existing random projection while greatly reducing the time for solving LPs.

Paper

Similar papers

Peer review

Reviewer hECG6/10 · confidence 4/52024-07-06

Summary

On the theoretical side, the paper studies the problem of sample complexity of learning data-driven projection matrices for accelerating high-dimensional LP solving. Given n-dimensional LPs drawn from some problem distribution, the goal is to bound the number of problem instances needed to learn an $n\times k$ projection matrix such that a good solution to the original LP can be found by (efficiently) solving the k-dimensional projected LP. The main result is a $\tilde{O}(nk^2)$ upper bound on pseudo-dimension of the class of functions that measure the optimal value of the projected LP, and a corresponding $\Omega(nk)$ lower bound. While finding optimal projection matrix on a training set is hard, the paper proposes two practical approaches for learning a projection matrix from the training LP instances. The first approach applies Principle Components Analysis to a matrix of optimal solutions of training instances. The second approach uses stochastic gradient updates to learn the projection matrix. Empirically, data-driven projections seem to perform better than random projections, with comparable gains in speed-ups.

Strengths

- Speeding up high-dimensional LPs is fundamental to operations research and data-driven projections appears to be a promising tool. - A polynomial upper bound on the sample complexity theoretically establishes that projection matrices are learnable from data. - There is also a lower bound, with is tight up to a factor of $k$ and logarithmic factors. - Even though exact optimization on training LP instances is hard, the paper provides practical solutions that work well empirically. Practical algorithms are often hard to design in the data-driven literature.

Weaknesses

- The i.i.d. assumption needed in theoretical results may be too strong in practice. - The proposed methods for learning the projection matrix are not efficient e.g. PCA approach needs optimal solutions of training problem instances. - While the authors provide bounds on the time complexity of their learning methods, there are no guarantees about the quality of solutions.

Questions

- How does the proof of Lemma 4.3 differ from the techniques of Balcan et al., NeurIPS 2022? - It seems like the proof for upper bound can be simplified by using the GJ framework of Bartlett et al. COLT 2022 (P. L. Bartlett, P. Indyk, and T. Wagner. Generalization bounds for data-driven numerical linear algebra.) - The $\log (H/\varepsilon)$ term is not needed in the sample complexity bound $N$. (Line 153) - Line 209: sign patters

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

Yes.

Reviewer E5r36/10 · confidence 3/52024-07-09

Summary

This paper considers data-driven approach for learning projections for LPs. Given an LP with $m$ constraints and $n$ variables, it establishes bound on learning a projection $P\in \mathbb{R}^{n\times k}$ that reduces $n$ to $k$. The main contribution is to establish uniform convergence bound on pseudo-dimension, for an upper bound of $\tilde O(nk^2)$ and a lower bound of $\Omega(nk)$. Authors also propose two algorithms for learning the projection, one is based on PCA of the optimal solution of LP instances, the other is a gradient ascent approach.

Strengths

This paper establishes bound on pseudo-dimension of of performance metrics for data-driven projections for LP, the bound is tight up to a factor of $k$. Experiments are also performed for the two methods proposed in this paper, and it provides speed up over the column-randomized approach that is data-oblivious and performance upgrade.

Weaknesses

The techniques for proving the upper and lower bounds on pseudo-dimension are quite standard, authors should emphasize the technical difficulties for proving these bounds. Empirical side, while "after* learning the projection, it gives good performance, the process of learning the projection is quite slow. Authors might consider adding more discussions on how to learn these projections more efficiently (from an algorithmic perspective).

Questions

Suppose you learn the projection for both the primal and dual, could this lead to an even more efficient downstream algorithm, as one only needs to handle LP instance of size $k'\times k$? Or are there obvious reasons this is not a good idea?

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

Yes.

Reviewer nj7Z5/10 · confidence 3/52024-07-10

Summary

The paper attempts to theoretically analyze a method called Data-Driven Projections in Linear Programming. As discussed in the paper, projection methods aim to reduce the size of high-dimensional LPs. While random projection methods have improved the efficiency of LPs, data-driven projections have achieved better results. In this paper, it is assumed that the parameters of the LP (parameters in the constraints and the objective) come from some distribution. Based on a set of training samples of these parameters, data-driven methods try to find a subspace where future optimal solutions are expected to appear. The main contribution of the paper is that they propose a generalization bound for the difference between the empirical and statistical performance of the projected LP.

Strengths

- The paper is well-written and well-motivated. - The proofs are rigorous and well-written.

Weaknesses

- I think the contribution of the paper is marginal. - Although it is mentioned in the paper that the generalization is independent of the choice of the projection matrix $P$, the final goal is to minimize the expected value of the objective. To achieve this, it is necessary to choose a good $P$ that minimizes the empirical optimal value. I believe it should be proven somehow that the algorithm proposed by the paper can suggest a projection matrix that captures the subspace where future optimal solutions are expected to appear. I couldn't find a proof for this in the paper.

Questions

Please refer to the weaknesses section.

Rating

5

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

Please refer to the weaknesses section.

Reviewer vPup6/10 · confidence 4/52024-07-16

Summary

The paper proposes a data-driven approach to an accelerated solution of linear programming problems belonging to a common family. To this end, the dimensionality of the problems is reduced by a projection learned from a training set of problems. The paper first gives a theoretical generalization bound for this learning problem. Also, the paper proposes two specific projection learning algorithms, based on PCA and stochastic gradient ascent. Finally, the performance of the proposed methods is experimentally illustrated on several test cases.

Strengths

The paper is generally quite nicely written and very readable. I could follow it everywhere without difficulty and didn't even notice any typos. The theoretical generalization bound (Theorem 4.4) is derived using more or less standard approaches of statistical learning theory, but the result itself is new and the proof seems to be correct. The proof is presented in the main text and is concise and nice. The idea of data-driven projection and its implementations based on PCA and gradient ascent seem to be original. The paper includes a comparison of the proposed algorithm on 8 test cases; the proposed algorithms show good performance there.

Weaknesses

The most significant issue that I see is that the derived generalization bound is vacuous for the LP families experimentally studied in the paper, but this point is for some reason completely ignored by the authors. Specifically, the bound in question is $\epsilon\lesssim Hk\sqrt{n/N}$ (lines 257 and 359). As far as I understand, in the experiments $N\sim 200, n\sim 500, k\sim 20$. Substituting these numbers gives $\epsilon\lesssim 20H,$ which is vacuous because the expected performance of a solver is in the interval $[0,H]$ anyway. Not to mention that the $O(\cdots)$ in the bound may contain additional large constants. In fact, asymptotic generalization bounds are well-known to (significantly) overestimate the true generalization gap and so are more suitable as a mere theoretical assurance of convergence. However, the paper seems to suggest (say, in the end of section 6) that the derived bound has some practical value while making no attempt to actually discuss specific numbers associated with its experiments, which I find completely misleading. (This does not undermine the theoretical merit of Theorem 4.4; on the other hand convergence rates $O(\sqrt{pdim/N})$ are standard in SLT.) The second general significant issue is that the whole setting of data-learnable projected optimization proposed in the paper seems fairly artificial to me. This setting obviously assumes the variables and constraints to be somehow aligned between different LP instances (in contrast, random projections do not require any such alignment). This point is discussed in Remark 3.2, where it is mentioned that such scenarios arise in daily production planning and flight scheduling. It would be interesting to see a particular real scenario of this type. All the example experimentally studied in the paper, even those that the authors call realistic, do not look realistic to me. As far as I understood, each of the 8 considered families is obtained by synthetically perturbing a single LP instance. This artificial setting is obviously very favorable to the proposed algorithms compared to the baselines. If the perturbation goes to 0, the family degenerates into a collection of identical LP instances that can all be perfectly "solved" by recalling a single solution, so the proposed training-based methods can be made to look arbitrarily more efficient than the baselines by tuning the perturbation magnitude.

Questions

N/A

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

N/A

Reviewer E5r32024-08-07

I thank the authors for answering my questions. I'll keep my score as is.

Reviewer hECG2024-08-09

Thank you for the detailed response, I appreciate the answers to my questions. I retain my score.

Reviewer nj7Z2024-08-10

I thank the authors for their answers. They addressed my questions, so I increased my score accordingly.

Reviewer vPup2024-08-10

I thank the authors for their reply and addressing both of my concerns. I'm raising my score.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC