Response to the reviewer's comments/questions (1/2)
We thank the reviewer for providing feedback on our paper, and for highlighting the novelty of the positivity certification application. We hope to clarify the reviewer’s questions below.
### Transvectant/Clebsch-Gordan Clarification
Although the form may look unfamiliar, the transvectant, which we denote by $\psi$, indeed describes precisely the map from two irreps to their tensor product’s decomposition back into irreps. It may be helpful to recall that the finite dimensional irreducible vector spaces of $SL(2,\mathbb{R})$ can be identified with the homogeneous polynomials of a given degree, so the Clebsch-Gordan coefficients for $SL(2,\mathbb{R})$ should map two input polynomials to an output polynomial. ($\psi_n$ indeed maps two input polynomials to an output polynomial.) Please also see the first paragraph of the introduction to Böhning [1] for formal clarification that the transvectant operation, which we denote by $\psi$, is exactly the Clebsch-Gordan map for this group.
### Section 4.2 and pseudocode
Thank you for pointing out the ambiguity in the functions in the pseudocode in Section 4.2. We have corrected this in the updated draft.
We have also rewritten the description of the last layer in Section 4.2, deferring the exact details of the mathematical derivation (which may have been confusing) to the appendix.
### Relevance of Section 4.3 to polynomial optimizations, and experimental details
Positivity verification, as described in Section 3, is an equivariant learning task. Therefore, one might naturally design an equivariant architecture for this problem, as we do in Sections 4.1-4.2. The surprising, and central, theoretical contribution of this paper is that an equivariant network ostensibly designed for the very purpose of positivity verification, is in fact unable to approximate the function of interest for positivity verification. Therefore, any network designed to solve this problem must be able to output more than just equivariant polynomials, which is a significant restriction on the architectural design space relative to prior work! To the best of our knowledge, such a finding is also without precedent in the equivariance literature. It also provides some motivation for using data augmentation, rather than an equivariant architecture. This is the content of Section 4.3. Corollary 1 implies that it doesn’t matter what loss function we use -- no equivariant polynomial can approximate the function of interest under any standard loss function. In the paper, we used the normalized mean squared error loss (with a small additive stabilization term in the denominator), as noted in the caption of Figure 2.
By “natural” polynomials, we just meant that they come from a naturally occurring, application-specific distribution, such that it is empirically possible to well-approximate an NP-hard optimization like polynomial positivity verification over this distribution. (Intuitively, we would expect such a distribution to have low entropy -- it should differ significantly from a uniform distribution over polynomials in a norm-bounded ball, e.g.) For example, one of our experiments is on the polynomials that arise from spherical code bounds; this provides one example of a “natural” set of polynomials, but there is no single definition of such a distribution. This point should not be critical to our main arguments.
### Q1: Utility of $SL(2,\mathbb{R})$ equivariance
While the symmetry group is not crucial for studying the problem, equivariant learning has been helpful from a sample complexity perspective in many other contexts. Since our positivity verification problem is equivariant with respect to the large group $SL(2,\mathbb{R})$, it is natural to try exploiting this structure in the learning pipeline. Please see the paragraph after (1) for an explanation of the precise equivariance property, and please feel free to follow up with further questions if this point remains unclear.
### Q2: Clarification on timings
Indeed, the MLP times are for inference, while the Mosek times are full solves. The motivation of this work is to speed up the *online* determination of nonnegativity, which is possible with just MLP inference. Amortizing the runtime of a traditional solver over many instances is not possible, hence why we compare to the full solve time. For reference, the training time for the MLP was on the order of 90 minutes, which even included many different loss evaluations (e.g. evaluating the loss under random $SL(2,\mathbb{R})$-augmentations).