Summary
The paper **Distributionally Robust Linear Quadratic Control** considers the case of finite-time linear quadratic optimal control with process and measurement noises and known time-varying dynamics. The novelty lies in the fact that the distributions of the initial state and noises are unknown but lie in an ambiguity set defined as a 2-Wasserstein ball centered in a known Gaussian distribution and with known radius. The objective is to minimize the expected LQR cost with the distributions chosen adversarially in that ball. The authors call this adversarial problem "distributionally robust Linear Quadratic Gaussian". It strict generalizes LQG control, which assumes these distributions are known and Gaussian.
The main contribution is twofold. The first part is theoretical, as the authors prove the existence of a Gaussian adversarial distribution, that is, the distributionally robust LQG reduces to classic LQG with unknown Gaussian distributions. A consequence is that the optimal controller is a linear state feedback, like in the classic LQG case. This leads to the second contribution: an algorithm to efficiently estimate the adversarial Gaussian distribution from data. The authors claim that, then, the LQG problem with estimated noise distributions can be solved efficiently by using classical methods.
The authors illustrate convergence of their estimation algorithm on a simulated example.
Strengths
The paper is interesting and the clear exposition makes the reasonings easy to follow. The appendix is well-managed, and I appreciate Appendix A on the solution of classical LQG with a Kalman filter.
I like the idea of allowing for a whole family of noise distributions rather than assuming a fixed one. The main theoretical result helps mitigate the restrictiveness of the commonly-accepted "white noise assumption"; indeed, it shows that allowing for a larger family of noises does not provide any benefits (for distributions close enough to Gaussians). This new problem formulation thus seems relevant. The subsequent algorithm to solve the distributionally robust LQG problem follows naturally and further justifies the interest of the theoretical result. Overall, the reasoning exposed is sound and well-motivated.
Finally, I appreciate that the authors went the extra mile in Section IV by augmenting the theoretical result with a data-efficient algorithm.
Weaknesses
The three main weaknesses of the paper are, in my opinion, 1. the lack of thoroughness of the simulation study; 2. the lack of discussion of the effect of hyperparameters; and 3. some arguments are unclear and should be explicited (although I do not question their conclusions). I detail these three points. These points are not critical for acceptance, but I believe the paper would be improved by addressing them.
W1) The simulation study shows convergence of the algorithm on a randomized use case. This is a good sanity check, but I would also appreciate a simulation of the case when the noise distribution is not normal but still within the Wasserstein ball. The theoretical result ensures that Nature's adversarial strategy _is_ normal, but an illustration of the case when Nature is sub-adversarial would be welcome. In particular, is the cost of the learned policy reduced compared to when the noise is normal?
W2) The role of the choice of ball radii is not discussed. While there is obviously no notion of "optimal" radius, since it simply corresponds to a degree of robustness, I would like to see the evolution in performance with increasing radii. Intuition tells that performance should decrease as robustness increases, but a confirmation in simulation would be welcome.
W3) Some arguments are unclear to me, although their conclusions seem to be valid. In particular:
1. The authors make multiple claims on convexity of sets of probability measures. Examples are lines 112, 151, 189. I understand that these claims are made by considering Borel measures as a subset of the vector space of signed Borel measures with standard addition and scalar multiplication. I believe this superset should be mentioned explicitly at least ones, as it is rather unusual and notions of convexity of a metric space exist (and differ).
This is also relevant on line 151, where $\mathcal{W}$ is claimed to be infinite-dimensional despite not being a vector space.
2. With this understanding of convexity, why is the set $\mathcal{W}$ non convex, as claimed e.g. on lines 112, 151 and 189? As far as I understand, each set $\mathcal{W}_{z}$ is convex, with $z\in\{x_0, w_t, v_t\}$. Then, $\mathcal{W}$ should be convex as the cartesian product of these sets. The only way I understand this non-convexity is if $\mathcal{W}$ is not _equal_ to the cartesian product, but only isometric to this cartesian product by the mapping that computes the marginal distributions. I believe this should be mentioned explicitly, at least in a footnote, as this questions distracted me from more central claims of the paper for a while.
3. I am unsure about the claim on line 131 that the controller can compute the fictious states $\hat x_0,\dots,\hat x_t$ from the real observations $y_0, \dots, y_t$ without knowing the initial state. Indeed, this would imply in particular that the controller can reconstruct the initial state. Since the time origin is arbitrary, any state could be reconstructed. This claim appear unnecessary for the rest of the argument and should be either removed or clarified in my opinion.
4. I find the formulation of the first sentence of Proposition 4.1 extremely confusing. In particular, what comes after "then" in the first sentence reads as a logical consequence of what precedes whereas it is actually a definition of the symbols $\mathbb{P}^\star$ and $V_t^\star$. I recommend reformulating.
5. In Proposition 4.2, I recommend avoiding using the term "smooth". While it has a precise meaning in a specific branch of mathematics, it is often only used informally in control. I would prefer the more standard "infinitely differentiable with $\beta$-Lipschitz gradient".
Questions
My questions are detailed in the above paragraph on weaknesses. I would appreciate if the authors could respond to these.
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Limitations
The authors have adequately addressed limitations and potential negative societal impact of their work.