Summary
The paper proposes a new algorithm for sampling on a constrained space via a Hamiltonian Monte Carlo algorithm, which uses a Riemannian Manifold HMC algorithm, with the Hessian metric given by a self-concordant barrier. This generalizes and improves upon existing work in constrained sampling, while removing a source of bias by adding an “involution checking step”.
Strengths
The intuition for the algorithm, including the choice of self-concordant barrier, is well-informed and intuitive, although it has appeared in some form in Kook et al. (2022). Thus, I will discuss many of the strengths and weaknesses in relation to the prior work of Kook et al. (2022).
This paper possesses two generalizations of Kook et al. Firstly, the primary problem setting in Kook et al. (Eq 11 of that paper) is less general than that in this paper. The proof for this method is also cleaner, and purports to avoid some technical issues present in the proof Kook et al. (I did not carefully verify this point). Furthermore, it avoids the problems of bias presented in Kook et al., which is carefully justified by a Theorem.
The method is clearly unbiased when evaluated on the hypercube and simplex, by a margin that is usually many standard deviations.
The broad details of the proof seem to be correct and well-written, although I did not check every detail.
Weaknesses
No rigorous non-asymptotic theory was presented, although this is also true of Kook et al., likely because of the complexity of these algorithms.
It is difficult to draw firm conclusions from the real-world instances in the experiments, since the method does not show improvement in all/a clear majority of cases.
The notation in this paper is a bit cumbersome, particularly in the Appendix. Nonetheless this is understandable given the complexity of the algorithm, and the authors did a good job in reducing the notational burden within the main text.
To summarize, it was difficult to review all the details/proofs in this manuscript, given its length and technical nature. Nonetheless, I feel like it presents an important development within constrained sampling and merits publication. This informs my final score.
Questions
How difficult is it to establish any non-asymptotic convergence results in this context? What would be the expected rates, and would they differ (in the order of dependencies) with Kook et al.?
As noted under “weaknesses”, the experiments on “real-world data” do not seem fully convincing. Can the authors elaborate further on why this would be the case, and if there was extensive tuning of the parameters involved? It seems from the Appendix that the parameters of both algorithms were not extensively tuned, which then begs the question whether this is really a fair comparison.
It is curious that a (constrained) Gaussian distribution is chosen for the synthetic data, when Kook et al. consider a uniform distribution instead. Their choice allows for some simple tests of uniformity. Does considering a uniform distribution make any appreciable difference in the output?
The discussion in Appendix J is quite useful and I think some more of these details could be also summarized in the main text.
It is my personal preference, but I believe text in the paper should not be highlighted. Use boldface or perhaps underlining to emphasize the text in Ll. 192-193 and in Algorithm 1.
L. 91 “motivates to” -> “motivates us to”
L. 170 “the definition domain” -> “the domain”
L. 213 “enables” -> “enables one/enables us”
The authors should be careful about the formatting of some equations in the Appendix, since they exceed the column width.
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Limitations
None beyond those raised in earlier sections.