Summary
The paper introduces a technique for rare event sampling that combines normalizing flows with importance sampling. The authors refer to this technique as NOFIS (NOrmalizing Flows assisted Importance Sampling). They justify their work by highlighting the limitations of standard sampling algorithms, such as MCMC, in sampling regions of low probability, where the density, denoted as $p$, is approximately $10^{-X}$, with X being an integer greater than 4. In this context, known as the regime of rare event sampling, algorithms like MCMC would require an impractical number of samples, rendering these approaches highly inefficient. The authors propose that employing normalizing flows-aided importance sampling holds promise as a solution to this problem.
Strengths
- The paper flows smoothly and is enjoyable to read.
- The authors provide great level of details and do not take anything for granted, which I appreciate.
Weaknesses
- **Novelty**: I don’t find much novelty in the proposed paper. The technique presented by the authors has already been explored in many prior works in different fields, particularly in physics, where rare event sampling is often a challenging problem (see below).
- **Related Works**: Despite many prior works combining normalizing flows with importance sampling, and beyond, exist, this paper lacks a dedicated *Related Work* section. Several seminal works have been completely overlooked despite their significant contributions to the field of normalizing flow-aided importance sampling in statistical physics [1], chemistry[2], and quantum field theory[3,4,5].
- **Annealed Importance Sampling**: There is no reference to *annealed importance sampling* [6], which I believe is highly tight to the idea of the paper. Besides [6], several relevant works [7,8,9] perform annealed importance sampling within the context of normalizing flows, falling within the same category as the CRAFT method referenced in the paper, though only marginally. What these methods do closely aligns with what the authors propose in the paper: instead of learning the target distribution in one step, they 'anneal' towards that distribution by learning and sampling from intermediate distributions, ensuring that the final learned probability density has as much support as possible, including regions where the target density is small enough to fall within the rare event regime. I believe it is crucial for this paper to be published in this or any other venue to highlight the connection to these (and the previously referenced works).
- **Rare Event Sampling**: A recent paper [10] discusses similar behaviors in training normalizing flows and combining them with importance sampling to ensure full support over the target density, including rare event regions. I would find it interesting if the authors commented on this work within the context of their findings. Some of the metrics and tools proposed in [10], such as the mode-dropping estimator, could also be used to assess the performance of a sampler in approximating regions of low probability where a shallow sampler is likely to lose some of the probability mass.
- **Idea of Anchor Points**: The notion of *anchor points* has implicitly been explored in some of the prior works mentioned above, albeit with a slightly different connotation that may have escaped the authors' attention. For instance, in the paper by Kanwar et al. [4] (Fig. 4), the authors use a technique very similar to what is suggested in this paper, although with slightly different connotations (e.g., they use previously trained flow-based models as starting (anchor) points to sequentially train more challenging distributions).
- **Additional Related Works**: Other closely related works, such as [11], are not mentioned in the manuscript despite having similar titles. This may cause confusion for potential readers.
- **Experiments**: I find the results presented in the paper not entirely convincing. Although the authors compared their approach to a large set of baselines, this alone does not seem sufficient to claim the superiority of the proposed method. I am surprised that the proposed approach is not compared against prior works, such as Annealed Importance Sampling with Normalizing Flows [7], and naive RealNVP training with a sufficiently large number of couplings and no anchor points.
As a side note, I strongly recommend that the authors conduct an extensive literature search to include and acknowledge existing prior works, and eventually, compare and discuss potential differences and similarities
Questions
- I'd like to see how the author would compare their work (and its corresponding novelty) to previous works. In particular, I'd like to see comparisons with Refs. [6-9] for the annealing aspect and Ref. [10] for the theoretical discussion regarding low-support regions (e.g., the rare event regime). Furthermore, discussing the differences concerning Ref. [11] would be helpful for the readers.
- I'd appreciate if the authors could perform an extensive literature search and create a Related Work section to place their paper in the context of existing prior works. Please refer to Refs. [1-11].
- I found the last paragraph in Section 3.1 and the discussion in Appendix B to be a bit unintuitive. It has been shown in the literature that using Forward KL, instead of Reverse KL, generally results in larger support and, therefore, has some benefits when combined with importance sampling. In that sense, I am surprised by the author's claim that training using Forward KL deteriorates performance. Do the authors consider the case where NO samples are given from the target density? If so, then I may understand this point. Otherwise, when a sample set from the target density, even if small, is available, it should be possible to show that training with Forward KL is feasible.
- It would be informative to see the density plot from Figure 4 for the other baselines as well.
- On page 8, referring to Figure 4, the authors write "[…] the right part further reveals that when increasing $N_{IS}$, the estimation could become even more accurate." This result does not seem neither novel nor unexpected. Indeed, it was already demonstrated in prior works, as seen in [1,5], that the variance of the importance sampling estimators scales with $N^{-1}$, with N being the number of samples. Could maybe the authors comment on this?
**Minor**
- The quality of the plots on pages 7-8 is quite poor. The axis labels are missing, and the font size for the x-y tick labels is too small.
- As a side note, I sometimes find the MK notation a bit confusing. However, I understand that it would require a substantial effort to rewrite the manuscript and adapt to a clearer notation. Nevertheless, this my be a feedback worth keeping in mind for the authors for future iterations of the manuscript.
- I find it somewhat unintuitive to completely relegate the discussion of the datasets to the appendix. Perhaps the authors could add corresponding references in the main text when mentioning the datasets and also refer to the Appendix for further details.
- In the conclusion, statements like *using nested subset events as bridges* agains strongly reminds of annealed importance sampling. I believe that a discussion comparing the present method to AIS, highlighting potential differences, or connecting them through their analogies is an essential element currently missing in the manuscript.
**References:**
- [1] [Nicoli, Kim A., et al. "Asymptotically unbiased estimation of physical observables with neural samplers." Physical Review E 101.2 (2020): 023304.](https://link.aps.org/accepted/10.1103/PhysRevE.101.023304)
- [2] [Noé, Frank, et al. "Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning." Science 365.6457 (2019): eaaw1147.](https://www.science.org/doi/10.1126/science.aaw1147)
- [3][Albergo, Michael S., Gurtej Kanwar, and Phiala E. Shanahan. "Flow-based generative models for Markov chain Monte Carlo in lattice field theory." Physical Review D 100.3 (2019): 034515.](https://journals.aps.org/prd/abstract/10.1103/PhysRevD.100.034515)
- [4][Kanwar, Gurtej, et al. "Equivariant flow-based sampling for lattice gauge theory." Physical Review Letters 125.12 (2020): 121601.](https://link.aps.org/pdf/10.1103/PhysRevLett.125.121601)
- [5] [Nicoli, Kim A., et al. "Estimation of thermodynamic observables in lattice field theories with deep generative models." Physical review letters 126.3 (2021): 032001.](https://link.aps.org/pdf/10.1103/PhysRevLett.126.032001)
- [6][Neal, Radford M. "Annealed importance sampling." Statistics and computing 11 (2001): 125-139.](https://arxiv.org/abs/physics/9803008)
- [7] [Midgley, Laurence Illing, et al. "Flow annealed importance sampling bootstrap." arXiv preprint arXiv:2208.01893 (2022).](https://arxiv.org/pdf/2208.01893)
- [8] [Wu, Hao, Jonas Köhler, and Frank Noé. "Stochastic normalizing flows." Advances in Neural Information Processing Systems 33 (2020): 5933-5944.](https://proceedings.neurips.cc/paper/2020/hash/41d80bfc327ef980528426fc810a6d7a-Abstract.html)
- [9] [Caselle, Michele, et al. "Stochastic normalizing flows as non-equilibrium transformations." Journal of High Energy Physics 2022.7 (2022): 1-31.](https://arxiv.org/pdf/2201.08862.pdf)
- [10] [Nicoli, Kim A., et al. "Detecting and Mitigating Mode-Collapse for Flow-based Sampling of Lattice Field Theories." arXiv preprint arXiv:2302.14082 (2023).](https://arxiv.org/pdf/2302.14082)
- [11] [Falkner, Sebastian, et al. "Conditioning normalizing flows for rare event sampling." arXiv preprint arXiv:2207.14530 (2022).](https://arxiv.org/pdf/2207.14530.pdf)
Rating
3: reject, not good enough
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.