Summary
This paper explores the use of energy based models to modeling the distribution of jets in high energy particle physics. Here, the data consists of a vector of events, each of which is described by a momentum transverse to the beam and 2D spatial coordinates. The goal is to learn the true distribution of jets from real data via an energy based model, which can subsequently be used for Out-Of-Distribution detection and classification, in the manner of the paper "Your classifier is secretly an energy based model and you should treat it like one".
The model is trained with gradient ascent, using Langevin sampling to estimate the intractable expectation in the loss function, and making use of the (persistent) Contrastive Divergence of the loss. An additional KL term to minimize the difference between the true model distribution and the Langevin approximation is used.
Strengths
The paper appears to achieve success in a difficult domain of application. Their validation suggests that the energy based model they train successfully captures the desired data distribution, at least up to well-known features. Furthermore, they show that the energy based model can be altered to a supervised setting, and used as a classifier.
The use of the transformer as the neural architecture to model inter-particle interactions also appears to be a successful choice.
Weaknesses
The ideas in this paper were often conveyed poorly, and it was quite hard to understand the technical details, either of the energy based model training procedure, or of the scientific problem in question.
In equation 2, x+ and x- were not clearly explained, making the paper hard to read for someone unfamiliar with the details of contrastive divergence. Using the expectation notation of (in Latex) $E_{x^+ \sim p_D}$ would have helped clarify that the x+ are drawn from the data distribution and the x- from the (approximation of) the model distribution.
Algorithm 1 describes the "stopping gradient", but more information is needed here.
The equation in Figure 1 are blurry - it needs to be redone in higher resolution.
More explanation is needed for the anomaly detection task. I did not understand the set up in adequate detail. You say: "With an energy-based model, which is trained on the QCD background events or directly on the slightly signal-contaminated data, we expect unseen signals to have higher energies and correspondingly lower likelihoods". But what is a signal vs a background event here? This needs to be clear for a non-HEP audience.
What the labels are in the classification task is unclear.
The paper notes that "A more intriguing property of EBMs is that spurious correlations can be better handled", but it wasn't clear to me either what spurious correlations referred to here, or why EBMs would be better.
"More interestingly, compositionality can be easily incorporated within the framework of EBMs by simply summing up the energy functions ": while I understand what is meant here, the word "compositionality" means many things in different contexts, and is too vague. While other papers on EBMs have used "compositionality" in this sense, you should either cite them, or explain directly what you mean by it.
Questions
Which are the original contributions of the paper on the machine learning side, if any? In particular, is the particular version of Contrastive Divergence used, (in the section KL Divergence-Improved EBM Training) key to your success here? I'd like to see more discussion of this KL term, which is quite interesting.
I would be interested to know how this paper fits into a larger research project. Are there further directions that could be pursued, in terms of applications? Did the authors find the approach satisfactory, or were there limitations that concerned them?
Suggestions:
I was able to understand the structure of the paper fully only after reading the following two papers: https://arxiv.org/pdf/1912.03263.pdf and https://arxiv.org/pdf/1903.08689.pdf. The first describes the three applications of energy models you are interested in (generative modeling, OOD detection, and classification). The second describes the Contrastive divergence approach, and cites detailed notes which explain its derivation; your training algorithm appears to be a modification of their Algorithm 1. I would summarize your paper as "an application of the approach of these papers to HEP Jet modeling". While you do cite both of these papers, I would recommend highlighting them more, since they form the backbone of your approach and will help the reader to follow along.
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Limitations
Are there ways of incorporating domain specific models of HEP into the current approach? Currently it seems that the physics incorporated into simulators is not leveraged here. It would also be interesting if known expressions or heuristics for the energy could be incorporated into the energy function.