End-To-End Latent Variational Diffusion Models for Inverse Problems in High Energy Physics

High-energy collisions at the Large Hadron Collider (LHC) provide valuable insights into open questions in particle physics. However, detector effects must be corrected before measurements can be compared to certain theoretical predictions or measurements from other detectors. Methods to solve this \textit{inverse problem} of mapping detector observations to theoretical quantities of the underlying collision are essential parts of many physics analyses at the LHC. We investigate and compare various generative deep learning methods to approximate this inverse mapping. We introduce a novel unified architecture, termed latent variation diffusion models, which combines the latent learning of cutting-edge generative art approaches with an end-to-end variational framework. We demonstrate the effectiveness of this approach for reconstructing global distributions of theoretical kinematic quantities, as well as for ensuring the adherence of the learned posterior distributions to known physics constraints. Our unified approach achieves a distribution-free distance to the truth of over 20 times less than non-latent state-of-the-art baseline and 3 times less than traditional latent diffusion models.

Paper

References (61)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer GAKk7/10 · confidence 4/52023-06-25

Summary

The paper introduces a diffusion model based approach to tackle inverse problems. The method is then applied to High Energy Physics in order to reconstruct kinematic quantities.

Strengths

The paper is well written with clear structure, definitions, and figures. The paper includes testing of the proposed methodology, VLD, on an important HEP inverse problem, which is unfolding semi-leptonic tt events. It is benchmarked on this task against other state of the art algorithms such as CINN, LDM, and VDM, as well as variations of the authors own proposed algorithm. Their VLD algorithm does appear to significantly outperform the other considered algorithms. Another strength of this paper is the fact that the authors are deliberate in construction of their network, explaining the advantages and necessity of each of the components.

Weaknesses

Limited testing of the proposed unifying architecture is a weakness of the paper. Indeed, the authors do acknowledge that further testing on different event topologies would be beneficial, yet including at least one more benchmark would increase the confidence in the algorithm’s performance.

Questions

1. Page 4, line 143, typo, should be “in an abstract” 2. Page 6, line 204, please define E to be energy, m mass, p momentum, and mention that you are using the HEP convention of setting c to be 1. 3. Please discuss the reason why UC-VLD outperforms the VLD on some metrics and vice-versa. 4. Figure 4, b quark, I don’t really see the bimodal nature of the distribution 5. Figure 4, neutrino, how is the neutrino “truth” line showing the bimodal nature? 6. What would happen with the LDM’s performance if a different prior weight is given? 7. What would happen in the benchmarks if the CINN uses the MMD loss instead? 8. To further show the improvement of the described unified training loss, it would be beneficial to show what would happen if the various components would be trained in parallel.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

Yes

Reviewer sYuc4/10 · confidence 4/52023-06-28

Summary

This paper proposes a unified framework, which combines the latent diffusion model and variational diffusion model, and applies the method to the inverse problem in the field of High Energy Physics. The loss terms for VAE and variational diffusion model are combined to achieve end-to-end training. The proposed VLD and corresponding variants achieve state-of-the-art performance as shown in experiments.

Strengths

* A unified framework to support end-to-end training for latent variational diffusion model. * The proposed VLD and variants achieve state-of-art performance existing generative models.

Weaknesses

Overall, the main contribution of this paper is the unified framework to combine latent diffusion model and variational diffusion model, where the core lies in the extra VAE loss (3rd term in Equation 8). 1. Technical novelty seems limited, since the only contribution could be summarized as an extra loss term in diffusion model. 2. The experiments to emphasis the importance of the extra loss term are limited. Among all the baselines, the LDM at L243-245 is the most similar method to the proposed one, where the only difference is the extra loss term that leads to an end-to-end training process. The small gaps between LDM and C-VLD in Table 1 also suggest the similarities. The main question is, is the comparison between LDM and VLD fair enough? The details about training LDM are missing, such as whether they adopt the same feedforward block as VLD, especially how to pretrain the VAE and what is the performance of the pretained-VAE. Existing experiments seem insufficient to support that the benefits of VLD are from the unified training process. 3. Since there are results for C-VLD and UC-VLD, which mainly differ in the conditional signal, is it possible to compare LDM with similar settings?

Questions

* How to define that the dataset at L273 includes enough variations for the high energy physics? For example, is there any existing similar settings for reference?

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

Technically, this paper provides a unified way, which seems like a straight-forward combination of the loss terms, to train a latent variational diffusion model in an end-to-end manner. However, experiment results are insufficient to support the importance of the extra loss term, or the benefits brought by the unified training process.

Reviewer Btsg9/10 · confidence 5/52023-07-05

Summary

The paper benchmarks several network architectures on a real-life problem in particle physics, i.e. the problem of inverting the effect of limited detector resolution and guess the features of a given collision from what is actually observed in the detector (unfolding). Considering several metrics to assess the accuracy of a given unfolding, the authors show that a novel architecture for diffusion models provides the best performance.

Strengths

Very solid analysis, with clear explanation of the various steps. Shows potential progress in applications, thanks to novel architecture

Weaknesses

none

Questions

none

Rating

9: Very Strong Accept: Technically flawless paper with groundbreaking impact on at least one area of AI/ML and excellent impact on multiple areas of AI/ML, with flawless evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

none

Reviewer aaox4/10 · confidence 4/52023-07-06

Summary

Thanks for the author's rebuttal. I have read all of the rebuttals and reviews and decided to keep the current rating. Current work proposed an extension of the variational diffusion model, called the variational latent diffusion model. It is applied to inverse problems in the high energy physics field and tested on a single topology. The result shows that the distance is three times less than current latent diffusion models.

Strengths

Originality: The model is a combination of two existing models, the latent diffusion model and the variational diffusion model. Moreover, the author defined an appropriate loss function to train the model. Quality: The result demonstrated that the proposed model outperforms other baseline models. Significance: The experiment provides unique data in the high energy physics field.

Weaknesses

The paper has several typos and unclear points. Moreover, some claims are not well supported by the experiment results. The reviewer is confused to some presented points. Details can be found in the questions part.

Questions

1. There is no general caption for Figure 1. Moreover, the sub-caption is not consistent with the sub-figure. For example, ($e$) in caption while ($e^{-}$) in the subfigure. ($\nu$) in caption while $\bar{v}$ in subfigure. $d$ in caption while $\bar{d}$ in subfigure. Moreover, subfigure (b) is too small to see the detail; the reviewer did not get the point of how it is related to your problem or the model. The font in this subfigure is too small to see. The review suggests presenting the subfigure (b) in a better way to give more insights into your problem or model. 2. There are several typos in the paper. For example, line 8, ``latent variation diffusion model``, is supposed to be ``latent variational diffusion models`` Line 192, the $\hat{\cdot}$ is on $z_t$ or $t$? 3. Some definitions are not clear. For example, how did you calculate $\hat{E}$ and $\hat{p}$ from the predicted value in Equation 10? What is the norm of the $\lambda_c|\cdot|$ in Equation 10? Is it $L_1$ norm or $L_2$ norm? 4. The author uses a deterministic encoder and claims it is better than the variational encoder. According to the author, the latter only has limited benefits while increasing training variance and complexity. However, there is no experiment result to support the claim. More ablation study is needed to justify the claim. Still, it sounds strange to the reviewer that the model is called a variational latent diffusion model but only uses a deterministic encoder. 5. The author proposed three variants of the model. Conditional, unconditional, and the last one, conditional encoder and unconditional decoder. The way of presenting the results confuses the reviewer. Firstly, which one do you promote to use in the conclusion? It seems in Table 1. VLD is better in the latter three metrics, while the UC-VLD is slightly better in the first three metrics. However, Figure 2 shows the framework of VLD, while all the Figures in the result part and the appendix part only show the results for the UC-VLD. If the authors want to promote VLD, consider adding relevant plot results for VLD. If the UC-VLD is better, consider changing Figure 2 to reflect this point. 6. A physics-informed consistency loss is proposed with a hyperparameter $\lambda_{c}$; how did you adjust the value of this hyperparameter? Moreover, what would be the effect of including and not including this loss? 7. The author claims the model is aimed at high-dimensional inverse problems. However, the training cost for $55$ dimensional variables is expensive, $24$ hours. Moreover, the designed latent space has a higher dimension than the input instead of the commonly used lower latent space dimension for compression. This design could introduce additional costs. And the reviewer is concerned with the scalability of the current model to higher dimensional problems. 8. What is the limitation of the current work?

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

The author did not mention the limitations explicitly. I would be curious what would be the limitation of the proposed model.

Reviewer GAKk2023-08-10

Dear authors, Thank you for the time taken to write the rebuttal and addressing the points I raised.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC