Summary
The paper addresses challenges in training physics-based neural networks when dealing with partial integro-differential equations. Evaluating integral with naive approximation methods leads to bias loss and solutions. As a remedy, the paper proposes three potential solution approaches: the deterministic sampling approach, the double- sampling trick, and the delayed target method. A set of experiments on three benchmark problems confirms the proposed methods (in particular, the delayed target approach) produce solutions that are comparable to the standard sampling approach with many samples.
Strengths
- The paper is overall well-written (but there are some confusions due to the structure and typos, which will be elaborated further in the next item, weaknesses).
- The motivation is clear; an extension of applications of PINNs to integro-de problems.
- The experiments are conducted in a manner that ensures the resulting outcomes serve as compelling empirical evidence showing that the proposed method greatly improves sample efficiency over the regular sampling methods.
Weaknesses
- Some of the proposed methods (the three methods) seem to work well with the benchmark problems. At the same time, some of them do not seem to work very well with the benchmark problems and it is a bit hard to find what would be a take-away message. For example, double-trick does seem to work for the Maxwell problem, but struggles with Poisson problems (Figure 2). Also, the results of them are not provided for the third benchmark problem. The delayed target method seems to work well with the first two example problems, but for the third problem, it’s hard to see if the proposed one does better than the standard one. In particular, by looking at the training loss curve, it does seem that the proposed method produces higher loss (or is it a coloring issue?). Although there is a small paragraph of recommendation and limitation at the end, it only mentions the delayed target method and it is still less convincing due to the reasons above described.
- The authors provided in-depth ablation studies on the Poisson problem, which can be considered as strength. However, it’s less clear if these findings from the one benchmark problem would also generalize to other intro-DE problems. It would be better if the authors could provide some insights or some paragraph describing tips to set up those hyper-parameters.
- Discussion on computational aspects would also be needed. Computational wall-time would be the best, but at least some discussions on flops would be okay.
- Although the paper is well-structured (section-wise), there are some room for improvements for the presentation.
- It seems that there should be some break after Eq. (9). Right after Eq. (9) is the description of a loss term for all three examples.
- Figure 2 describes the results of the delayed target method, but the text reads differently in one place: Page 5 in the paragraph of “The deterministic sampling strategy”, “For a demonstration, Figure 2 shows a number of example sets used for applying the divergence theorem to the Poisson problem.”. Also, in Figure 2, is the pointer to Eq. 24 correct?
- Figure 6: it’s hard to interpret the figure on the right panel.
- Recommendation and limitations paragraph: Figure 2 seems to show the performance of the model for varying $M$, not about $\gamma$ or $\lambda$. Should $M$ be interpreted as the hyper-parameters of the proposed methods like $\gamma$ or $\lambda$? $M$ does seem to be a problem-specific parameter. Does this suggest that there can be other problem-specific parameters which needs to be tuned when one want to use the algorithm?
- The placement of Figures would require many back-and-forth, which makes it difficult to focus.
- Conclusion: there is an incomplete sentence: “The delayed target method.”
- Table 3, is it SiLU? Or Tanh? I think there is a mismatch in the text and the table entry.
Questions
- There are questions in the Weakness section. Please refer to them.
Additional questions would be:
- Would there be any insight or guidance on how to properly choose the hyper-parameter? Or would it be problem specific?
- Relatedly, would there be any reason why the authors test values of $N$ only from $\{1,100\}$?
- From the related work, it seems that there is some related work to solve the same problem. How would this method compare to those methods? Variational-PINNs (VPINNs) or hp-VPINNs seem the closest ones. Do the authors have a sense on how these related work perform on the benchmark problems?
Rating
5: marginally below the acceptance threshold
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.