Summary
The authors introduce a new training procedure for PINNs which are adapted to unknown measurement noise, i.e., a training procedure which works for any noise model. This is done via EBMs, which are trained jointly with the PINN. Here the EBMs estimate a 1d noise model based on the estimation of the PINN (conditional to the point $t_i$). Since they only estimate a 1d distribution, the (usually intractable) normalization constant can be estimated via numerical integration. The approach is tested on several (partial) differential equations and benchmarked against the standard PINN.
Strengths
The paper is easy to follow (except a few minor points). The idea is interesting and well-executed. The approach outperforms the standard PINN and offset PINN baseline. The experiments are well described, so that I think reproduction should be easy.
Weaknesses
1) While the idea is heuristically clear, it would be interesting whether one can obtain theoretical guarantees. I have got the hunch that it should be possible to cast the framework into one of expectation maximization (EM) algorithms (maybe one slightly needs to change the loss and train alternating instead of jointly). Did the authors give this some thought? This would greatly strengthen the paper in my opinion. For this see e.g. [1]
2) The discussion in 4.1 and 4.2 is a bit confusing. While I think I got the gist of it, please make clear what variables the functions $\mu_{\varepsilon}$ and $\theta_0$ depend on.
3) The metric logL is not clearly defined. How is that calculated in the case of a standard PINN, just Gaussian likelihood?
4) The non-Gaussian noise is a GMM. I would like to see physically more realistic noise models. One thing that could be interesting is whether this approach is able to learn mixed Gaussian noise, i.e., $y = f(t) + \eta_1 + f(t)\ \eta_2$ for normal $\eta_1,\eta_2$ with some variances. While this is still Gaussian, this is a noise model used in practice.
5) Please make the relation to model errors [2] and [3] more clear. Although the model error framework tries to solve a different problem (Bayesian inversion) the ideas are somewhat similar.
6) A very similar is to train a surrogate on the data only (no PINN loss), then estimate the noise via an appropriate model, such as an EBM and then to train the surrogate on a combined loss. Please comment on this.
[1] DeepGEM: Generalized Expectation-Maximization for Blind Inversion, Gao et al
[2] Iterative Updating of Model Error for Bayesian Inversion, Calvetti et al
[3] Noise-aware physics-informed machine learning
for robust PDE discovery, Thanasutives et al
Questions
See weaknesses. I overall like the idea and think it has a lot of merit. A consideration of more realistic noise models and some theoretical guarantees would strenghten the article imo.
Rating
6: marginally above the acceptance threshold
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.