Summary
The authors propose DeepLag as an approach to simulating Eulerian fluid dynamics, which makes use of Eulerian-Lagrangian co-design to improve performance. In particular, the idea is to transfer information back and forth between the Eulerian grid and initially randomly placed Lagrangian particles, which themselves exist on multiple but connected discretization scales. The intuition is that Lagrangian particles are better at capturing interactions with boundaries and, in general, tracking the location of material elements, e.g., smoke. Empirically, the gain in performance over existing models is consistent and around 10%. The limitations of the approach are its algorithmic complexity and hyperparameters.
Strengths
- The idea of having "attention" between the Eulerian grid and Lagrangian particles is interesting and seems to work well.
- The multi-scale approach makes a lot of sense to map fine details to the coarser resolution. Not sure whether this is only a strength though, as this means higher algorithmic complexity of the model.
- The approach by construction offers Lagrangian particle trajectories, which might be useful in e.g. ocean dynamics research
Weaknesses
- **Factual mistakes**: the authors make some very crude false statements here and there, and I'm questioning whether a person or LLM wrote sections of the manuscript, particularly the introduction.
- Lines 24-25: "curse of dimensionality" is NOT the reason for high computational cost in fluid dynamics! Please open the Wikipedia article on the curse of dimensionality and modify the sentence in the paper.
- Lines 40-42 state that "[Lagrangian approach] helps get around the CFL condition", which is absolutely wrong.
- **Related work taxonomy**: in section 2.2, a categorization of neural solvers is presented; however, this grouping of methods is new to me and, as much as I know, not what the community typically uses.
- The 1st group, "ODE-based Generative Models", refers to some rather old and outdated work,
- the 2nd group, "Neural PDE Solvers", actually only talks about Physic-Informed Neural Networks, and
- the 3rd group, "Neural Operators for PDE", basically lists all methods that I would call modern neural PDE solvers and what is typically referred to as neural operator learning (DeepONet, FNO, etc.) is mentioned along with GNNs and CNNs.
See PDE Bench [1] and PDE Arena [2] for a more modern categorization of common PDE learning approaches.
- **Ablations**: I'm missing an ablation on (a) the number of scales and (b) overall model size. Judging by Fig 7, in which the U-Net is 5x faster and also smaller, I wonder how a U-Net with similar (a) parameter count or (b) runtime would perform. I'm a bit worried that the claimed good performance might be just a bad hyperparameter choice of the baselines.
---
[1] Takamoto et al., "PDEBench: An Extensive Benchmark for Scientific Machine Learning", NeurIPS D&B 2022
[2] Gupta and Brandstetter, "Towards Multi-spatiotemporal-scale Generalized PDE Modeling", 2022
Questions
- Fig. 7: in this figure, it looks like the U-Net has fewer parameters than DeepLag, but in lines 345-346, as well as Table 10, you state the opposite. Can you explain that?
Limitations
- Thinking of Occam's Razor, it is questionable how many people would adopt the proposed approach, as it requires (a) a significantly more involved implementation effort than a typical U-Net, (b) tuning various hyperparameters like number of scales and number of particles per scale, and (c) is ~5x slower (Fig 7), while offering on average 10% performance improvement. Could the authors add some more hints on where their approach would be of practical interest? I would appreciate extending section 5 into, for example, one paragraph dedicated to the summary/strengths/applications, and one with the limitation/future work.