Summary
This paper proposes a Bayesian framework for shape optimization that uses a "reentrant" GNN. The reentrant GNN iterates over its own outputs to refine the predictions, and predicts the approximated surface pressure as well as a key metric, such as the drag for cars and lift-to-drag ratio for airfoils. At inference, the slower convergence rate of the reentrant GNN for out-of-distribution shape serves as a proxy to prediction uncertainty.
Strengths
The paper tackles an important topic, namely uncertainty quantification of surrogate models for geometric design optimization. They have conducted a set of experiments on a dataset of airfoil shapes, and car shapes. The car dataset appears to be particularly challenging since the latent geometric parameter is high dimensional. The model with the entrant GNN obtains good experimental results in predicting the drag and lift-to-drag coefficients, especially when there are a few training samples.
Weaknesses
First, the paper is really dense and quite hard to read. I would suggest some rewriting to make it less compact.
Then, I am not convinced by the approach taken by the authors w.r.t the output of the surrogate model. Here, the surrogate model learns to predict both the surface pressure and the metric value. Even though this metric is computed from approximate pressure values with another neural network, it still breaks the physical integration between the pressure field and the different forces. To foster generalization, the GNN should capture the physical phenomenon through the pressure and other volume fields if necessary, and compute the metrics from them through integration. This leads in a trustful and physically-based metric, which is not the case with this method. It is claimed and showed in the appendix that using the integration of the pressure values has a higher MAE for the considered metrics. This is not surprising as regressing a single value is less challenging than predicting the pressure on the whole surface. However, if a surrogate model were to correctly predict the pressure on the surface, then the drag or lift-to-drag would be actually meaningful.
Besides, the reentrant GNN does not provide uncertainties on the predictions, but only a proxy for it. This is a key and crucial difference with the other methods. Techniques such as Deep Ensembles and MC-Dropout provide approximate Bayesian inference in deep gaussian process, while the reentrant GNN does not. If we take twice the same shape, the output will give the same prediction both times.
As a consequence, the motivation behind the reentrant GNN is not transparent to me. It seems to be a rapid and not ingenious solution with a wrong tool.
I tend to agree on the receptive field analysis. A reentrant GNN with $k$-iterations of depth $d$ should have the same receptive field as a simple GNN with depth $d*k$, and a higher receptive field than a simple GNN with the same depth. Still, in Figure 5 it seems that it converges most of the time in 1 step for in-distribution shapes, and thus the gain in performance seems marginal. The same can be deducted from the comparison between reentrant GNN and GNN ensembles which are both very close in prediction performance.
Questions
What are the metrics given in Table 1 ? Could you provide an explanation for them ?
How do you modify the inputs with Deep Ensembles ?
How do the ground truth pressure and predicted pressure compare ? How the pressure evolve over the iterations?
What is the initial performance estimate ?
Rating
3: reject, not good enough
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.