Physics-informed neural networks for transformed geometries and manifolds

Thank you for your review. We would like to address the following comments. _One of the major weakness is that the paper does not include experiments of comparison between modified PINN and vanilla PINN. In order to show the effectiveness of the proposed method, the author should also test the performance of PINN on all the problems in section 4._ Unfortunately, we do not understand how vanilla PINNs should be applied to the case of manifolds, as directional derivatives have to be computed. 
It might be possible to elaborate the differential operators (e.g., Laplace-Beltrami) explicitly and compute those in a “vanilla” way, but this is what our approach implicitly does, formulated in local coordinates, putting all the work into the automatic differentiation. In the case of equi-dimensional transformations, vanilla PINNs could actually be applied and would lead to a well-studied reference solution. Thank you for this suggestion. _If the original problems $L(u)=f$ in $\Omega$ is transformed to $L_x(u \circ \phi) = f$ on reference domain $\Omega$ ref, then $L_x$ should not equal $L$. The calculation of $L_x$ should use chain rule. In your paper, this part is hardly touched. How did you actually implement your method in experiments?_ In the manifold case, chain rule applies and is carried out by the automatic differentiation framework. In the transformation case, we explicitly neglect the chain rule. It’s somewhat arbitrary (and we are sorry if we didn’t make this sufficiently clear), but this idea leads to general transformed domains. Regarding the actual implementation: We provide our full source code as supplementary material and refer to it for all implementation details.

Thank you for your review. We would like to answer your comments and questions as follows. $\mathcal{L}_x$ _and_ $\mathcal{L}_y$ _need more explanation. The subscripts have not been explained. Diagram conveys that one is in the reference domain and other is in the computational domain yet it's better to write near the equation (4)-(5) and following equation._ Thanks for your feedback, other reviewers addressed the same issue. We only stated “$\mathcal{L} = \mathcal{L}_y$ with respect to global coordinates” and assumed that it’s clear that the subscript indicates the derivation variable. Obviously, this it is not the case and the subscript needs more explanation. _Exact boundary condition with output transform: Kindly help me understand the approximation of $\hat u$, given that the inverse must hold and the proposed approximation is not linear._ As the smooth distance function $b$ is zero at the boundary, $N(y) u(y)$ is zero on the boundary, and, therefore, $\hat u(y) = g(y)$ satisfies the Dirichlet boundary values $g(y)$ at the boundary. 

Unfortunately, we do not understand what you are referring to with ‘inverse must hold’, and why the approximation should to be ‘linear’. _I am not entirely convinced with the imposed boundary condition. What could be considered a weak boundary condition is not fully exposed in the paper._ A weak boundary condition (in the context of PINNs) means imposing the boundary condition by adding a penalizing loss term, as outlined in (8). You noted correctly that we missed introducing this wording in the context of (8) and it should be added. _Experiments are limited on toy-example and missing on the manifolds which are widely used in science and engineering application._ We tried to demonstrate our method on minimal working examples to make the setup and implementation clearly understandable. It’s unfortunate if you consider them as too limited, and we will take this feedback into account. _The paper misses to highlight the limitations of the proposed approach._ Unfortunately, that is true, we will add a paragraph on this. _Furthermore, I request the authors to perform some experiments and analysis of the proposed theory on negative curvature surfaces with the introduced local approach. Also, the use of Laplace-Beltrami operator for shapes._ We do not see why negative curvature should have any relevant impact on our method, the proposed method also works with negative curvature surfaces. 
Also, our second example demonstrates a Poisson problem on a part of a sphere, which - as we formulate the derivatives in local coordinates - corresponds to a Laplace-Beltrami on the manifold. Our apologies that we didn’t point this out explicitly. _In addition to the above, experiment on Low-Dimensional manifolds is simple and not convincing to me for real application. I request the authors to provide some analysis and results on popular manifolds such as low-dimensional SPD, Grassmannian manifolds, etc._ Which PDEs are commonly formulated on low-dimensional SPD or Grassmannian manifolds?

Thank you for your review! Let us address your comments as follows. _Unfortunately, it appears that the problem tackled in the paper is somewhat incremental, and the proposed solution lacks a surprising or profound aspect. In the context of an ICLR paper, I'm seeking a novel problem that has not previously been successfully addressed yet, made attainable through this approach, or a novel method to solve a well-established problem that has been extensively explored. Unfortunately, neither of these elements seems to be present in the paper._ Thank you for your valuable feedback. We agree that one can consider the problem tackled somewhat incremental, and that our work does not propose a novel method to solve a well-studied problem. We tried to introduce the approach of including a diffeomorphism within PINNs as a (novel) general concept, and we believe it could benefit from a common introduction. Furthermore, we consider the application of PINNs to manifolds as a problem that has not been successfully addressed, which is made attainable through this approach. A surprising aspect of our approach is that - in the transformation case - a latent representation of the PDE solution on the reference domain arises. This is likely to improve generalization capabilities for parametrized geometries, but we recognize that we were not able to demonstrate this effectively on a well-studied problem. _Furthermore, the examples provided mainly consist of small-scale 2D toy examples. To comprehensively assess the efficacy of this approach, it would be necessary for the authors to set up larger-scale problems that are well-documented in CFD/JCP/CMAME papers._ As mentioned above, we tried to introduce the approach as a general concept and, therefore, we set up easily understandable examples that demonstrate the efficacy of the approach. We tried to choose examples with a clear setup and analytical solutions, accompanied by a transparent and manageable implementation. However, we accept that there is a request for applying our method to larger-scale, well-documented problems that would comprehensively assess the efficacy for non-toy examples. _How does this work compare with Bonev+ ICML 2023? These authors propose a neural PDE approach using spherical coordinate. Your paper seems to be more general. Can you reproduce some of the examples in their paper so we can have an apple to apple comparison?_ Bonev+2023 propose Spherical Fourier Neural Operators (Spherical FNOs) that put the concept of FNOs efficiently onto spheres. As they use spherical harmonics for the Fourier transform, their approach is limited to spheres by definition.
 Our approach is more general in the sense that we can model arbitrary manifolds. We think an apple-to-apple comparison doesn’t make sense, because it would be restricted to the case of spheres where the SFNOs will clearly outperform all aspects of our method (accuracy and speed) as it is explicitly tailored to this case.

Thank you for your review! Allow us to respond to your comments as follows. _The method relies on the output transformation trick to enforce boundary conditions (BC), which is well suited for Dirichlet BC only. It would not be applicable as is for different kinds of BC, but the authors have a much more general claim._ We are not sure which claim you are referring to. Does this refer to “For simplicity, let us denote Dirichlet boundary conditions only”?

 We are not claiming that our approach is applicable to different kinds of BC than Dirichlet ones, and we definitely didn’t intend to let it appear more general. Besides, regarding Neumann boundary conditions, one can use the domain transformation to compute the outer normal of the transformed geometry, but it’s unclear how to strongly impose Neumann boundary conditions, as it’s not clear for PINNs in general. _Except for the last example, which we will discuss next, the diffeomorphism is known a priori. Therefore the method in such case simply looks like a change in variable with a known function. How can you apply this method on a domain which is not equipped with such a transformation?_ In our work, we assume that the diffeomorphism is given, and we assume that the domain geometry is defined by the mapping of the reference domain via the diffeomorphism. If, vice versa, a geometry is given, e.g., by a mesh, the diffeomorphism might be learned, but it’s definitely an open question how this would be achieved effectively. _The last example is very mysterious to me. I actually do not understand what the method is supposed to achieve by learning simultaneously to impose the PDE constraint and the geometric transformation. Do we know what target geometry the network should converge to ? Besides, the network that learns the transformation is not a diffeomorphism, so there is no guarantee that the optimization problem finds a correct solution._ In the last example, we tried to make the mapping more general by using an NN as transformation (targeting the concern of the limitation that the diffeomorphism has to be known a priori, as in your previous comment). It was intended as an explorative example to show what could be done incorporating a parametrized transformation. We believe that we were able to demonstrate that this idea results in a geometrically very flexible PDE solver, e.g., in contrast to classical FEM solvers that would require re-meshing. However, as you mentioned (and we stated clearly), the network has no guarantee to be diffeomorph and this stretches the framework we outlined beforehand beyond its assumptions. We agree that our objective function - not including an explicit target - is somewhat arbitrary and that we are not able to state analytically what the network should converge to. Experience with PINNs suggests that a convex shape is easiest to optimize for w.r.t. weak boundary conditions, and this is what our example actually results in. As this example opens too many concerns, we conclude in removing it completely upon further investigation. _The authors do not compare their method with any existing work. There is no literature review. As a result, we do not really understand why these problems cannot be tackled with existing methods. Why do they fail ?_ We recognize that our examples need more comparisons with similar problems from literature.

 Unfortunately, to the best of our knowledge, we are not aware of previous works that applied PINNs to manifolds, which severely limits the possibilities of a comparison as well as a literature review. _The authors do not provide any numerical results for their methods, and even the qualitative results do not include the ground truth solutions. It is therefore impossible to judge the effectiveness of the method._ / _What does the following sentence mean? "transformed PINN finds the exact length with an error of = 0.1 %”_ The first example is constructed in a way that the numerical solution can be compared to an analytical solution. We were not providing a plot for both solutions, because they are trivial and closely aligned s.t. a plot didn’t make sense. Instead, we stated that the numerical solution fits the analytical solution “with an error of 0.1%”. This is what the sentence meant, sorry for the incomprehensible expression. We acknowledge that our work is lacking analytical results for example 2 and a comparative study for example 3. _What is the difference between L, L_x and L_y, and concretely for each example ?_ The subscript indicates the derivation variable of the differential operator. As other reviewers addressed the same issue, we have to make this more explicit along with equations (4)-(5).

 The examples are split into manifold and transformation cases, where manifold corresponds to L_y and transformation to L_x. We will provide more clarity here.