Answer to the questions
We appreciate the reviewer's comments and the thoughtful questions raised. Below, we provide a detailed response to address the concerns highlighted.
>whether the equations in your simulations could not be used to generate a physics based loss for your ode
The paper by Linka et al aimed at learning daily new COVID-19 cases (\hat{x}). Since it was evident that the data oscillates in time, the authors stabilized the prediction of \hat{x} by constraining a neural network (NN) model (which takes time t as input and outputs x) to exhibit a similar behavior to a harmonic oscillator model (where f\{x\} := \ddot{x} + (c/m)\dot{x} + (k/m)x = 0 for x). Specifically, the constraint was applied by adding the term f\{x\}^2 to the loss function. Using this approach, the authors successfully predicted future time series which is not included in the training data.
In contrast, our work aims at studying a system where we can not use a term like f\{x\}^2. The very purpose of the learning process is to discover the model we used to create the training data. We used one of the models that has been proposed to explain the collective behavior of cells. However, this model does not fully account for the characteristics observed in the experimental data we aim to analyze, such as phase separation-like behavior between different cell types. Additionally, when considering other systems, both the collective behavior as a whole and the individual cell behavior differ significantly between systems, lacking any unifying pattern. So far, no universal model nor principle governing these systems have been found. Given this background, we avoided assuming a specific model in the training process.
> what I understand from your paper section 5.2 (last paragraph) the model is overfitting on the interaction forces, … adding some form of physics loss can help regularise and allow models to extrapolate …
The last sentence of Section 5.2 may have caused confusion. There was no overfitting but rather, it was underestimation described here. We will more clarify this point in the final version.
We should note that the extrapolation performed in the Linka et al paper is indeed achieved in our study. In Section 5, This is demonstrated by our model’s ability to reproduce the dynamics of collective motion from initial states not included in the training data.
>a neural network is a universal approximator, so … it is not entirely surprising
While the universal approximation theorem guarantees the existence of a solution where a NN can approximate any function, this does not imply that such a solution can be found, nor that the solution will generalize well to similar data. Our case is an inverse problem where each individual in a group is influenced by forces from numerous neighbors. The challenge is to estimate individual forces from the integrated trajectories observed, which are the result of summing and integrating those forces. It is not guaranteed that such forces can always be computed, and even if the computation converges to a solution that explains the trajectory, due to the nature of inverse problems, this does not ensure that the individual forces before summation are correct. However, we have confirmed that our computations do converge, and that the individual forces are accurately estimated.
>what the longterm projection of this work is?
In biological physics, many including us are attempting to address various types of collective motion exhibited by a group of groups of cells. Understanding such dynamics is fundamental to knowing how tissues form. Our present approach should be applicable in extracting individual cell behavioral rules from real data.
>How would they validate that the model works?
This is related to the above comments. Our plan is to test this approach by applying real experimental data and from there, make corrections and improve the assumed model.
>why is the graph neural ODE paper (Poli et al 2019) not relevant?
The method by Poli et al is relevant to the present work. We will clarify the difference between our method and theirs in the camera-ready version.
Poli et al addresses systems where the graph structure changes much slower than ours. Their extrapolation is limited to a few steps during which the edge structure remains almost unchanged. If the prediction extends beyond a few steps, the predicted node states deviate from the correct values, leading to inconsistencies with the edge structure. To perform long-term predictions with their method, one would need to compute the neural ODE over short intervals, updating the edge structure based on the resulting node states, and then repeat the neural ODE computation for another short interval. This approach is inconvenient and assumes that the edge structure remains unchanged during the neural ODE computation.
In contrast, our method does not require pre-defined edges, allowing one to make long-term predictions based solely on the initial state of nodes.