Integrating GNN and Neural ODEs for Estimating Non-Reciprocal Two-Body Interactions in Mixed-Species Collective Motion

Analyzing the motion of multiple biological agents, be it cells or individual animals, is pivotal for the understanding of complex collective behaviors. With the advent of advanced microscopy, detailed images of complex tissue formations involving multiple cell types have become more accessible in recent years. However, deciphering the underlying rules that govern cell movements is far from trivial. Here, we present a novel deep learning framework for estimating the underlying equations of motion from observed trajectories, a pivotal step in decoding such complex dynamics. Our framework integrates graph neural networks with neural differential equations, enabling effective prediction of two-body interactions based on the states of the interacting entities. We demonstrate the efficacy of our approach through two numerical experiments. First, we used simulated data from a toy model to tune the hyperparameters. Based on the obtained hyperparameters, we then applied this approach to a more complex model with non-reciprocal forces that mimic the collective dynamics of the cells of slime molds. Our results show that the proposed method can accurately estimate the functional forms of two-body interactions -- even when they are nonreciprocal -- thereby precisely replicating both individual and collective behaviors within these systems.

Paper

Similar papers

Peer review

Reviewer Sm497/10 · confidence 1/52024-07-11

Summary

This work proposes a graph neural network-based deep learning model for two-body interactions, which has better efficiency based on two numerical experiments.

Strengths

Two-body interactions are not my research field. Thus, I don't have enough expertise to evaluate this work. I sent an email to AC to re-assign this paper when I received the assignment. Unfortunately, I didn't receive any feedback. My justification is just based on the writing without any methodology consideration.

Weaknesses

see strengths

Questions

see strengths

Rating

7

Confidence

1

Soundness

3

Presentation

3

Contribution

3

Limitations

see strengths

Reviewer T7k95/10 · confidence 4/52024-07-11

Summary

This paper presents a model combining GNN and Neural ODEs for the task of two-body interaction prediction. Two numerical experiments were conducted to evaluate the model's performance.

Strengths

- Using a GNN and Neural ODE to describe the dynamics of two-body interaction state changes is well-motivated and reasonable. - Experiments from interaction models are implemented from simple to complex setups, showing the efficiency and capacity of the proposed method. - The paper structure and writing is clear and easy to follow.

Weaknesses

One of the main contribution claimed in this paper (as well as in its title) is the combination of GNN and Neural ODE, but this approach has been studied in multiple works back since 2020, the authors seem to overlook the important and related body of works [1,2,3,4]. [2] is a direct competing method (GDE) where the authors proposed a model integrating graph NN and Neural ODEs. GDE-related applications exist as well, for example, social network embedding [5], action recognition [6]. Given the missing discussion of related work, and my concern about the major technical contribution claim in this paper, I do not see much merits of this study to be novel and interesting to the community. [1] L. Xhonneux et al.: Continuous Graph Neural Networks. PMLR 2020. [2] M. Poli et al.: Graph Neural Ordinary Differential Equations. AAAI 2020. [3] L. Chen et al.: Signed Graph Neural Ordinary Differential Equation for Modeling Continuous-Time Dynamics. AAAI 2024. [4] A. Han et al.: From Continuous Dynamics to Graph Neural Networks: Neural Diffusion and Beyond. TMLR 2024. [5] Y. Zhang et al.: Improving Social Network Embedding via New Second-Order Continuous Graph Neural Networks. KDD 2022. [6] L. Pan et al.: Spatial-temporal graph neural ODE networks for skeleton-based action recognition. Scientific Reports 2024.

Questions

The author should discuss in their response regarding the missing related works that involve integrating GNN+Neural ODEs, and highlight their differences/contribution from previous work.

Rating

5

Confidence

4

Soundness

3

Presentation

2

Contribution

2

Limitations

Limitations were discussed in the paper.

Reviewer Shpk7/10 · confidence 4/52024-07-12

Summary

The paper introduces a novel deep learning framework for estimating two-body interactions in mixed-species collective motion. The authors combine graph neural networks (GNNs) with neural ordinary differential equations (Neural ODEs) to predict interactions between pairs of entities based on their states. This approach represents the system as a dynamic graph, using GNNs to efficiently calculate interactions and Neural ODEs to learn system dynamics. Authors demonstrate their method via two experiments: a simple harmonic interaction model and a more complex model simulating cellular slime molds. The framework successfully estimates interaction functions and replicates both individual and collective behaviors in these systems. The authors provide detailed quantitative analyses of estimation accuracy and demonstrate the method’s ability to generalize to single-species scenarios after training on mixed-species data.

Strengths

Strengths: - Novel integration of GNNs and Neural ODEs for collective motion analysis - Ability to handle mixed-species systems and estimate species-dependent interactions (and demonstrating its effectiveness across simple and complex models) - Successful generalization to single-species scenarios after training on mixed-species data - Potential applicability to a wide range of biological systems exhibiting collective motion

Weaknesses

Weaknesses: - High computational cost and long training times . - Current limitation to deterministic motion equations and pairwise interactions. - Lack of ablative studies on hyper parameter optimizations and method’s sensitivity to noise. - Lack comparative analysis with existing methods for estimating collective motion dynamics.

Questions

Questions for the authors: 1. How does the performance of your method compare to existing approaches for estimating collective motion dynamics, such as SINDy or Bayesian optimization methods? 2. Have you explored the method’s sensitivity to noise in the input data? How robust is the estimation process to measurement errors or stochastic fluctuations in the trajectories? 3. I’m intrigued by the use of LAMB optimizer. What specific characteristics of your problem or empirical results led you to choose LAMB over AdamW? 4. What are the primary factors limiting the scalability of your approach to systems with stochastic motion equations and higher-order interactions?

Rating

7

Confidence

4

Soundness

3

Presentation

2

Contribution

3

Limitations

yes

Reviewer KAZi3/10 · confidence 3/52024-07-13

Summary

This paper seeks to model dynamic interactions of biological agents (primarily cellular slimes) by adapting the (continuous time) Langevin model, which models the collective motion of the system from considering individual force terms and the summation of pairwise interactions. Such pairwise interactions can be modelled as a graph; thus the others model the interaction terms with message passing on a graph and the solution to the differential equations are modelled with a neural ODE. The main contribution seems to be the learning of the force terms with fully connected neural networks. The authors claim that the main problem with previous works is that no models have been proposed that can model systems that interact in unknown ways; however, the system is trained in a supervised way based on known simulations. The results suggest the learning model can replicate the simulation, I don’t believe there was any attempt to extrapolate beyond the training regime.

Strengths

1. The paper describes a very interesting problem which (to my knowledge) hasn’t been extensively explored through learning based approaches 2. It uses state-of-the-art message passing gnns and Neural ODEs as part of the 3. The model is accurately replicating simulation behaviour 4. Code is available

Weaknesses

1. The paper is really hard to read. It took me a very long time to understand how it was implemented and although the code was generously submitted, this isn’t commented making it difficult to follow in the time available. Gnns and ODEs are in the title of the paper but are barely mentioned in the text. 2. The methodological contribution is limited 3. The model is trained in a supervised way meaning that it cannot learn new behaviours or extrapolate beyond the training regime. As this was stated as the main limitation of previous works it suggests this isn’t adding anything beyond these? 4. Results are based on validating relative to simple simulations - and run time is apparently anyway slow, so what is the value of the proposed approach

Questions

1. The main contribution seems to be to use a fully connected network to learn the force terms through a data term that forces the model to match the simulation results. Why not use a Physics informed neural network to embed the physics into the model, and thus hopefully extrapolate beyond the training regime? 2. Wouldn’t it make more sense to model this as a graph ODE? http://arxiv.org/pdf/1911.07532 3. It would help future readers if the authors explain in more detail how the whole network is trained - including where the gnn and neural ODE are called (it took me a long time to work this out as they are only briefly mentioned in the text) 4. This paper seems like it might be relevant and seems to be going beyond what is proposed here through learning models of interactions https://arxiv.org/abs/2303.09906 with PiNNs - can the authors explain the advs/disadvs of their work relative to an approach such as this

Rating

3

Confidence

3

Soundness

3

Presentation

2

Contribution

2

Limitations

Limitations are not really discussed beyond runtimes

Reviewer KAZi2024-08-09

Less confident but still have questions

I've read your response and thank you for your attempt to qualify but this has raised more questions. For sure, although I have some basic understanding of PINNs its not my specialism and I do not know well the systems you are modelling - so I have reduced my confidence in my review. What I was wondering was whether the equations with which you use to define your simulations could not be used to generate a physics based loss for your ode - where my understanding is taken from this paper (which does not model interaction terms): Linka, K. et al. Bayesian Physics-Informed Neural Networks for real-world nonlinear dynamical systems. arXiv [cs.LG] (2022). I ask this because from what I understand from your paper section 5.2 (last paragraph) the model is overfitting on the interaction forces, and my limited understanding is that adding some form of physics loss can help regularise and allow models to extrapolate beyong the training regime? As it stands the current results perform well for simulations - but for these the model is known, and a neural network is a universal approximator, so I assume it is not entirely surprising that it can learn the function given enough data? In general, I'm not clear what the longterm projection of this work is? Can the authors comment on how this would be translated to real world data for which a model doesn't exist and what the potential advantage of that would be? How would they extrapolate beyond the training regime? How would they validate that the model works? I'm still not clear why the graph neural ODE paper (Poli et al 2019) is not relevant here - I admit to not having studied this paper in depth but they show results on multi agent trajectory extrapolation and discuss a spatio-temporally evolving graph.

Authorsrebuttal2024-08-13

Answer to the questions

We appreciate the reviewer's comments and the thoughtful questions raised. Below, we provide a detailed response to address the concerns highlighted. >whether the equations in your simulations could not be used to generate a physics based loss for your ode The paper by Linka et al aimed at learning daily new COVID-19 cases (\hat{x}). Since it was evident that the data oscillates in time, the authors stabilized the prediction of \hat{x} by constraining a neural network (NN) model (which takes time t as input and outputs x) to exhibit a similar behavior to a harmonic oscillator model (where f\{x\} := \ddot{x} + (c/m)\dot{x} + (k/m)x = 0 for x). Specifically, the constraint was applied by adding the term f\{x\}^2 to the loss function. Using this approach, the authors successfully predicted future time series which is not included in the training data. In contrast, our work aims at studying a system where we can not use a term like f\{x\}^2. The very purpose of the learning process is to discover the model we used to create the training data. We used one of the models that has been proposed to explain the collective behavior of cells. However, this model does not fully account for the characteristics observed in the experimental data we aim to analyze, such as phase separation-like behavior between different cell types. Additionally, when considering other systems, both the collective behavior as a whole and the individual cell behavior differ significantly between systems, lacking any unifying pattern. So far, no universal model nor principle governing these systems have been found. Given this background, we avoided assuming a specific model in the training process. > what I understand from your paper section 5.2 (last paragraph) the model is overfitting on the interaction forces, … adding some form of physics loss can help regularise and allow models to extrapolate … The last sentence of Section 5.2 may have caused confusion. There was no overfitting but rather, it was underestimation described here. We will more clarify this point in the final version. We should note that the extrapolation performed in the Linka et al paper is indeed achieved in our study. In Section 5, This is demonstrated by our model’s ability to reproduce the dynamics of collective motion from initial states not included in the training data. >a neural network is a universal approximator, so … it is not entirely surprising While the universal approximation theorem guarantees the existence of a solution where a NN can approximate any function, this does not imply that such a solution can be found, nor that the solution will generalize well to similar data. Our case is an inverse problem where each individual in a group is influenced by forces from numerous neighbors. The challenge is to estimate individual forces from the integrated trajectories observed, which are the result of summing and integrating those forces. It is not guaranteed that such forces can always be computed, and even if the computation converges to a solution that explains the trajectory, due to the nature of inverse problems, this does not ensure that the individual forces before summation are correct. However, we have confirmed that our computations do converge, and that the individual forces are accurately estimated. >what the longterm projection of this work is? In biological physics, many including us are attempting to address various types of collective motion exhibited by a group of groups of cells. Understanding such dynamics is fundamental to knowing how tissues form. Our present approach should be applicable in extracting individual cell behavioral rules from real data. >How would they validate that the model works? This is related to the above comments. Our plan is to test this approach by applying real experimental data and from there, make corrections and improve the assumed model. >why is the graph neural ODE paper (Poli et al 2019) not relevant? The method by Poli et al is relevant to the present work. We will clarify the difference between our method and theirs in the camera-ready version. Poli et al addresses systems where the graph structure changes much slower than ours. Their extrapolation is limited to a few steps during which the edge structure remains almost unchanged. If the prediction extends beyond a few steps, the predicted node states deviate from the correct values, leading to inconsistencies with the edge structure. To perform long-term predictions with their method, one would need to compute the neural ODE over short intervals, updating the edge structure based on the resulting node states, and then repeat the neural ODE computation for another short interval. This approach is inconvenient and assumes that the edge structure remains unchanged during the neural ODE computation. In contrast, our method does not require pre-defined edges, allowing one to make long-term predictions based solely on the initial state of nodes.

Reviewer KAZi2024-08-13

Raised score

After rebuttal I have raised score to 5, and my confidence down. It would probably still be more convincing if the method was validated on experimental data. I'm still of the opinion that imposing a physics loss as regularisation doesn't mean that the model is forced to follow it exactly.

Reviewer T7k92024-08-12

Thanks for the authors clarifying my concerns on the difference of the method from previous work, and highlighting their contributions on computational costs, I have increased my score to 5.

Reviewer Shpk2024-08-13

I thank the reviewers for their detailed response. I increase my score to 7.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC