Summary
The paper proposes ‘dynamic sparse identification of nonlinear dynamics’ (dynamic SINDy), a deep learning framework for identifying governing equations in noisy, non-stationary and nonlinear dynamical systems (DS). By combining variational autoencoders (VAEs) and previous work on SINDy, it enables unsupervised inference of the underlying ODE systems’ parameters while extracting a global and parsimonious nonlinear dynamical model. The approach is validated on both synthetic and real-world data and is compared to other methods in the field, demonstrating great potential for scientific machine learning community.
Strengths
- Learning a parsimonious representation of non-autonomous DS is extremely important and relevant in many scientific disciplines, which makes the approach very promising.
- I think it is highly interesting that the encoder-decoder architecture is able to predict the ODE parameters with this level of fidelity in an unsupervised fashion (as there is no direct reconstruction loss for the ODE parameter time series involved in the loss function (7)).
- The method is tested against other baselines and also on a real-world dataset (C. elegans).
Weaknesses
- The authors should stick to the ICLR style guide and use the 'author, year' reference style instead of mere numerical numbers (i.e. APA style instead of IEEE). This increases readability and helps the reader to understand the train of thought of the authors, as one directly sees on which work the authors base certain statements.
- Center box in Fig. 1B is in parts hard to read as (font) sizes vary a lot. I think it would be better to shrink down Fig. 1A a touch and to increase size of Fig. 1B, especially as it describes the main framework of the manuscript.
- I also think the figure group titles ('suptitles') above Fig. 1, 2, and 4 are superfluous and their message should be put into the figure caption. This would create additional space (e.g. to compensate for the change in referencing style).
- I think ‘dynamic HyperSINDy’ deserves a bit more attention in the main text, which lacks explanation on how this approach really works. Explaining this method in the supplement makes the corresponding results a bit hard to read and almost forces the reader to read the supplement section 1.2.2.
- All of the employed (benchmark) datasets are fairly low dimensional (2-3D). The authors do not address the scalability of the method to high dimensional systems (which can not be sufficiently described by the first few PCA components). I think this is a major drawback, as this setting is highly relevant to many real-world systems.
Minor details:
- typo: Fig. 3B y-axis label say “approxiate std”
- l. 353 It just says 6A and 6B, while the authors probably reference Fig. 5A and 5B? Also l. 360 it says 6C instead of 5C.
- typo: supplement l. 262 it says weight decay of 1e5 (I assume 1e-5?)
Questions
- For the switch signals (Fig. 2 a-c, also Fig. 3A low noise setting), the inferred ODE parameter time series seem to exhibit high frequency oscillations on top of the correct switch-like dynamics. Is there an intuitive explanation why the encoder-decoder architecture struggles in inferring the correct switching dynamics and how this could be addressed?
- Results of Fig. 3B look rather weak to me, can the authors report Pearson’s $r$ of noise lvl vs. std?
- I’m confused by section 4.6 & Fig. 7; How exactly does the dynamic SINDy approach compare now to the proposed baseline methods based on SLDS and (vanilla?) SINDy with a group sparsity norm? I think Fig. 7 would be much clearer if the authors would find a design to compare all comparison methods side-by-side.
- ll. 409-411: Can the authors provide references for the mentioned studies?
- How do other methods like reservoir computing compare to the dynamic SINDy approach qualitatively and quantitatively in the settings discussed in the manuscript (see e.g. [1])?
- How does the approach perform on e.g. benchmarks used in [2], which exhibit different bifurcations than the ones discussed in this paper?
I am very happy to increase my score if the authors adequately address my concerns and questions.
References:
[1] Köglmayr, Daniel, and Christoph Räth. "Extrapolating tipping points and simulating non-stationary dynamics of complex systems using efficient machine learning." Scientific Reports 14.1 (2024): 507.
[2] Patel, Dhruvit, and Edward Ott. "Using machine learning to anticipate tipping points and extrapolate to post-tipping dynamics of non-stationary dynamical systems." Chaos: An Interdisciplinary Journal of Nonlinear Science 33.2 (2023).