Author Response to Reviewer vbhs
We thank the reviewer for appreciating our research and acknowledging its significance. The questions raised have been addressed in the responses provided below.
----
**1. Motivation for specific implementation.**
* To address the reviewer’s concern, we have included a brief note on the key concepts and related works in Appendix A. We hope that this section helps improve the reviewer’s understanding of our approach. If there are still specific aspects or details the reviewer’s find unclear, please let us know, and we would be happy to address them further in the revised version.
----
**2. Cost functional for SOC problem.**
* The cost function in Eq (7) is central to our approach as it serves to balance two objectives: minimizing control effort and aligning the generated dynamics with observed data. The intuition behind this choice comes from SOC theory, where the goal is to control a system's trajectory in such a way that it closely follows a desired path (in our case, the latent dynamics inferred from observations) while minimizing control energy.
* The first term, $\frac{1}{2}||\alpha_t||^2$, represents the control energy, which penalizes large deviations in control inputs for **the principle of least action** to regularize the control effort, thereby encouraging smoother trajectories. In practice, we can interpret it as a regularization term to help generalization by discouraging overly complex control signals and ensuring that the model relies on inherent patterns in the data rather than excessively adjusting controls to fit noise.
* The second term, $-\log f_i(\mathbf{y}_{t_i} | \cdot)$, ensures that the controlled dynamics are consistent with the observed data points over the time interval [0, T] by adjusting the dynamics to maximize the likelihood of observations.
* This interpretation is closely related to variational inference, where the objective is to approximate a target posterior distribution by optimizing a variational distribution while including a regularization term (often a Kullback-Leibler divergence). In our formulation, we extend this approach to the path space, where the prior distribution is given by the dynamics in Eq (1), and the target posterior distribution is defined in Eq (2). Here, the control function \alpha plays the role of adjusting the prior dynamics to approximate the posterior path measure, in SOC problem Eq (7) (technically Eq (16)) often referred to as KL-control problem, where the first term acts as KL-regularization term while the second term acts as target log-likelihood. The cost function in Eq (16) acts as a variational bound to achieve this alignment while maintaining regularization over the entire trajectory.
----
**3. Assimilation scheme.**
* The use of full assimilation means that the controls for the controlled SDE indeed incorporate information over a full observation $\mathbb{o}_{t \in [0, T]}$. The type of assimilation is inspired by filtering/smoothing in the standard SSM algorithm, where the smoothing algorithm typically incorporates full observation. This design choice was made to allow for non-causal data assimilation, which improves the expressiveness and flexibility of our model in capturing long-range dependencies. We provided a conceptual illustration of the proposed information assimilation (full/history) scheme in Figure 4.
----
**4. Use of history attention scheme.**
* We would like to clarify that, in our implementation, we utilize a history assimilation scheme (which we believe the reviewer refers to as "masked attention") for the sequence extrapolation task, because the task involves forecasting based solely on past data. It prevents the use of the full assimilation scheme, which leverages future observations. With history assimilation, our approach effectively behaves like a traditional filtering method which typically relies on past observation to infer the latent states. As a result, in sequential extrapolation, the performance gap between our model and CRU narrows. Because CRU is genuinely a filtering method that typically relies on past observation and with history assimilation, our approach effectively behaves like a filtering method. In this setting, it is worth noting that our method achieves comparable performance with CRU, while avoiding numerical trajectory inference by leveraging parallel state estimation based on Theorem 3.8.