Revised Section 3 (Part 1)
Dear Reviewer,
To demonstrate the more specific effects after adjustment, please allow us to provide the complete Theoretical Results Section for your reference. Due to length constraints, we have divided it into two parts, each presented in separate comment boxes.
(Part 1):
## Theoretical Results: Convergence Analysis
Consider the CSODEs given in Equation (1), where $f_j: \mathbb{R}^{k_j} \to \mathbb{R}^{k_j}$ are nonlinear activation functions. For them, an imposed condition on $f_j^i$ (the $i$-th element of the vector-valued $f_j$) is presented as follows:
**Assumption 1:** For any $i \in \{1,\dots, k_j\}$ and $j \in \{1,\dots,M\}$, $s f^i_j(s) >0$ for all $s \in \mathbb{R} \backslash \{ 0 \}$.
**Remark 1:** Assumption 1 applies to many activation functions, such as $\tanh$ and parametric ReLU. It picks up the activation functions passing through the origin and the quadrants I and III. For more explanations, the reader is referred to Appendix B.1.
In this study, to analyze the convergence property of the NN (1), we first define the concept of *convergence*:
**Definition 1:** The model (1) is convergent if it admits a unique bounded solution for $t \in \mathbb{R}$ that is globally asymptotically stable (GAS).
In order to investigate the convergence, two properties have to be satisfied, that is, the boundedness and the GAS guarantees of the solution $x^{*}$ for Equation (1). In this respect, two assumptions are given as follows.
**Assumption 2:** Assume that the functions $f_j^i$ are continuous and strictly increasing for any $i \in \{1,\dots, k_j\}$ and $j \in \{1,\dots,M\}$.
Assumption 2 aligns with CSODE's structure, reflecting continuity and monotonicity of activation functions. This relates to model dynamics and is satisfied by most common activations.
In the analysis of convergence, one needs to study two models in the same form but with different initial conditions and their contracting properties. To that end, along with Equation (1), we consider the model
$\dot{y}(t)=A_{0}y(t)+\sum_{j=1}^{M}A_{j}f_{j}(W_jy(t))+g(u(t))$ with the same input but different initial conditions $y(0)\in\mathbb{R}^{n}$.
Let $\xi:=y-x$. Then the corresponding error system is
$$\dot{\xi}=A_{0}\xi+\sum_{j=1}^{M}A_{j}p_{j}(x,\xi),$$
where $p_{j}(x,\xi) =f_{j}(W_j(\xi+x))-f_{j}(W_jx)$. Note that for any fixed $x\in\mathbb{R}^{n}$, the functions $p_j$ in the variable
$\xi \in\mathbb{R}^{n}$ satisfy the properties formulated in Assumptions 1, 2. The following assumption is imposed for analyzing the contracting property of Equation (2).
**Assumption 3:** Assume that there exist positive semidefinite diagonal matrices $S_{0}^{j},S_{1}^{j},S_{2}^{j},S_{3}^{j,r},H_{0}^{j},H_{1}^{j},H_{2}^{j},H_{3}^{j,r} \left(j,r\in \{1,\dots,M\}\right)$ with appropriate dimensions such that
$$\begin{aligned}
p_{j}(x,\xi)^{\top} p_{j}(x,\xi) & \leq \xi^{\top} W_j^\top S_{0}^{j} W_j \xi+2\xi^{\top} W_j^\top S_{1}^{j}p_{j}(x,\xi) +2\xi^{\top} W_j^\top S_{2}^{j}f_{j}(W_j\xi) \\
& +2\sum_{r=1}^{M}p_{j}(x,\xi)^{\top} W_j^\top W_j S_{3}^{j,r} W_r^\top W_r f_{r}(W_r \xi)
\end{aligned}$$
and
$$\begin{aligned}
f_{j}(W_j \xi)^{\top}f_{j}(W_j \xi) & \leq \xi^{\top} W_j^\top H_{0}^{j} W_j\xi+2\xi^{\top} W_j^\top H_{1}^{j}p_{j}(x,\xi)+2\xi^{\top} W_j^\top H_{2}^{j}f_{j}(W_j \xi) \\
& + 2\sum_{r=1}^{M}p_{j}(x,\xi)^{\top} W_j^\top W_j H_{3}^{j,r} W_r^\top W_r f_{r}(W_r \xi)
\end{aligned}$$
for all $x,y\in\mathbb{R}^{n}$ and $\xi=x-y$.
Notice that Assumption 3 is at least more relaxed than Lipschitz continuity (see Appendix B.2 for an intuitive example of activation functions satisfying Assumption 3).
...