In this series of notes, we try to model neural networks as as discretizations of continuous flows on the space of data, which can be called flow model. The idea comes from an observation of their similarity in mathematical structures. This conceptual analogy has not been proven useful yet, but it seems interesting to explore. In this part, we start with a linear transport equation (with nonlinear transport velocity field) and obtain a class of residual type neural networks. If the transport velocity field has a special form, the obtained network is found similar to the original ResNet (He et al., 2016). This neural network can be regarded as a discretization of the continuous flow defined by the transport flow. In the end, a summary of the correspondence between neural networks and transport equations is presented, followed by some general discussions. 1 Transport Equation Consider the following terminal value problem (TVP) for linear transport equation: { ∂tu + v(t, x) · ∇u = 0, x ∈ R, t ∈ [0, T ] u(T, x) = f(x), x ∈ R. (1) Here v is a Rd-valued function, called the transport velocity field. It can be chosen in different ways. We will consider firstly the general form, then a special type: v(t, x) = W (t)a ( W (t) + b(t) ) + b(t), (2) where W (1)(t),W (2)(t) ∈ Rd×d, b(1)(t), b(2)(t) ∈ Rd. The activation a is a Rd-valued nonlinear function, which is Lipschitz continuous. ∗Department of Mathematics, HKUST. Email: lishen03@gmail.com †Yau Mathematical Sciences Center, Tsinghua University. Email: zqshi@tsinghua.edu.cn 1 ar X iv :1 70 8. 06 25 7v 1 [ cs .L G ] 2 1 A ug 2 01 7 It is well known that the solution of equation (1) is transported along characteristics. The characteristics are defined as solutions of the initial value problems (IVP) of the ODE: { ẋ = v(t, x), t ∈ [0, T ] x(0) = x0, (3) where x0 ∈ Rd. Along the solution curve x = q(t), it is easy to verify that d dt u(t, q(t)) = (∂tu(t, x) + q̇(t) · ∇u(t, x))x=q(t) (4) = (∂tu(t, x) + v(t, q(t)) · ∇u(t, x))x=q(t) = 0. (5) So u remains unchanged along the curve. See Figure 1 for a conceptual illustration. Therefore u(0, x0) = u(t, q(t)) = u(T, q(T )) = f(q(T )). (6) Figure 1: Illustration of characteristics. Here x, u(t, x) ∈ R. 2 Method of Characteristics In this part we will use the method of characteristics to solve (1). In order to make the following approximations reasonable, we assume that the change of v(t, x) with t and x is regular enough. Especially, we assume that the solution of (1) and (3) exist for the posed conditions, and they are regular enough. Let {tk}k=0 with t0 = 0 and tL = T be a partition of [0, T ] ⊂ R such that for any k = 1, . . . , L, sk = tk − tk−1 is small enough. Let x = q(t) be a characteristic of the transport equation (1), i.e. the solution of (3), and denote xk = q(tk). Denote Vk(x) = v(tk, x) and uk(x) = u(tk, x) for any x ∈ R. See Figure 2 for a illustration of the discretization.