We study the problem of sequentially recovering a sparse vector x<sub>t</sub> and a vector from a low-dimensional subspace ℓ<sub>t</sub> from knowledge of their sum m<sub>t</sub> = x<sub>t</sub> + ℓ<sub>t</sub>. If the primary goal is to recover the low-dimensional subspace where the ℓ<sub>t</sub>'s lie, then the problem is one of online or recursive robust principal components analysis (PCA). To the best of our knowledge, this is the first correctness result for this problem. We prove that if a good estimate of the initial subspace is available; the ℓ<sub>t</sub>'s obey certain denseness and slow subspace change assumptions; and the support of x<sub>t</sub> changes either at every frame or at least every so often, then with high probability, the support of x<sub>t</sub> will be recovered exactly, and the error made in estimating x<sub>t</sub> and ℓ<sub>t</sub> will be small. An example where this problem occurs is in separating a sparse foreground and a slowly changing dense background from surveillance videos.