The $δN$ formalism is a powerful approach to compute non-linearly the large-scale evolution of the comoving curvature perturbation $ζ$. It assumes a set of FLRW patches that evolve independently, but in doing so, all the gradient terms are discarded, which are not negligibly small in models beyond slow-roll. In this Letter, we extend the formalism to capture these gradient corrections by encoding them in a homogeneous-spatial-curvature contribution assigned to each FLRW patch. For a concrete example, we apply this formalism to the ultra-slow-roll inflation, and find that it can correctly describe the large-scale evolution of the comoving curvature perturbation from the horizon exit. We also briefly discuss non-Gaussianities in this context.