Solving inverse problems with iterative algorithms is popular, especially for\nlarge data. Due to time constraints, the number of possible iterations is\nusually limited, potentially affecting the achievable accuracy. Given an error\none is willing to tolerate, an important question is whether it is possible to\nmodify the original iterations to obtain faster convergence to a minimizer\nachieving the allowed error without increasing the computational cost of each\niteration considerably. Relying on recent recovery techniques developed for\nsettings in which the desired signal belongs to some low-dimensional set, we\nshow that using a coarse estimate of this set may lead to faster convergence at\nthe cost of an additional reconstruction error related to the accuracy of the\nset approximation. Our theory ties to recent advances in sparse recovery,\ncompressed sensing, and deep learning. Particularly, it may provide a possible\nexplanation to the successful approximation of the l1-minimization solution by\nneural networks with layers representing iterations, as practiced in the\nlearned iterative shrinkage-thresholding algorithm (LISTA).\n