We present a novel method for detecting outliers in astronomical time series based on the combination of a deep neural network and a k-nearest neighbor algorithm with the aim of identifying and removing problematic epochs in the light curves of astronomical objects. We used an EfficientNet network pretrained on ImageNet as a feature extractor and performed a k-nearest neighbor search in the resulting feature space to measure the distance from the first neighbor for each image. If the distance was above the one obtained for a stacked image, we flagged the image as a potential outlier. We applied our method to a time series obtained from the VLT Survey Telescope monitoring campaign of the Deep Drilling Fields of the Vera C. Rubin Legacy Survey of Space and Time