We revisit the long-standing problem of learning from a small sample, to\nwhich end we propose a novel method called GLICO (Generative Latent Implicit\nConditional Optimization). GLICO learns a mapping from the training examples to\na latent space and a generator that generates images from vectors in the latent\nspace. Unlike most recent works, which rely on access to large amounts of\nunlabeled data, GLICO does not require access to any additional data other than\nthe small set of labeled points. In fact, GLICO learns to synthesize completely\nnew samples for every class using as little as 5 or 10 examples per class, with\nas few as 10 such classes without imposing any prior. GLICO is then used to\naugment the small training set while training a classifier on the small sample.\nTo this end, our proposed method samples the learned latent space using\nspherical interpolation, and generates new examples using the trained\ngenerator. Empirical results show that the new sampled set is diverse enough,\nleading to improvement in image classification in comparison with the state of\nthe art, when trained on small samples obtained from CIFAR-10, CIFAR-100, and\nCUB-200.\n