We explore neural painters, a generative model for brushstrokes learned from\na real non-differentiable and non-deterministic painting program. We show that\nwhen training an agent to "paint" images using brushstrokes, using a\ndifferentiable neural painter leads to much faster convergence. We propose a\nmethod for encouraging this agent to follow human-like strokes when\nreconstructing digits. We also explore the use of a neural painter as a\ndifferentiable image parameterization. By directly optimizing brushstrokes to\nactivate neurons in a pre-trained convolutional network, we can directly\nvisualize ImageNet categories and generate "ideal" paintings of each class.\nFinally, we present a new concept called intrinsic style transfer. By\nminimizing only the content loss from neural style transfer, we allow the\nartistic medium, in this case, brushstrokes, to naturally dictate the resulting\nstyle.\n