The deployment of deep learning applications has to address the growing\nprivacy concerns when using private and sensitive data for training. A\nconventional deep learning model is prone to privacy attacks that can recover\nthe sensitive information of individuals from either model parameters or\naccesses to the target model. Recently, differential privacy that offers\nprovable privacy guarantees has been proposed to train neural networks in a\nprivacy-preserving manner to protect training data. However, many approaches\ntend to provide the worst case privacy guarantees for model publishing,\ninevitably impairing the accuracy of the trained models. In this paper, we\npresent a novel private knowledge transfer strategy, where the private teacher\ntrained on sensitive data is not publicly accessible but teaches a student to\nbe publicly released. In particular, a three-player\n(teacher-student-discriminator) learning framework is proposed to achieve\ntrade-off between utility and privacy, where the student acquires the distilled\nknowledge from the teacher and is trained with the discriminator to generate\nsimilar outputs as the teacher. We then integrate a differential privacy\nprotection mechanism into the learning procedure, which enables a rigorous\nprivacy budget for the training. The framework eventually allows student to be\ntrained with only unlabelled public data and very few epochs, and hence\nprevents the exposure of sensitive training data, while ensuring model utility\nwith a modest privacy budget. The experiments on MNIST, SVHN and CIFAR-10\ndatasets show that our students obtain the accuracy losses w.r.t teachers of\n0.89%, 2.29%, 5.16%, respectively with the privacy bounds of (1.93, 10^-5),\n(5.02, 10^-6), (8.81, 10^-6). When compared with the existing works\n\\cite{papernot2016semi,wang2019private}, the proposed work can achieve 5-82%\naccuracy loss improvement.\n