Although significant progress has been made in the field of automatic image\ncaptioning, it is still a challenging task. Previous works normally pay much\nattention to improving the quality of the generated captions but ignore the\ndiversity of captions. In this paper, we combine determinantal point process\n(DPP) and reinforcement learning (RL) and propose a novel reinforcing DPP\n(R-DPP) approach to generate a set of captions with high quality and diversity\nfor an image. We show that R-DPP performs better on accuracy and diversity than\nusing noise as a control signal (GANs, VAEs). Moreover, R-DPP is able to\npreserve the modes of the learned distribution. Hence, beam search algorithm\ncan be applied to generate a single accurate caption, which performs better\nthan other RL-based models.\n