Content Masked Loss: Human-Like Brush Stroke Planning in a Reinforcement Learning Painting Agent

The objective of most Reinforcement Learning painting agents is to minimize\nthe loss between a target image and the paint canvas. Human painter artistry\nemphasizes important features of the target image rather than simply\nreproducing it (DiPaola 2007). Using adversarial or L2 losses in the RL\npainting models, although its final output is generally a work of finesse,\nproduces a stroke sequence that is vastly different from that which a human\nwould produce since the model does not have knowledge about the abstract\nfeatures in the target image. In order to increase the human-like planning of\nthe model without the use of expensive human data, we introduce a new loss\nfunction for use with the model's reward function: Content Masked Loss. In the\ncontext of robot painting, Content Masked Loss employs an object detection\nmodel to extract features which are used to assign higher weight to regions of\nthe canvas that a human would find important for recognizing content. The\nresults, based on 332 human evaluators, show that the digital paintings\nproduced by our Content Masked model show detectable subject matter earlier in\nthe stroke sequence than existing methods without compromising on the quality\nof the final painting. Our code is available at\nhttps://github.com/pschaldenbrand/ContentMaskedLoss.\n