Artificial Intelligence and Misinformation in Art: Can Vision Language Models Judge the Hand or the Machine Behind the Canvas?

The attribution of artworks in general and of paintings in particular has always been an issue in art. The advent of powerful AI models that can generate and analyze images creates new challenges for painting attribution. On the one hand, AI models can create images that mimic the style of a painter, which can be incorrectly identified as made by that painter, for example, by other AI models. On the other hand, AI models may not be able to correctly identify the artist for real paintings, inducing users to attribute paintings incorrectly. In this article, both problems are experimentally studied at scale using state-of-the-art AI models for image generation and analysis on a large dataset with close to 40,000 paintings from 128 artists. This dataset has been extended to include AI-generated captions for the paintings, as well as AI-generated images derived from these captions. The enhanced dataset is publicly available for download and interactive visualization. The results show that vision language models have limited capabilities to: (1) identify the authors of real canvas and (2) to identify AI-generated images. As users increasingly rely on queries to AI models to get information, these results show the need to improve the capabilities of VLMs to reliably perform artist identification and detection of AI-generated images to prevent the spread of incorrect information.