Art authenticators: AI could be your new best friend

In one case, a museum thought they had a Raphael, as one AI art image analysis system estimated that the painting was indeed by the High Renaissance master, with an 83% probability – only for another AI system to announce that no, alas, it was not by Raphael, with a 95% certainty. In another, the mystery of how much (if any) of The Polish Rider, a mysterious painting in the Frick Collection in New York, was painted by Rembrandt has, if not been solved, at least confirmed. AI analysis matched what art historians and conservators thought was the case: most of the painting is by Rembrandt and his studio, but the horse’s legs and the ground the horse stands on is not. And on 23rd November, three items, including a Louise Bourgeois drawing, are going up for auction, their authorship bolstered by AI analysis.

Artificial Intelligence is the bold new frontier in many fields, with ChatGPT and its late-to-the-starting-line competitors now an integral part of our daily lives. After ChatGPT’s success, it seems that every major software system, from Google to Microsoft to Adobe and beyond, has integrated AI as a new feature.

Art is a traditional field, particularly the trade and museum wing of the arts. New technologies are met with suspicion and fear, with human experts nervous that their positions will be entirely usurped by computers. This concern is not unique to the art world, of course, but it is particularly keen there. From the creative side, the image-making abilities of the likes of Dall-e and Midjourney have made conceiving of what a digital image might look like (conceptualising an as-yet unmade artwork is a term that Renaissance artist and historian Giorgio Vasari called invenzione) a game – the software then creates a unique image that it thinks will match your prompt, fulfilling your invenzione with disegno. Disegno means “drawing” or “design” in Italian but Vasari used it to describe the physical ability, the long-trained skill, to manifest your artistic vision and make it a reality, in drawing, sculpture, painting and other media. Most of us can imagine a cool image but can’t produce it. Now AI software does it for us.

How AI threatens art experts is a more subtle issue. Of course it doesn’t really, but such is the concern. Could image analysis be sophisticated enough that a computer examines a digital image of an artwork and determines who made it?

Authorship has always been a sticky wicket for art experts. Traditionally three methods are used to determine authenticity, and perhaps its cousin, authorship. In my book, The Art of Forgery, I provide an equation that still seems relevant: (Perceived) Value = Perceived (Demand + Authenticity + Rarity)

For our purposes, authenticity is the focus, since most artworks worth discussing are unique (therefore rare) and demand is a matter of the market rather than the work itself. The value of art is entirely about perception. If you, the buyer, think that an artwork is in demand, authentic and rare then its value will be as if it were (even if, in reality, it’s not).

For millennia, art’s authenticity was a matter left to connoisseurship, the opinion of so-called experts (though art has never had an objective method of proving one’s expertise). Too often, however, experts couldn’t agree, most famously in the 1934 Wacker van Gogh case tried in Amsterdam in which Otto Wacker and his family were found to have forged van Gogh paintings. The issue was that the two leading van Gogh specialists, H.P. Bremmer and Bart de la Faille, couldn’t agree on which of the van Goghs in question were authentic – de la Faille changed his own mind several times about them. Had the two experts agreed, that would have been case closed. But in a historical first, the judge called in a tiebreaker, a chemist called Martin de Wild, who analysed the chemical composition of pigment samples. The pigments in the Wacker van Goghs contained pulverised lead and resin, which would have sped up the drying of oil paints to make them look older faster, which were not present in pigments from established van Gogh paintings. Forensics had solved what connoisseurship could not.

The next addition to the authentication equation came after the Second World War. Concern over the inappropriate and illegal movement of cultural heritage objects made provenance research a key feature of the art market – one shouldn’t sell any artwork without solid provenance, the documented history of an object, during the war period, for fear that it was appropriated or stolen. Today art is authenticated initially through connoisseurly visual analysis, then provenance research, followed by forensic testing for artworks that seem problematic or for which flags were raised by the expert opinion or provenance.

Authenticity and authorship often go together, but not always. It’s easier, with the help of the technological tool kit of conservation labs, to determine the age of an object, than to say with certainty who made it. The who-made-it question is particularly tricky when we understand the studio system through which artists worked for most of known history: the master led the studio and anything produced there was “his” (unfortunately it was almost always a “he”), but in practice the master would not necessarily work hands-on on every work, instead relying on paid assistants and unpaid apprentices to collaborate. Thus, The Polish Rider question is tricky because it’s not “this is either a Rembrandt or a copy by someone who Rembrandt never knew” but it could indeed be “by Rembrandt” but have emerged from his studio, with much or even most of it actually painted by assistants.

Connoisseurs have been nervous about new technologies each time one has been introduced. From X-rays to infrared spectroscopy, technology has never replaced human experts, but merely offered them an additional tool. It is also convincing to the general public, who tend to feel reassured by lab results which, in principle, seem more objectively true than the opinion of experts who merely cast their eyes over an artwork.

AI art analysis is different in that, at first glance, it appears to let a computer program do what connoisseurs once did. Carina Popovici, a physicist who shifted careers and founded Art Recognition, a leading AI art analysis firm based in Zurich, explains the process. “We use a proprietary AI, based upon a deep artificial neural network, to carry out art authentication analyses. We train the AI on photographs of all known works of art by a chosen artist, as well as negative examples such as images of known fakes, art created by followers, pupils, schools, circle, etc. and even digital art produced by a generative AI in the style of that artist. From all these images, the AI learns the main characteristics of the artist, and also distinguishes genuine art from non-authentic examples. The most important feature is the brushstroke, but the AI also learns other features, i.e. edge locations to distinguish object structures from one another, chromatics, high-level elements of composition, etc. Once the training has been completed, the program compares the learned characteristics with those identified on the image of an artwork whose authenticity needs to be established. Based on this comparison, the AI returns a probability for the authenticity of the new art piece.”

This is a key point and one that can reassure nervous connoisseurs. Technology is not yet at the point where you can point your phone at a painting, click a button and have its author’s name in seconds. AI software must be “trained” by humans providing input. If it is left to comb the internet, it will come up with so much incorrect information as to be rendered useless. So closed, proprietary systems that are taught what to look for (and what not to be fooled by) by humans are key. We’re a long way off from an AI being able to accurately name the artist behind a work without the artist being suspected ahead of time, and the AI fed a huge amount of accurate information about the artist in question. So if you show Art Recognition The Polish Rider as an open question it can’t help you. Teach it about Rembrandt and ask, “Is this a Rembrandt?” and you can get incredible results. As revealed in The Art Newspaper, the results showed that for most of the painting, there’s an 83% probability that Rembrandt painted it. For other parts it drops to 69%, while the section long suspected not to be by Rembrandt at all, the horse’s legs and land on which they stand, are not by Rembrandt (they are likely part of one of the two restorations, undertaken in 1833 and 1877).

This is an example of how AI can add a powerful and compelling weapon to the human art expert’s arsenal, without replacing experts. The AI requires human intervention to teach it what to look for and then human interpretation to explain the results. At a glance, the headline “83% probability that The Polish Rider is by Rembrandt” sounds convincing to the general public. But in practice it is just one more piece of evidence to weigh alongside decades of art historical debate (connoisseurship), document research (provenance), and pigment analysis (forensics).

AI bridges the categories of connoisseurship and forensics. It is a technological tool, which sounds like forensics. But it’s non-invasive, working only based on high-quality digital photographs of the object in question. It is effectively a computer program taught to be a connoisseur. Recall the film The Matrix, in which people could have information and skills uploaded directly to their brains and nervous systems via discs (“I know kung fu”). This is what humans can do with AI. Upload everything a connoisseur would need to know about Rembrandt and the AI suddenly “knows Rembrandt” to the point where it can objectively estimate the likelihood of authorship.

That objectivity is important, because human experts are often swayed by subjective matters. Maybe the expert gets a commission on a sale, so subconsciously would really quite like the object in question to be by a famous master. Maybe the expert feels peer pressure to weigh in one way or another. Maybe the expert is having a bad day and isn’t fully present emotionally. The cold, emotionless objectivity of software guards against the dangers of personal opinion.

Humans also need to know how to teach the software so as not to get false positives. Art Recognition made headlines and distinguished itself in what The Guardian called “Battle of the AIs.” Another AI approach, from the University of Bradford, did a study of the de Brécy Tondo and determined it was by Raphael with a 97% “similarity between the madonnas” in this and confirmed Raphael paintings. When I read that headline, as an art historian, I immediately knew it sounded fishy. Raphael is famous for having chosen an ideal face for his holy figures and reusing the same face throughout his oeuvre. His followers replicated the same face, this ideal of beauty. So of course every holy face by Raphael and his followers looks pretty much the same. The basic question asked by the University of Bradford: ’Do these faces match?’, was a bad one. Their AI software also used facial recognition as the main factor, without taking into account things like brushstroke, backgrounds, hands, and without feeding the software info about what not to look for. Art Recognition countered with their own study, which found that there was an 85% certainty that the de Brécy Tondo was not by Raphael.

Firms like Art Recognition are blazing a trail for how best to harness AI as a tool for human art experts. In the end, the human experts come out the winners. This technology, like those that have come before it, is nothing to fear, but provides another way forward, and one compelling for its cold objectivity.