Dutch artist Rembrandt’s 1642 masterpiece “The Night Watch” –has been digitally restored to its original size after 300 years. The painting is on display at Rijksmuseum in Amsterdam.
“The Night Watch – the most famous painting in the Netherlands and one of the most renowned paintings in the whole world and Rembrandt’s absolute masterpiece. And since the beginning and the opening of the Rijksmuseum, this became the centre-piece of this building. But not many people know that originally the Night Watch was larger than it is today,” said PieterRoelofs, Head of Paintings & Sculpture, Rijksmuseum.
The painting was trimmed to fit the wall in Amsterdam’s Townhall back in 1715. The chopped-off pieces have not been recovered till date. Now, thanks to the power of artificial intelligence, the painting is reinstated to its original glory.
Back in the day, Captain Frans Banninck Cocq commissioned artist Gerrit Lundens to make a copy of The Night Watch. Though the scale and colour schemas were different from the original, the Lunden’s painting served as a reference for restoring the classic painting.
AI magic
Robert Erdmann, Senior Scientist of the Operation Night Watch project, said, “Our attempt here is to make the best guess without the hand of an artist into what the missing parts of the Night Watch looked like.”
“So this an attempt to use the latest technology to use the Lundens to help us visualise what the missing parts of the Night Watch looked like,” he added.
“The first problem that the neural network helped us solve was to make semantic correspondences between the two. I can use a second neural network to help me make a plan, a so-called flow field that would basically show how to stretch and compress and rotate the Lundens image so that it’s perfectly aligned with the Night Watch image.
“But now I have another problem, and that is, Lundens is a different painter than Rembrandt. So, then comes a third neural network. And in this third neural network, I use this alignment between the Lundens and the Night Watch to basically send the neural network to art school. So there is a little tile from the Lundens and then the same tile from the Night Watch. And so this is basically a quiz that we’re giving a neural network. We say, all right, paint this like Rembrandt would paint it. And by performing training with neural networks, we can use hundreds of thousands of iterations of this process until the network gets better and better, so that it can take something painted in the style, scale and palette of Lundens and approximate what it might look like if Rembrandt did that,” Erdmann explained.
AI in art
AI is making huge inroads into the field of art. In October 2018, Edmond de Belamy, a generative adversarial network portrait painting from Paris-based arts-collective Obvious, was sold for a whopping $432,000 Recently, NVIDIA introduced the NVIDIA Canvas app built on real-time AI art application tool — GuaGAN, transforming rough doodles into stunning scenes using deep learning models. Users begin by drawing simple shapes and lines using a palette of real-world objects such as grass and clouds. The AI model then fills the screen with jaw-dropping results instantly.
Another AI painting project from Yamaha Motor, and IBM Japan involves a robot arm that uses water and acrylic colours to create paintings on its own.
AI in digital restoration
Researchers including Yannis Assael, Thea Sommerschield, Jonathan Prag from the University of Oxford and Google’s DeepMind published a Greek epigraphy case study titled: ‘Restoring ancient text using deep learning’. Epigraphy is the study of documents (inscriptions) written on a durable surface (stone, ceramic, metal). The study presents PYTHIA, the first ancient text restoration model that recovers missing characters from damaged text input using deep neural networks.
“To train it, we wrote a nontrivial pipeline to convert PHI, the largest digital corpus of ancient Greek inscriptions, to machine-actionable text, which we call PHI-ML. On PHI-ML, PYTHIA’s predictions achieve a 30.1% character error rate, compared to the 57.3% of human epigraphists. Moreover, in 73.5% of cases, the ground-truth sequence was among the Top-20 hypotheses of PYTHIA, which effectively demonstrates the impact of this assistive method on the field of digital epigraphy, and sets the state-of-the-art in ancient text restoration,” said researchers.
Of late, models are becoming increasingly adept at deciphering art styles. AI also has a lot of potential in the domain of art authentication.
