Google AI Digitally Reconstructs Gustav Klimt’s Lost Paintings

Google Arts & Culture recently launched a new interactive hub called "Klimt vs Klimt – The Man of Contradictions," which will offer the visitors an insight into Gustav Klimt's biography, artistic inspiration, legacy, and more, along with scholarly articles penned by experts at Austria's top institutions, including the Belvedere and the Wien Museum.

As part of its initiative, Google also digitally recreated three of Klimt's lost works from 1899, using artificial intelligence technology to colorise black-and-white photography of the works.

The three paintings—Medicine, Philosophy, and Jurisprudence—together form the "Faculty Paintings," commissioned by the University of Vienna. Upon the works' unveiling, university officials deemed them "pornographic" and "perverse." They were ultimately sold to a private buyer. 

Gustav Klimt's "faculty paintings" were stolen by the Nazis and destroyed by fire in the last days of World War II. Klimt's "Faculty Paintings" were representations of the faculties of philosophy, medicine and jurisprudence that the Austrian Ministry of Education had commissioned for the University of Vienna.

The machine learning experiment recoloured photographs of lost Klimt paintings, while a "Pocket Gallery" brings some of his most iconic works into one's living room in augmented reality and 3D.

The cutting edge technology was trained on a data-set of Klimt's paintings and allowed the 'Faculty Paintings' to be restored to their former glory. The huge data set consisted of 1 million pictures of the real world and 91,749 images of artworks by various artists. The created algorithm ascertained a sense of skin tones and the hues of the sky from the real world images and a sense of composition, object boundaries, and textures from the paintings. 

Then, added to that initial data set were 80 full-colour reproductions of Klimt paintings, meant to key the algorithm into the Vienna Secession artist's style. The resulting AI-colored images provide what might be the closest we will ever get to seeing a complete image of those lost paintings. 

Google Arts & Culture collaborated with 30 partners and institutions to present hundreds of paintings, drawings, letters and more, many of which have been digitised for the first time.

Dr Franz Smola, a curator at the Belvedere, worked with Emil Wallner, a Google Arts & Culture Lab resident, to develop the algorithm that could accurately recolour the "Faculty Paintings."