Recently, Google Research has open-sourced a platform that can reconstruct historical cities in 3D visualisation, known as Kartta Labs. The platform is a modular and scalable designed scalable system for virtually reconstructing cities from historical maps and photos.

The researchers introduced Kartta Labs as a collaborative time travel and explained that the ultimate goal of this research is to create a time travel experience that helps users to travel back in time virtually. They further compared it with Google StreetView and mentioned that the platform is almost similar to the StreetView with an additional feature of travelling far back in time. 

Behind Kartta Labs

According to the researchers, Kartta Labs relies on crowdsourcing and artificial intelligence and consists of two major modules, which are maps and 3D models. Further, each module consists of sub-modules that enable the system to reconstruct a city from historical maps and photos. 

Maps: The Maps module creates a map server along with a time dimension. This module is built up of a suite of tools that permits the users to upload historical maps, georectify them to match real-world coordinates as well as convert them to vector format by tracing their geographic features. The sub-modules of maps include georectification, vectorisation, tiling and visualisation.

3D Models: On the other hand, the 3D Models module aims to reconstruct the detailed full 3D structures of historic buildings using the associated images and maps data. The historical images and vector historical maps are used as inputs, which presents an output with a 3D representation of an area across time. The sub-modules of this module include photo annotation, facade parsing and rectification, 3D reconstruction, 3D model repository, 3D rendering and data.

They mentioned various key steps that were taken to build this project. They are-

• Firstly, to build a modular and scalable system to collect, process, and serve map data indexed by time and space. The Maps project consists of a stack of web applications that crowdsources collecting and vectorising historical maps.
• Secondly, to reconstruct the historical buildings as 3D models. To do so, the researchers introduced an image processing pipeline in which the first step was an image segmentation job to identify buildings facades. The improved pipeline extracts 3D features of the facades and incorporates accurate footprints from historical maps.
• Finally, the result is stored as a 3D mesh model in an online repository, accessible through a public API.

How It Works

The researchers followed the principles of systems design to lay out the architecture of Kartta Labs and build a modular system. The modules and their sub-modules are primarily defined based on their input and output. According to the researchers, the output of one sub-module becomes the input to another, creating an organic workflow. 

Kartta Labs is implemented in several different languages using different technologies and development frameworks. The deployments of all the applications are unified using Docker containers and Kubernetes. In addition to its Kubernetes engine, the managed databases and storage of Google Cloud Platform (GCP) is used to leverage the scalability, security, and reliability of the system. They also used the Google Clouds Functions, which is a serverless execution environment for running simple, single-purpose cloud services into some simple services.

Applications of Kartta Labs

Kartta Labs has the ability to provide accurate historical geospatial data over time as a reference. According to the researchers, Kartta Labs is the only system that is capable of not only vectorising historical maps, but also reconstructing them in 3D across time. There are several other use-cases and applications of this system, some of them are mentioned below-

• The Kartta Labs system provides a valuable platform and resource for research and education. It can be used to facilitate numerous educational and research applications and use cases, such as topics in sociology, urbanisation, biodiversity, etc.
• Kartta Labs can be used for change detection in various application domains from urban planning to transportation and public health policymaking.
• Entertainment can be a major use case of Kartta Labs as this platform can be used for location-based games such as Ingress, as it can extend their maps in the time-dimension, etc.
• Kartta Labs can be used by movie industries to recreate accurate and photorealistic historical scenes.

Wrapping Up

According to the researchers, this system empowers the users to experience collaborative time travel such that they work together to reconstruct the past and experience it on an open-source and open data platform. The platform also differentiates itself from the prior work by combining the features of several similar applications and providing them as an open-source and open data platform.

They used Kubernetes to deploy and manage the tools as well as to make it easier for others to redeploy the suite of applications either for development or production purposes. The system is being run on Google Cloud platform and is being planned to launch a full version by the end of this year. 

Read the paper here.