Researchers at Meta announced the release of iSDF, a real-time mapping system for robot navigation and manipulation. iSDF trains a neural signed distance field online via continual learning and is able to fill in partially observed regions and adaptively allocate memory capacity to map at different levels of detail.
The model is self-supervised by minimising a loss that bounds the predicted signed distance using the distance to the closest sampled point in a batch of query points that are actively sampled. In contrast to prior work based on voxel grids, this neural method can provide adaptive levels of detail with plausible filling in of partially observed regions and denoising of observations, all while having a more compact representation. In evaluations against alternative methods on real and synthetic datasets of indoor environments, iSDF produced more accurate reconstructions and better approximations of collision costs and gradients useful for downstream planners in domains from navigation to manipulation.
iSDF takes as input a stream of posed depth images captured by a moving camera and, during online operation, learns a function that approximates the true signed distance field of the environment. The signed distance function is modelled by a multilayer perceptron (MLP), that maps a 3D coordinate to the signed distance value at that point. The model is initialised with random weights and is optimised in real-time with respect to incoming measurements.
