NVIDIA’s EdgeRunner Can be a Game changer in 3D Mesh Generation

NVIDIA recently unveiled EdgeRunner, which can generate highly detailed 3D meshes with up to 4,000 faces at a spatial resolution of 512, derived from both images and point clouds—resulting in sequences that are twice as long and four times higher in resolution compared to previous methods.

This seems to break new ground in real time, 3D reconstruction providing far more accurate geometries, especially in challenging scenarios like edge cases or areas with partial occlusions, where traditional methods struggle.

One of the key aspects of EdgeRunner is its mesh tokenisation method, which compresses sequence length by 50% and reduces long-range dependency between tokens, significantly improving the training efficiency. 

EdgeRunner’s Auto-regressive Auto-encoder (ArAE) can compress variable-length triangular meshes into fixed-length latent codes. This latent space can be used to train latent diffusion models conditioned on other modalities, offering better generalisation capabilities. 

In contrast, other existing models for 3D mesh generation, such as Open3D or point cloud libraries (e.g., PointNet) often cut corners as they struggle with either less dense meshes or require extensive post-processing to achieve similar levels of detail. These methods are not appropriate for real-time applications.

Furthermore, research on 3D generation has explored multiple approaches like optimisation-based methods, such as using score distillation sampling, lifting 2D diffusion priors into 3D without requiring any 3D data, and so on.

However, these  approaches depend on continuous 3D representations, such as NeRF or SDF grids, which lose the discrete face indices in triangular meshes during conversion. As a result, they require post-processing, such as marching cubes and re-meshing algorithms, to extract triangular meshes. 

Additionally, such meshes differ significantly from artist-created ones, which are more concise, symmetric, and aesthetically structured. They are also limited to generating watertight meshes and cannot produce single-layered surfaces.

With such improvisation, EdgeRunner looks quite promising for autonomous systems, gaming, and VR/AR environments where high fidelity and real-time capabilities are essential.