Guide to Open3D: An Open Source Modern Library For 3D Data Processing

Open3D is an open-source library designed for processing 3D data. It was introduced by Qian-Yi Zhou, Jaesik Park and Vladlen Koltun - researchers at Intel Labs (research paper). 

Its backend enables parallelization while the frontend exposes several useful algorithms and data structures of C++ and Python programming languages. With a small set of dependencies, it requires little effort for compiling Open3D code from source and setting it up on various platforms. 

Highlighting features of Open3D

• It enables 3D visualization and a physically based rendering (PBR) approach of computer graphics.
• It allows using GPU hardware accelerator for fundamental 3D operations.
• It enables the reconstruction of 3D scenes and alignment of surfaces.
• It supports PyTorch and TensorFlow machine learning frameworks.
• It provides a C++ interface and also an easier Python-based interface.
• It was originally written in standard C++11.
• It supports the following compilers to date (according to the Open3D website):

• Visual Studio 2019 and higher versions on Windows operating system
• GCC 5.X and higher on Linux
• XCode 10+ and higher on OS X 10.14+

Major modules of Open3D

• I/O
• Camera
• Geometry
• Odometry
• Integration
• Registration (local and global)
• Visualization
• Utility
• Python API and its tutorials

Practical implementation

Here’s a demonstration of plotting a point cloud and a mesh of a rabbit using Open3D. The viewpoint of the plot can be moved using a mouse. The code has been implemented using Python 3.7.10 and open3d 0.12.0 versions. Step-wise implementation of the code is as follows:

1. Install Open3D library

!pip install open3d

2. Import required libraries

 import numpy as np
 import open3d as o3d
 import plotly.graph_objects as go 

3. Download the point cloud

!wget https://raw.githubusercontent.com/PointCloudLibrary/
pcl/master/test/bunny.pcd

Read the point cloud from bunny.pcd file

rabbit_cloud = o3d.io.read_point_cloud("bunny.pcd")

Exit the program if the read file is empty

if rabbit_cloud.is_empty(): exit()

4. Compute normals of the point cloud

rabbit_cloud.estimate_normals(search_param=o3d.geometry.KDTreeSearchParamHybrid(radius=0.1, max_nn=30))

5. Download the rabbit mesh

!wget https://graphics.stanford.edu/~mdfisher/Data/Meshes/bunny.obj

Read the triangular mesh from downloaded bunny.obj file

rabbit_mesh = o3d.io.read_triangle_mesh("bunny.obj")

Exit the program if the mesh file is empty

if rabbit_mesh.is_empty(): exit()

6. Compute vertex normals and triangle normals of the mesh if they do not exist.

if not rabbit_mesh.has_vertex_normals():    
    rabbit_mesh.compute_vertex_normals()
if not rabbit_mesh.has_triangle_normals(): 
    rabbit_mesh.compute_triangle_normals() 

7. Define a function to plot the point cloud and mesh

 def draw(geometries):
 #Initialize an array for Plotly graph objects
     graph_obj = []
     for gm in geometries:
 #First, determine whether the geometry is point cloud or mesh
         geometry_type = gm.get_geometry_type()
        #If it is point cloud
         if geometry_type == o3d.geometry.Geometry.Type.PointCloud:
           #array to store point coordinates of the point cloud
             pts = np.asarray(gm.points)  
           #array to store colors of the point clous
             clr = None  #for colors
"""
If the point cloud has point colors, store RGB colors of the point cloud in clr array
"""
             if gm.has_colors():
                 clr = np.asarray(gm.colors) 
 #If the point cloud has point normals, update clr array accordingly
             elif gm.has_normals():
               clr = (0.5, 0.5, 0.5) + np.asarray(gm.normals) * 0.5
             else:
   #Paint each point with the same color first
               gm.paint_uniform_color((1.0, 0.0, 0.0))
 #Update the clr array with point cloud’s colors
               clr = np.asarray(gm.colors)
 #Scatter plot of the point cloud using Plotly
             sc = go.Scatter3d(x=pts[:,0], y=pts[:,1], z=pts[:,2],   mode='markers', marker=dict(size=1, color=clr))
  #Add the scatter plot to the graph objects array
             graph_obj.append(sc)
        #If the geometry if the mesh
         if geometry_type == o3d.geometry.Geometry.Type.TriangleMesh:
     #Store triangles’ coordinates of the mesh
             tri = np.asarray(gm.triangles)
    #Store vertices’ coordinates of the mesh
             vert = np.asarray(gm.vertices)
  #Initialize the tuple to store colors of the mesh
             clr = None
 #if the mesh has triangle normals
             if gm.has_triangle_normals():
               clr = (0.5, 0.5, 0.5) + np.asarray(gm.triangle_normals) * 0.5
               clr = tuple(map(tuple, clr))
             else:
               clr = (1.0, 0.0, 0.0)
        #Define the 3D mesh     
             mesh = go.Mesh3d(x=vert[:,0], y=vert[:,1], z=vert[:,2], 
             i=tri[:,0], j=tri[:,1], k=tri[:,2],facecolor=clr, opacity=0.50)
   #Add the mesh to graph objects array
             graph_obj.append(mesh)
 #Plot the figure using Plotly  
     fig = go.Figure(
  #use data from graph objects array i.e. point cloud and mesh
         data=graph_obj, 
   #Layout of the plot
         layout=dict(
             scene=dict(
            #Disable axes’ display
                 xaxis=dict(visible=False),
                 yaxis=dict(visible=False),
                 zaxis=dict(visible=False)
             )
         )
     )
     fig.show()     #Display the figure 

8. Draw the geometric objects 

 o3d.visualization.draw_geometries = draw #replace function
 #draw the point cloud
 o3d.visualization.draw_geometries([rabbit_cloud])
 #draw the mesh
 o3d.visualization.draw_geometries([rabbit_mesh]) 

Output: (point cloud followed by mesh)

Note: The output shown above are static images whereas, for the actual output in the colab notebook, the viewpoint of each of the two plots can be changed using a mouse.

• Google colab notebook
• Code source

References

For detailed information on the Open3D library, refer to the following sources:

• Research paper
• Official website
• Documentation
• GitHub repository