Open3D is an open-source library designed for processing 3D data. It was introduced by Qian-Yi Zhou, Jaesik Park and Vladlen Koltun – researchers at Intel Labs (research paper).
Its backend enables parallelization while the frontend exposes several useful algorithms and data structures of C++ and Python programming languages. With a small set of dependencies, it requires little effort for compiling Open3D code from source and setting it up on various platforms.
Highlighting features of Open3D
- It enables 3D visualization and a physically based rendering (PBR) approach of computer graphics.
- It allows using GPU hardware accelerator for fundamental 3D operations.
- It enables the reconstruction of 3D scenes and alignment of surfaces.
- It supports PyTorch and TensorFlow machine learning frameworks.
- It provides a C++ interface and also an easier Python-based interface.
- It was originally written in standard C++11.
- It supports the following compilers to date (according to the Open3D website):
- Visual Studio 2019 and higher versions on Windows operating system
- GCC 5.X and higher on Linux
- XCode 10+ and higher on OS X 10.14+
Major modules of Open3D
- I/O
- Camera
- Geometry
- Odometry
- Integration
- Registration (local and global)
- Visualization
- Utility
- Python API and its tutorials
Practical implementation
Here’s a demonstration of plotting a point cloud and a mesh of a rabbit using Open3D. The viewpoint of the plot can be moved using a mouse. The code has been implemented using Python 3.7.10 and open3d 0.12.0 versions. Step-wise implementation of the code is as follows:
- Install Open3D library
!pip install open3d
- Import required libraries
import numpy as np import open3d as o3d import plotly.graph_objects as go
- Download the point cloud
!wget https://raw.githubusercontent.com/PointCloudLibrary/ pcl/master/test/bunny.pcd
Read the point cloud from bunny.pcd file
rabbit_cloud = o3d.io.read_point_cloud("bunny.pcd")
Exit the program if the read file is empty
if rabbit_cloud.is_empty(): exit()
- Compute normals of the point cloud
rabbit_cloud.estimate_normals(search_param=o3d.geometry.KDTreeSearchParamHybrid(radius=0.1, max_nn=30))
- Download the rabbit mesh
!wget https://graphics.stanford.edu/~mdfisher/Data/Meshes/bunny.obj
Read the triangular mesh from downloaded bunny.obj file
rabbit_mesh = o3d.io.read_triangle_mesh("bunny.obj")
Exit the program if the mesh file is empty
if rabbit_mesh.is_empty(): exit()
- Compute vertex normals and triangle normals of the mesh if they do not exist.
if not rabbit_mesh.has_vertex_normals(): rabbit_mesh.compute_vertex_normals() if not rabbit_mesh.has_triangle_normals(): rabbit_mesh.compute_triangle_normals()
- Define a function to plot the point cloud and mesh
def draw(geometries): #Initialize an array for Plotly graph objects graph_obj = [] for gm in geometries: #First, determine whether the geometry is point cloud or mesh geometry_type = gm.get_geometry_type() #If it is point cloud if geometry_type == o3d.geometry.Geometry.Type.PointCloud: #array to store point coordinates of the point cloud pts = np.asarray(gm.points) #array to store colors of the point clous clr = None #for colors """ If the point cloud has point colors, store RGB colors of the point cloud in clr array """ if gm.has_colors(): clr = np.asarray(gm.colors) #If the point cloud has point normals, update clr array accordingly elif gm.has_normals(): clr = (0.5, 0.5, 0.5) + np.asarray(gm.normals) * 0.5 else: #Paint each point with the same color first gm.paint_uniform_color((1.0, 0.0, 0.0)) #Update the clr array with point cloud’s colors clr = np.asarray(gm.colors) #Scatter plot of the point cloud using Plotly sc = go.Scatter3d(x=pts[:,0], y=pts[:,1], z=pts[:,2], mode='markers', marker=dict(size=1, color=clr)) #Add the scatter plot to the graph objects array graph_obj.append(sc) #If the geometry if the mesh if geometry_type == o3d.geometry.Geometry.Type.TriangleMesh: #Store triangles’ coordinates of the mesh tri = np.asarray(gm.triangles) #Store vertices’ coordinates of the mesh vert = np.asarray(gm.vertices) #Initialize the tuple to store colors of the mesh clr = None #if the mesh has triangle normals if gm.has_triangle_normals(): clr = (0.5, 0.5, 0.5) + np.asarray(gm.triangle_normals) * 0.5 clr = tuple(map(tuple, clr)) else: clr = (1.0, 0.0, 0.0) #Define the 3D mesh mesh = go.Mesh3d(x=vert[:,0], y=vert[:,1], z=vert[:,2], i=tri[:,0], j=tri[:,1], k=tri[:,2],facecolor=clr, opacity=0.50) #Add the mesh to graph objects array graph_obj.append(mesh) #Plot the figure using Plotly fig = go.Figure( #use data from graph objects array i.e. point cloud and mesh data=graph_obj, #Layout of the plot layout=dict( scene=dict( #Disable axes’ display xaxis=dict(visible=False), yaxis=dict(visible=False), zaxis=dict(visible=False) ) ) ) fig.show() #Display the figure
- Draw the geometric objects
o3d.visualization.draw_geometries = draw #replace function #draw the point cloud o3d.visualization.draw_geometries([rabbit_cloud]) #draw the mesh o3d.visualization.draw_geometries([rabbit_mesh])
Output: (point cloud followed by mesh)
Note: The output shown above are static images whereas, for the actual output in the colab notebook, the viewpoint of each of the two plots can be changed using a mouse.
References
For detailed information on the Open3D library, refer to the following sources:
