Our Experience with Pseudo LiDAR Technology

Remember When Elon said, “LIDARS ARE A CRUTCH”. Well, that sparked both debate and further research in the field of Computer Vision and led to the birth of a new branch called the Pseudo LiDAR concept.

Before proceeding further, We should first understand the difference between LiDAR and Pseudo LiDAR

LiDAR, or light detection and ranging, is a popular remote sensing method used for measuring the exact distance of an object on the earth’s surface. A LiDAR system measures the time it takes for emitted light to travel to the ground/obstruction and get back. It calculates the traveled distance by measuring that time. Then it converts the traveled distance into elevation. LiDAR system uses its key components to measure these parameters. The key components of a lidar system include a GPS that identifies the X, Y, Z location of the light energy and an Internal Measurement Unit (IMU) that provides the orientation of the plane.

Pseudo LiDAR uses Convolutional Neural Network to convert image-based depth maps to pseudo-LiDAR representation.

One of our Team members at HEVA named Ayushman Kumar, had been researching this field on Pseudo Lidar and converting Depth Maps into point cloud but didn’t find any sample implementation of the procedure or any code. So we worked it out from scratch and thus got what you see below now in the video.

We have a Normal RGB Image from which we did 3D image Reconstruction and Visualization using Intel Open3D and then did Self Supervised Learning.

The Process that we followed to design the Pseudo LiDAR

We found out the Depth Map of the image and created the point cloud using that. Finally, to color the point cloud, we used the intensity of the original RGB image. In simple words convert the whole image into a 3D World image and then treat each pixel from that 3D image as a point cloud data. We generated the heat map based on the blackness and whiteness, which helped in realizing the distances between everything in the image. Then replace each point cloud data back with the pixel and give its color back to it. (P.S. I know this is a lot but hey things like these are never simple).

We trained this model on AWS for about 72 hours. The above video that you see is the models’ performance in a PC running at 12 FPS on NVIDIA GTX 1650.

Application of this technology –
1. Self-driving Cars
2. Drone Technology
3. UAV Technology
4. Space Technology
5. Safety & Security in the real world.
6. Construction
7. Manufacturing
8. Literally any other application that uses LiDAR.

HEVA – HEVA is an idea working towards the future of Autonomous Mobility in every sector, pioneering innovative ways of generating and storing Clean Energy and applying Artificial Intelligence in every industry/business to make it more profitable and help serve their customers even better. We stand for House of Energy, Vehicles, and AI thus called HEVA. Right now the team consists of 6-8 students only.

Article By: