Saturday, May 2, 2015

3D scanning with blender and python

For my physics project this year I arrived on the conclusion that I needed to make a 3d scanner. It was a gutsy move since I knew that if I became committed to this I could not buy this anywhere in the market and would have to build it myself. This burnt all the white flags I may have had and made sure I made my own project. Let me tell you it was horrible. What I had in mind was something awesome, what I obtained was something organic. Graciously Anurag helped me with this herculean task.

First was the problem of the laser. It was too damn expensive. I bought a laser diode from a friend and quickly burnt. What I did not realize was that intensity of the laser depended on the current provided and not on the voltage.

Image of laser and camera setupThen came the problem of making a line laser out of a point laser. The first method I stumbled upon was to use a rotating mirror placed in front of the point which would cause a circle of laser light. This failed miserably as the required RPM was not met. Then I came upon the idea of using a cylindrical lens. A glass stirrer cut to size( not cut in prototype 1) was nimbly attached to the front of the laser pointer and lo behold we had a laser line.

Now to tackle the problem of holding my mobile phone upright. This was indeed a messy one. After some time I gave up and replaced it with a friend's DSLR. Problem solved. The camera now sits perfectly on it's own body.

The rotating mechanism was simplicity itself and was simply too easy to build. A little DIY(or jugaad for that matter) and we had a rotating pedestal.

What came next was the mathematics. After digging around a lot I still could not understand exactly how this thing was supposed to work. Then came a moment of truth and everything was a walk in the park. Using blender I managed to extract frames from the video and ended up with about a thousand frames to work with.

Next came the cleaning of the frames. A simple blur, increased contrast and grey scale conversion gave me a very good image of the laser. Then we selected the brightest point in every row of the image and marked it as the laser line.

Next came the reconstruction. With a simple python script I managed to get the cylindrical coordinates of every point in the picture. With the knowledge that the object was rotated 360 degrees and with the assumption that the rate of rotation did not change much I  recreated the scene.

A collection of points was created and saved as a scene. This was then put into blender to create a 3d representation of the object which was very very wrong. What had happened was that the glow of the laser had reflected off the laser and created data points where there should have been none. This created a scan which had a lot of errors.

I later realized that I was calculating angles in degrees and python implicitly(any good program) uses radians. Recalculating the slices led to a new plot which fared a lot better than the previous ones. A lot of the reconstruction was noise but I could make out the nose, and ears of the Buddha statue. It was a magical moment.
Finally Anurag scanned another object, an emergency flashlight. The results were good and funny at the same time. The flashlight had a small volume and so the point cloud was very dense. During the scan the strap attached to the flashlight was also scanned. It was pleasing to note that the scan reconstructed the strap too. Due to the dense point cloud it was not easy to make out the rest of the geometry of the flashlight.
To see the  geometry we moved the point of view to inside the flashlight and could see the objects clearly.

The next problem to be tackled was the problem of mesh regeneration from the point cloud. The problem was that our cloud had non uniform density. This led to some algorithms being discarded. Ball Pivoting Algorithm and Poisson Surface Reconstruction are what got my eye. Will be writing about them soon. All the source code is available on my Github Page.