Kinect as Webcam Real-Time 3D Video Capture Project

The Kinect, a webcam-style motion sensor add-on for Xbox 360 and possible PC under Windows 8 which enables controller-free gaming and entertainment experience, has been proven to have much better functionality and usage possibility than just a gaming controller.

Just officially been hacked last week for use with other systems besides the XBox 360, Kinect is now can be used as 3D webcam to capture video in 3D. A PhD student developer from University of California (UC) Davis named Oliver Kreylos, has managed to reverse engineer the open source Kinect driver for Linux by Hector Mactin who won a $3000 bounty last Thursday for the hack, and develop his own hack which able to create live 3D image of the surrounding.

The hacker directs the Kinect camera’s infrared depth and RGB color feed streams to a custom program that interpolated and reconstructed the result as 3D image live stream or 3D video virtual reality environment that user can manipulate, scroll, spin around, zoom in, zoom out, rotate turn, twist and look behind object at will. In other word, Kinect is now a 3D video camera or webcam.

View the demo video showing 3D video capturing with Kinect:

The 3D video capture is not ideal though, as with only a single camera it is impossible to capture all angles necessary for a full 3D reconstruction. Nonetheless, the 3D reconstruction can also be used to measure objects digitally on computer, as shown in the video below:

The 3D reconstruction code is entirely written from scratch in C++, using the author’s own Vrui VR toolkit for 3D rendering management and interaction. The initial version of the Kinect 3D Video Capture Project has been released publicly, and is available for download from project homepage at idav.ucdavis.edu.

This software is based on the Vrui toolkit, version 2.0. However, the most recent public Vrui release is version 1.0-068. This means the software will not build until Vrui-2.0 is released in the near future.