Team Develops Faster, Higher Quality 3-D Camera

When Microsoft released the Kinect for Xbox in November 2010, it transformed the video game industry. The most inexpensive 3-D camera to date, the Kinect bypassed the need for joysticks and controllers by sensing the user’s gestures, leading to a feeling of total immersion into the game. Microsoft sold 8 million Kinect units within 60 days, making it the fastest-selling electronic device ever.

“But then something interesting happened,” said Oliver Cossairt, assistant professor of electrical engineering and computer science at Northwestern’s McCormick School of Engineering. “Microsoft made the software available for the 3-D capture part of the device. People were able to integrate this inexpensive consumer device into a variety of platforms, including robotics and navigation.

But users quickly discovered the Kinect’s limitations. It does not work outdoors, and it produces relatively low-quality images. Now, Cossairt’s team has picked up where the Kinect left off and developed a 3-D capture camera that is inexpensive, produces high-quality images, and works in all environments—including outdoors.

Supported by the Office of Naval Research and the US Department of Energy, the research is described in the paper “MC3D: Motion Contrast 3D Scanning,” presented on April 24 at the IEEE International Conference on Computational Photography. Nathan Matsuda, a graduate student in Cossairt’s lab, is first author, and Mohit Gupta from Columbia University is co-author and collaborator.

Both first and second generation Kinect devices work by projecting light patterns that are then sensed and processed to estimate scene depth at each pixel on the sensor. Although these techniques work quickly, they are less precise than expensive single-point scanners, which use a laser to scan points across an entire scene or object. Cossairt’s camera uses single-point scanning in a different way. Modeled after the human eye, it only scans parts of the scenes that have changed, making it much faster and higher quality.

“If you send the same signal to your eye over and over, the neurons will actually stop firing,” Cossairt said. “The neurons only fire if there is a change in your visual stimulus. We realized this principle could be really useful for a 3-D scanning system.”

Another problem that plagues the Kinect: it does not work well outside because the sunlight overpowers its projected light patterns. The laser on Cossairt’s camera, however, can be sensed in the presence of the sun because it is much brighter than ambient light.

“In order for a 3-D camera to be useful, it has to be something you can use in everyday, normal environments,” Cossairt said. “Outdoors is a part of that, and that’s something the Kinect cannot do, but our Motion Contrast 3-D scanner can.”

Cossairt believes his camera has many applications for devices in science and industry that rely on capturing the 3-D shapes of scenes “in the wild,” such as in robotics, bioinformatics, augmented reality, and manufacturing automation. It could potentially also be used for navigation purposes, install on anything from a car to a motorized wheelchair. Cossairt’s group received a Google Faculty Research Award to integrate their 3-D scanning technology onto an autonomous vehicle platform. Their scanner will provide high-quality 3-D scans in real time without only a fraction of the power of competing technologies.