2-D? Looks Like 3-D To Me

Researchers at Carnegie Mellon University's School of Computer Science have come up with a new way for computers to generate 3-D reconstructions of 2-D images.

Using machine-learning techniques, Alexei Efros and Martial Hebert and graduate student Derek Hoiem have taught computers how to spot visual cues that differentiate between vertical surfaces and horizontal surfaces in photographs of outdoor scenes.

Identifying vertical and horizontal surfaces and the orientation of those surfaces provides much of the information necessary for understanding the geometric context of an entire scene. Only about 3 percent of surfaces in a typical photo are at an angle, they found.

Using images located via a Google search, Hoiem presented to a computer examples of vertical and horizontal surfaces, letting a machine-learning program develop statistical associations between certain shapes, shadings, and other characteristics typical of each orientation. The program also takes advantage of the constraints of the real world--skies are blue, horizons are horizontal and most objects sit on the ground.

Hoiem found the computer often discerned which surfaces were vertical or horizontal, and whether a vertical surface faced left, right, or toward viewers. Based on the examples it was shown, the computer identified each feature in an image and assigned to it a probability that it had a horizontal or vertical orientation.

A program they've written lets the computer generate 3-D reconstructions of scenes based on a single image. Animations of the 3-D models are also available. The program, which is freely available in executable form, takes in an original image, a superpixel image, and learned models of geometry. It outputs a labeled image, confidence maps for labels, and the VRML files. Data files (in .mat format) are also available for for training and testing the system.

Posted by Jon Erickson at 09:03 AM  Permalink