Steven K. Feiner | Augmenting Reality

Steven K. Feiner
Professor of Computer Science
This profile is included in the publication Excellentia, which features current research of Columbia Engineering faculty members.
Photo by Eileen Barroso

What happens when a mechanic must work on an unfamiliar piece of equipment? He or she will pull out a manual and keep referring to it while making repairs. Steven Feiner has a better alternative, one that changes how we see the world around us.

His approach to this problem involves augmented reality (AR). Unlike virtual reality, which creates an artificial world, AR adds virtual information to the real world. AR can guide people through complex tasks.

“Instead of looking at a separate manual while disassembling a PC, imagine putting on lightweight eyewear containing a see-through display that graphically highlights the screws in the order you need to remove them,” Feiner said.

Feiner has been developing experimental AR maintenance applications for 20 years. This involves delivering information about a system, quickly and naturally, as workers move around a workpiece. He does this by tracking the position and orientation of their eyewear, then aligning information with their perspective.

In recent studies with U.S. Marines at Aberdeen Proving Ground, Feiner’s lab found that AR helped professional mechanics find the location of parts they needed to repair faster than using manuals. “With manuals, the documentation is separate from the task. Workers are always going back and forth. AR keeps them focused on the work by integrating the documentation with the task,” Feiner explained.

Feiner is also working on better ways to display AR information for people interacting with their surroundings. He has come a long way since 1996, when his lab created the world’s first outdoor mobile AR system. Before the era of smartphones, ubiquitous GPS, and Wi-Fi, it consisted of head-worn and hand-held displays—plus a 45-pound backpack stuffed with electronics.

That system let users tour Columbia’s campus, overlaying the names and websites of academic departments on their buildings. Within a few years, Feiner’s lab had added multimedia news stories and created AR restaurant guides.

Today’s smartphones are far smaller and more powerful than those early AR systems. Feiner and his students are harnessing their power, both alone and with other computers and displays ranging from wearable to wall-sized.

AR displays can create a compelling experience. That is why Feiner wants to ensure that every AR system respects the physical environment and the user’s relationship to it. “We don’t want users losing awareness of the world around them while trying to cross a busy street,” he said.

B.A., Brown, 1973; Ph.D., Brown, 1985

500 W. 120th St., Mudd 510, New York, NY 10027    212-854-2993