Dr. Dobb's | Gesturing Interface Developed for Doctors

Gesturing Interface Developed for Doctors

A hand-gesture recognition system that lets doctors manipulate digital images during medical procedures

June 16, 2008
URL:http://www.drdobbs.com/windows/gesturing-interface-developed-for-doctor/208404216

Researchers at Ben-Gurion University of the Negev (BGU) have developed a hand-gesture recognition system that lets doctors manipulate digital images during medical procedures by motioning instead of touching a screen, keyboard, or mouse. This could replace touchscreens now used in many hospital operating rooms which must be sealed to prevent accumulation or spreading of contaminants and requires smooth surfaces that must be thoroughly cleaned after each procedure -- but sometimes aren't.

The vision-based hand-gesture capture and recognition system called "Gestix" interprets in real-time the user's gestures for navigation and manipulation of images in an electronic medical record (EMR) database. Navigation and other gestures are translated to commands based on their temporal trajectories, through video capture. Gestix has been tested during a brain biopsy procedure. In the in vivo experiment, this interface prevented the surgeon's focus shift and change of location while achieving a rapid intuitive reaction and easy interaction. Data from two usability tests provide insights and implications regarding human-computer interaction based on nonverbal conversational modalities.

Helman Stern, a principal investigator on the project, explains how Gestix works: "[There is] an initial calibration stage where the machine recognizes the surgeons' hand gestures, and a second stage where surgeons must learn and implement eight navigation gestures, rapidly moving the hand away from a "neutral area" and back again. Gestix users even have the option of zooming in and out by moving the hand clockwise or counterclockwise." To avoid sending unintended signals, users can enter a "sleep" mode by dropping the hand. To activiate the system, the user waves the hand in front of the camera.

The gestures for sterile gesture interface are captured by a Canon VC-C4 camera positioned above a large flat-screen monitor, and Windows XP-based 600-Mhz Intel Pentium system and a Matrox Standard II video-capturing device. Ongoing research is targeting system expansion to include additional control modes (e.g., voice) so as to create a multimodal telerobotic control system.