Feeling Virtual Environments With Ultrasound

Force feedback has become commonplace in today's game controllers. To many, the simulated forces create a more realistic game-play environment. A group of scientists at the University of Tokyo in Japan are on working on a new technology that can potentially allow force feedback-type sensations and controller input without the need of a joystick or other physical input device at all--all feedback sensations and input could be done simply in "mid air."

The scientists, Takayuki Iwamoto, Mari Tatezono, Takayuki Hoshi, and Hiroyuki Shinoda, have developed what they call an "Airborne Ultrasound Tactile Display." Tactile feedback in "3D free space" is not necessarily a new concept: in the abstract for a paper the researchers presented at SIGGRAPH08, they mention that "Immersion Corporation developed CyberTouch [Immersion 2007] which features small vibrotactile stimulators on each finger and the palm to interact with objects in a virtual world with tactile feedback." But they also criticize this approach as they state that it "inherently degrades tactile feelings due to the contact between the skin and the device occurring even when there is no need to provide tactile sensation." Another possible approach is to use air jets, but they claim that an "air-jet can not produce localized force due to diffusion," as well as a number of other limitations. These types of touch-based interfaces are referred to as haptic technology.

Instead of requiring a special glove or using air jets, the University of Tokyo researchers use ultrasound. Ultrasound refers to sound waves that are at a frequency usually above that of the threshold of human hearing (20kHz). Ultrasound is commonly used in the medical field, such as with pre-natal sonograms, and other more commonplace applications such as teeth cleaning and in some humidifiers. The researchers explain how their technology works:

"The airborne ultrasound tactile display is designed to provide tactile feedback in 3D free space. The display radiates airborne ultrasound, and produced high-fidelity pressure fields into the user's hands without the use of gloves or mechanical attachments...

The method is based on a nonlinear phenomenon of ultrasound; acoustic radiation pressure. When an object interrupts the propagation of ultrasound, a pressure field is exerted on the surface of the object. This pressure is called acoustic radiation pressure."

There are two primary hardware components to the implementation: a "display" made up of ultrasonic transducers and a hand-tracking camera. As a user's hand moves over the display, the ultrasonic beams are shifted in order to respond to the movement to simulate interaction with the hand and the virtual object. The BBC News reports that "the result is a feeling of tracing the edge or surface of the virtual object," and "at the moment, the system provides a small force only in the vertical dimension, but the team is improving the geometry of the array and the amount of power it can produce so that future devices will provide a stiffer feel and more contoured objects... the team is working to adjust how the transducers are driven in order to produce realistic textures as well as shapes ." Multiple users can interact with the display simultaneously and users can use both hands.

Credit: University of Tokyo

The researchers envision the technology will be used with 3D modeling applications and video games. In their abstract they state that "it is expected that by superimposing the acoustic radiation pressure onto the 3D graphic objects presented with stereoscopic displays, it effectively enhances the reality of the 3D virtual objects."

The researchers do not specifically mention holographic displays, but being able to interact with virtual objects in mid air without being physically tethered to any devices, such as 3D glasses, makes their technology even more compelling. Holographic displays still have a ways to go before we'll see them go anywhere near mainstream implementation, but companies such as Obscura Digital and IO2Technology have publicly demonstrated mid-air display technologies that have a lot of potential.

We're definitely at least a few years away from being able to interact with virtual objects in "3D free space," but that day is coming. The interactive holographic displays from the film Minority Report, or even the immersive holodeck environment from Star Trek: The Next Generation, might be closer to reality that we thought.