Seeing and feeling shapes described by a stream of sound

Researchers at IBM’s lab in Tokyo have developed a new method to convert visual shapes on the web into a real-time stream of sound to help visually impaired better understand the shape of an object shown on a web site.

Still an early prototype, the technology converts the shape of an object, such as an image shown on a webpage, into a stream of sound. By listening to the sounds, the visually impaired can mentally feel the shape or better-understand turn-by-turn directions. The technology may also be useful as a real-time guide for online maps which shows route to destination via a mobile phone.

According to a preliminary experiment conducted by the IBM researchers, both blind and sighted people quickly adapted to following directional sound, and recognizing shapes such as randomly drawn paths, letters in the alphabet, and numbers. People with musical training tended to adopt the technology at a faster rate. And some of the participants felt as if they could mentally “see” and “feel” the shapes described by the sounds.

The prototype has the potential to not only help the visually impaired, but to also benefit any user of a mobile device. In situations where a person could use the assistance of a mobile device, but cannot easily look at a screen; or in instances when not all visual information is present (such as in online gaming), the system can provide real-time guidance in the real world or a virtual world.

IBM researchers in Tokyo are advancing the prototype technology with the goal of helping empower the visually impaired to better-navigate the ever increasing non-textual, visual information on the web.