IBM computer system a scientific achievement in natural language processing

Editor's note: This is a guest post from Charles Lickel, Vice President, Software, IBM Research.

The IBM Watson computer system, named after company founder Thomas J. Watson, Sr. catapults data analysis by understanding the nuances of "natural language" that humans use every day.

A breakthrough in the scientific field of DeepQA – a field aimed at furthering computing intelligence – when we began this project more than four years ago, others in the scientific community believed this task to be impossible and we were uncertain about what we would be able to achieve. In fact, it took our research team two years to get the system to be able to analyze a Jeopardy! clue and provide an accurate response in less than three seconds. That was the moment we knew we had a chance of achieving a breakthrough in artificial intelligence and natural language processing, as well as a chance to compete against Jeopardy! champions.

Watson uses Unstructured Information Management Architecture (UIMA), IBM’s open-source framework for analysis of unstructured content, to understand natural language text, speech, images and video. The complex algorithms for the analytics engine behind Watson are tuned specifically for “open-domain” QA – covering a wide domain of knowledge including history, geography, arts, science, sports, and popular culture. It helps Watson handle the broad range of information that language can express and evaluate the “evidence” it collects to determine a confidence level to answer a question.




This fall, the Watson system and the Research team achieved remarkable results, when Watson played more than 55 “sparring games” against former Jeopardy Tournament of Champions contestants.

Watson's ability to understand the meaning and context of human language, and rapidly process information to find precise answers to complex questions, holds enormous potential to transform how computers help people accomplish tasks in business and their personal lives. Watson will enable people to rapidly find specific answers to complex questions. The technology could be applied in areas such as healthcare, for accurately diagnosing patients, to improve online self-service help desks, to provide tourists and citizens with specific information regarding cities, prompt customer support via phone, and much more.

Like Deep Blue, the IBM supercomputer that defeated the reigning world chess champion in 1997, Watson represents a major leap in the capacity of information technology systems to identify patterns, gain critical insight and enhance decision-making despite daunting complexity. But while Deep Blue was an amazing achievement in the application of compute power to a computationally well-defined and well-bounded game, Watson faces a challenge that is open-ended and defies the well-bounded mathematical formulation of a game like chess. Watson has to operate in the near limitless, ambiguous and highly contextual domain of human language and knowledge.

Win or lose, Watson will open eyes to what kinds of questions a computer can answer, and open doors to what kind of problems a computer can solve. For more about the technology behind Watson, visit ibmwatson.com.

Labels: , , , ,