IBM 5 in 5: Big Data & sensemaking engines start feeling like best friends

Edior's note: This IBM 5 in 5 post about Big Data and analytics is by Jeff Jonas, IBM Distinguished Engineer and Software Group's Chief Scientist of Entity Analytics. He blogs here, and can be found on Twitter @jeffjonas.

Click through rates for unsolicited advertisements range from near zero to roughly five percent. From the recipients’ point of view, just about every such communication is more time wasting spam.

Imagine a future where some sources of unsolicited advertisement produce such useful and perfectly timed ads, that you would signup. A world where virtually ever text message or email pushed at you is so relevant that this “service” starts feeling like a best friend.

Here at IBM we are working on sensemaking technologies where the data finds the data, and relevance finds you. Drawing on data points you approve (your calendar, routes you drive, etc.), predictions from such technology will seem ingenious. 

Imagine this: You actually sign up for such an unsolicited advertisement service. Three days later, it has suggested nothing. Why? Because there wasn’t anything worth your attention.  But on day four, 10 minutes before you hop in the car to drive to a meeting over coffee, you get this text: “Don’t take the 405, take the 110, then exit 10b, and 3 blocks up there’s a Starbucks.”

You think: Well that is nuts; my coffee shop meeting with my buddy Kenny is at least 15 miles from there. So you text “?” as a reply.

The answer: “Big accident on the 405, will affect Kenny too; already cleared this with him, and this Starbucks is the proposed compromise when considering all the factors.”

Now you could text another “?” to see what these other factors are, but you know it’s pointless. As you pull out of your driveway you are thinking “I love you” as you think about this new best friend in your phone.

This new era of Big Data is going to materially change what is possible in terms of prediction. Much like the difference between a big pile of puzzle pieces versus, the pieces already in some state of assembly – the latter required to reveal pictures. This is information in context, and while some pieces may be missing; some may be duplicates; others have errors; and a few are professionally fabricated lies – nonetheless, what can be known only emerges as the pieces come together (data finding data).

Big Data in context is one of the most significant trends in the information technology field.

This type of technology is going to be real time. Today, smart insight being produced at the end of every week, after a customer left a web site or after a bank already approved a loan only leaves organizations wondering why the answers are so late. Sensemaking systems will deliver sub-200ms insight, fast enough to do something about a transaction, while the transaction is still happening (aka perfect timing).

We at IBM are well down this road towards massively scalable sensemaking analytics.  And whether you will benefit by ingenious advertising services (versus spam) or better health care outcomes, the future will bring higher quality predictions, faster.

Think this topic is the most-likely prediction, or maybe just the most innovative, among the Next 5 in 5? Vote for it by clicking "like" on IBM's smarter planet.  

Labels: , , ,