7.02.2012

Your personal shopping assistant

It's mobile, informative, and smart.

Have you ever found yourself in the supermarket staring at a shelf full of different cereal boxes, wishing someone could just point out the one with the best price, lowest sugar content, and the best reviews? New solutions for smarter retail will soon give customers the same type of information they get online when researching or comparing products – delivered inside the store as they shop.

A new augmented reality mobile shopping app being developed by researcher scientists at IBM’s lab in Haifa, Israel, is about to change the way we shop in stores. When shoppers use their smart phone or tablet video camera to pan over products on the shelf, the application will instantly display recommendations and offers based on their specific preferences.

Coupon via mobile shopping app.

"We're going way beyond simple facial recognition for products to provide superimposed information that points out the products shopper prefer – whether based on previous purchases, price, consumer rating, sodium content, environmentally friendly packaging, or other considerations," said Amnon Ribak, project leader for the augmented shopping advisor.

For example, a shopper looking for a high-quality facial moisturizer can specify important characteristics of the product, such as having a sun protection factor (SPF) of 15, is hypoallergenic, was not tested on animals, and that is on sale.

As the shopper points a smart device camera at the shelf of moisturizers in the pharmacy, the app recognizes the merchandise and displays information on the device’s screen, superimposed on the product images. It also highlights information based on the shopper's stated preferences, and can offer coupons or special discounts that may apply.

"The idea of standing in an aisle in the supermarket and having your mobile device point out the gluten-free cookies you need can be a real time saver," Ribak said. "This has the potential to completely change the shopping experience from one of hunting, reading, and searching to simply picking up those products you prefer." 

Product recognition doesn't require bar codes or RFID tags. It is done using a blend of different approaches, including image recognition for the packaging that matches the colors and shapes to a database collection, optical character recognition techniques, as well as on-shelf context and positioning. 
To develop this new technology, Ribak leads a team of research scientists whose image processing expertise has already contributed advances in such areas as license plate recognition at night or inclement weather for the Swedish toll road system; identification of malignant tumors in medical images; and optical recognition for ancient texts in libraries across Europe.

The researchers wrote algorithms that combine techniques used in facial recognition, color and shape matching, and associations with surrounding products. The app can take into account the mobile device’s camera angle, and distance from a shelf to help distinguish between products.

"Our first mission was to create our own mini-supermarket in the lab, so we could test the various approaches and challenges involved," Ribak said. "We've already submitted a number of patent applications based on the new techniques we discovered to overcome the challenges in recognizing products with less than ideal lighting, shadows, and reflections."

Aside from assisting shoppers, the new application could also help retail managers organize their stores, instantly point out what's missing on the shelves, or summarize which products were on sale during the week. As more and more shoppers use the application, it will give retailers deeper insight into how their customers shop – which is an analytics goldmine for optimized shelf and store arrangement.   

Read more about IBM's e-commerce technology on the Smarter Planet blog.

19 comments:

  1. I like this idea but wonder what it will do to the shopping aisles if everyone is trying to scan sections of shelves. What do you think?

    ReplyDelete
  2. From what I understand in this article, any time you actually save in the supermarket will just get transferred to time spent at home programming your phone with requirements for your next shopping trip. It also removes much of the 'pleasure' that some shoppers get from the browsing experience as well as those impulse buys which are a very useful way of increasing basket values.

    ReplyDelete
  3. Basically computer can work by the help of three major things. That is hardaware , software and the user. Among these three software plays a very important role in the computer system. Only due to the software we able to do very complex operations. Software is a set of programs,procedures, algorithms and its documentation concern with the operation of a data processsing system.Software systems mainly of three major categories. system software, programming software and application software.EDI VAN

    ReplyDelete
  4. Sounds like a reworking of tech developed by Autonomy's Aurasma and Blinkx's Cheep...

    ReplyDelete
  5. Will change the face of retail shopping. So, when can I pilot it? :)

    ReplyDelete
  6. When will this be available?

    ReplyDelete
  7. Ditto for me! I'd love to beta test this application. When I was part of an Extreme Blue team at the Almaden lab, we designed a similar application. It's great to see the Haifa lab take the application this far!

    ReplyDelete
  8. How can I become a beta tester?

    ReplyDelete
  9. Would also like to beta test and learn more about this product. This type of application is what our mobile customers want to see. Thanks for developing this app!!

    ReplyDelete
  10. When will this be released for use? What is the name of it?

    ReplyDelete
  11. I think this is the best application and people can get good assistance from this application and in this days people prefer online shopping.

    outlet stores online

    ReplyDelete
  12. Its really nice article. I'm not the only person who's thinking of selling my car, despite living in a village. Nearest supermarket, 10 miles; petrol minimum £5 added to shopping bill. Lots of my neighbours thinking like me.
    The mobile shop was once a feature of rural life; is it likely to make a comeback, given motoring costs rising so steeply ?

    online electronics store UAE

    ReplyDelete
  13. This comment has been removed by a blog administrator.

    ReplyDelete
  14. Very exciting development...I can see it be adopted by many consumers looking to shorten the search time in supermarkets. Be interesting if it could link with the location of items in the store, I end up doing many laps of the supermarket to find what I need.
    I think this could also be applied to the Automotive industry for showroom information, cars performance and specification criteria..etc

    ReplyDelete
  15. What would be the process for offering coupons - working out deals with individual manufacturers, or stores? We're not techies, just couponers - sounds like a promising concept even though we poke a little bit of fun here! http://couponsinthenews.com/2012/07/11/in-america-you-find-coupons-in-future-coupons-find-you/

    ReplyDelete
  16. The technology is currently under development. We are partnering with retailers to advance and customize the application based on their needs -- to make the in-store shopping experience more informative and personalized. More to come.

    ReplyDelete
  17. Wonderful blogpost. I'm your new follower. More power to you!

    ReplyDelete
  18. I have to give credit to development of such a complicated software, however unfortunately it doesn't seem to 'change our world'. I cannot see the value of it and I'm not a status quo type...

    ReplyDelete