Unilever partners with Be My Eyes for AI and QR assisted cooking for visually impaired

19 Jan 2024 --- Unilever has partnered with accessibility app specialist Be My Eyes, expanding the use of on-pack Accessible QR (AQR) to provide AI-assisted cooking experience for shoppers who are blind or have low vision. The collaboration will begin with Unilever’s trial of Be My Eyes technology in the UK through its Colman’s Singapore Noodles Meal Maker. 

By scanning the AQR code on the front-of-pack, shoppers can access the Be My Eyes app, connecting them to a volunteer for live cooking instructions or to a virtual AI-chat bot utilizing Chat GPT-4 capabilities to address recipe or cooking questions.

The FMCG giant will use the AQR codes to provide product information such as usage direction, recycling and ingredient information and nutritional value, interacting with standard smartphone accessibility through the Be My Eyes app to share information via audio description or displaying information in larger text.

This development marks the first integration of Be My Eyes AI technology with a food product, offering an AI-assisted cooking experience at home. 

It builds on the addition of on-pack AQR, developed by computer vision specialists Zappar, to Unilever’s Persil and Colman’s products in the UK last year and is part of Unilever’s global connected pack strategy, which includes using new digital experiences and technology to evolve and differentiate the way shoppers interact with and use Unilever’s products.

“We’ve accelerated digitizing our packs to offer new opportunities for brand engagement and elevated shopping experience and now we’re also focusing on how we can also use digital experiences to make our products more accessible,” Rachana Dongre, senior digital engagement and strategy lead of Nutrition & Ice Cream at Unilever.

Unilever will deploy AQR codes to provide product information by interacting with standard smartphone accessibility through the Be My Eyes app.“Zappar’s AQR codes mean we can support blind and low vision shoppers to have equal access to information and integrating Be My Eyes into these codes offers a new way to make the full experience of our products more inclusive, from the shopping aisle right through to cooking at home.”

7x normal QR scanning range
The Be My Eyes app, which is free, allows users with blindness, low-vision, or deaf-blindness to receive live accessible information from volunteers, trained customer support representatives and AI.

“The 285 million blind and low vision consumers in the world buy the same products as everyone else,” comments Be My Eyes CEO Mike Buckley. “The difficulty in reading package ingredients and cooking instructions has long marred the cooking experience of these consumers. Unilever’s commitment to using new Zappar technology with Be My Eyes is pioneering a new era in product accessibility and disability convenience.”

The Be My Eyes service is integrated into the AQR codes added to Unilever products. These codes, developed by Zappar, work by adding a dot-dash pattern around one corner of the code and are reportedly detectable from further away — “over 7x that of normal QR scanning” — allowing blind consumers to scan them through accessibility apps, making it easier to find desired products.

Caspar Thykier, Zappar’s co-founder and CEO, shares: “Our integration with Be My Eyes is a significant step toward making access to information a basic human right. By combining Zapvision’s computer vision technology with Be My Eyes’ new OpenAI, ChatGPT4 Vision integration, we’re excited to underpin a more conversational and assistive experience for users with unambiguous, structured and accurate data.”

Unilever plans to gather user feedback in collaboration with Be My Eyes and Zappar to optimize the technology for broader integration across different products, categories and countries. The company aims to continue partnering with accessibility experts and apps to share best practices and support more comprehensive integration of AQR technology.

By Radhika Sikaria