When Sodexo, a company that operates over 400 university dining programs, was looking for a futuristic, seamless experience to provide students in place of the usual buffet meal options, it wasn’t necessarily thinking of AI and computer vision.

The only thing the corporation knew was that they wanted to build shop-and-go platforms, a.k.a shops with no cashiers, similar to Amazon Go. That is a store where customers may stroll in, choose products off the shelves, and leave without waiting in line at the register or swiping a code at a self-checkout.

Shop-and-go platforms are popular among students

“Students today want things they can partially or fully prepare in their room or apartment, with organic, highly-local options. We also wanted to remove friction, but many solutions still require the interaction of the guest with a cashier – this generation really doesn’t want to talk to a lot of people in their service interactions,” said Kevin Rettle, global vice president of product development and digital innovation at Sodexo.

Sodexo selected the San Jose-based AiFi, which provides a frictionless and cashierless AI-powered retail system, for the University of Denver. According to Steve Gu, who along with his wife Ying Zheng co-founded AiFi in 2016, it stands out due to its adaptability (the firm claims it can install two stores per week) and variety of venues (sports stadiums, music festivals, grocery store chains, college campuses, and more). Gu and Zheng have doctorates in computer vision and worked for both Apple and Google.

AiFi, which relies solely on cameras and computer vision technology, declared that it already operates 80 checkout-free locations across the globe with partnerships with companies like Carrefour, Aldi, Loop, and Verizon. Additionally, it has opened 2 NFL stores and 53 Zabka stores throughout Poland. Gu claims that this serves as a baseline for the industry in terms of how this technology can scale, unlike Amazon Go, which currently operates in more than 42 locations.

Gu said that the stores of Amazon Go had been outfitted with specialized cameras, sensors, and weighted shelves. Because of this, he claimed, the approach is exceedingly costly and challenging to scale. Instead, AiFi combines what he calls the real power: Computer vision with the “cheapest-possible off-the-shelf cameras.”

Gu explained that AiFi uses many cameras distributed over the ceiling to deploy powerful AI models to comprehend everything taking place in the shop-and-go store. While computer vision recognizes objects and tracks various activities, such as placing items on or grabbing items off the shelf, cameras follow customers as they shop.

Neural network models created specifically for people-tracking, activity detection, and product recognition are hidden behind the shop-and-go platform’s surface. Advanced calibration algorithms created by AiFi also enable the business to recreate the shopping environment in 3D.

“We spend quite a lot of effort building those simulated environments so we can train the AI algorithms and the models inside them. That really helps us develop those models faster and make them more scalable,” Gu said.

AI is the key to being a competitive business

In a virtual environment, he added, you can quickly change human forms and traits in addition to the arrangement of the shelves and the appearance of the goods. A congested, messy store environment or a clean, organized one can be created.

“Things that cannot be done in the real world can be easily done in a simulated world. The AI can learn about those scenarios and then be able to perform or outperform in a real setting,” he explained.

Computer vision technology is improving day by day

Gu emphasized that the AiFi system is developing and would improve with time, highlighting present difficulties with the platform’s ability to distinguish between little objects like gum or lipstick.

“If they are not placed in the right place, it’s very hard for the computer vision to discern what it is. If they are placed together in adjacent spaces, it sometimes confuses the cameras and computer vision to recognize these products,” he said. “But the good thing is that it’s not purely based on the visual texture – you also have the 3D scene geometry, the location, the context as well,” he said.

The store’s size and the number of customers it can track are also currently constrained. “The question is can the solution also be scalable to supercenters of 100,000 square feet? Also, the system can track hundreds of people shopping simultaneously in a shop environment. But to further scale, to track thousands of people, with very complex shopping behavior, that’s something that is still a work in progress,” he stated.

Customers can utilize a credit card swipe or the retailer’s app to enter an AiFi-powered store instead of a biometric scan or the AiFi app. For instance, Sodexo sought a neutral partner from the start at the University of Denver.

“We were able to use our wallet and payment processing and tie the AiFi technology, the cameras, and the AI into our system,” stated Rettle.

Adopting customers is important

“From a product ownership perspective, you always kind of hold your breath. Is it going to work?” he said. But in the end, students at the University of Denver embraced the AiFi idea immediately.

“We didn’t have to teach any of the students what to do. They get it without having a bunch of prompts,” he said.

The AiFi technology would be a “loss-prevention nightmare — that the students will figure out how to game the system,” according to critics in the retail industry, Rettle added. Instead, he claimed that the AiFi solution currently has a 98.3% accuracy rate and that the shrink rate—the proportion of customers who leave without paying—has actually decreased.

Rettle added that he does not see a campus or stadium transitioning to entirely autonomous shopping.

“For us, it’s something that complements. But I see a strong future in terms of continuing to deploy and drive ubiquity with the solution based on consumer acceptance,” he stated.

Researchers are developing an AI pilot capable of navigating crowded airspaces

With over a dozen more outlets planned and developing cooperation with Microsoft as an independent software vendor partner, Gu believes that AiFi’s potential is “huge” (AiFi runs its solution on Azure).

“You’re going to see a lot of autonomous retail in a variety of verticals — not just stadiums, festivals and universities, but offices, movie theaters and other spaces,” he explained.