Is Facial Recognition in Retail Market Research the Next Big Thing?
Remember the memory-erasing Neuralyzer in "Men in Black"? Or more recently, "Ex Machina," the Oscar-winning story of a humanoid robot that uses emotional persuasion to outsmart humans and escape from the secluded home of its creator?
While movies have been envisioning crazy, new technology for decades, some of those inventions are starting to become reality. From virtual reality and wearable devices to facial and emotional recognition technologies, these products and systems are changing the way we communicate, interact and conduct market research (MR) in several industries, most notably, retail.
One of the hottest areas of technology development in retail research is facial and emotion recognition. Understanding emotions is powerful in areas of research such as ad testing, but difficult to achieve. Facial expressions are linked to emotions, and research organizations have used human observation of recorded videos in retail settings to try to assess emotional response for years. Human assessment has many limitations, and facial expression recognition technology offers an opportunity to overcome some of these limitations, delivering a much greater level of insight about personal sentiment and reactions.
Organizations managing research programs and retail customer experience activities can use emotion detection technology to analyze people’s emotional reactions at the point of experience. This knowledge not only gives researchers a greater understanding of behavior patterns, but also helps predict likely future purchasing actions of that consumer.
The result? Remarkable insight into what impacts customer emotions, as well as valuable information that can drive better business decisions, resulting in improved product and service offerings and experiences.
Do we need this? How will we use it?
Competition only continues to grow with retail, making experiences more important than ever. Thus, market researchers are under increasing pressure to deliver business value to their customers. Adding to that pressure is ongoing declining survey response rates and challenges with collecting data from specific demographic groups. Emotion detection provides real opportunities to drive customer spending and enhance loyalty.
The primary use case for those researchers implementing emotional detection is ad testing. Within a survey an advertisement can be shown, during which time, the respondent’s webcam will record their reaction.
Traditionally, respondents will answer questions about the advertisement they’ve been shown, rating it on various scales. While broadly effective in most cases, this is dependent on the respondent’s ability to recall what they’ve just been shown, their interpretation of their own emotions, and their ability to put those emotions into words. Researchers can also observe and record emotions while the video content is being shown, but this needs specific skills and is difficult to perform consistently.
Does it work?
The ability to use video to recognize, understand and report back on the tiniest facial movements doesn’t sound far away from the "Ex Machina" humanoid. Research shows there is a broad array of expressions and micro-expressions that relate to specific emotional responses, and so using technology to capture those facial movements and analyze them against the benchmark data is hugely powerful. Some tests report an accuracy rate of around 95%, which by any measure is impressive.
The technology is already in use by a number of retailers, who have been able to refine their advertising campaigns according to respondents’ reactions to test adverts.
What’s the downside?
If you’re looking at bringing emotion detection into your arsenal, consider the global nature of your programs. People from different nationalities and cultures have different levels of emotional response, and different facial structures, so your benchmark data needs to take this into account.
A second issue to consider is how to deliver your content. While many people are now used to engaging with video content through a variety of media, facial recognition technology requires a two-way view. This means that not only must your respondent be able to clearly and effectively view your video content, they must be in a position where their camera is capturing their expressions clearly. Different lighting levels and different angles of viewing all need to be taken into account.
Finally, you will need to specifically ask respondents for permission to access their webcam and record their faces while they’re watching your content. For many, this won’t be an issue, but if you’re specifically targeting, for example, an older demographic who may think you’ve shifted from curious to creepy, then you may be on safer ground to show video content and ask questions instead of observing their expressions.
Emotion detection software simply adds to the toolkit available to retailers who are looking to improve their customer experiences and create more effective advertising campaigns. It may further reduce the need for focus groups, but beyond that, it’s an addition, not a replacement. Such videos will, in most cases, be embedded into a survey, and additional information will be required to understand more about the shoppers themselves.
No doubt new applications of the software will emerge in both MR and customer experience disciplines – some of which will fly and some of which won’t. As with most advances of the last decade, emotion detection will find its place and help forward-thinking retailers add additional value to the services they provide to their customers. In turn, this will ideally help progress the retail space, helping retailers take their ability to provide customer service and implement customer feedback to the next level.
Terry Lawlor is executive VP of product management at Confirmit, where he is responsible for all aspects of product management, including strategy development, product definition, and product representation in client and marketing activities.
What the Age of the Connected Customer Means for Brick-and-Mortar Retailers
For years, retail was neatly divided into two categories: e-commerce and brick-and-mortar. But with the advent of smartphones, digital and physical worlds are melding together, creating a new retail environment in which almost every customer journey involves both online and offline activity. This is particularly true for Generation Z consumers, the first generation to have grown up with digital technologies at their fingertips and who now make up 25% of the total U.S. population.
In response, Amazon has revealed its vision for the future of retail: Amazon Go, a brick-and-mortar grocery store featuring a new checkout-free shopping experience powered by a smartphone app and driven by “computer vision, sensor fusion and deep learning” technologies. While this is exciting for consumers and companies alike, retailers must now brace themselves for a market wherein creating shopping experiences that meld the digital and physical are standard practice.
So how can traditional retailers keep pace with Amazon Go’s disruption and engage the “connected customer?” What technologies do brick-and-mortar stores need to set the foundation for digital-powered shopping experiences? And how is the role of the store associate poised to change with this digital retail revolution?
Taking advantage of mobile
Engaging customers on their smartphones is one of the biggest opportunities in retail. For many of us, smartphones have become an extension of ourselves and we cannot leave home without them. In the U.S., we spend roughly 4.7 hours on our phones every day, relying on our trusted devices for everything from driving directions to product reviews. Given this, it’s not surprising that mobile devices influenced more than $1 trillion in total purchases in 2015 between online and offline transactions.
Customers still prefer shopping in stores for most products, but they want to use their smartphones to inform their purchasing decisions and make the process easier. PwC found that 29% of U.S. shoppers compared prices using their smartphones in store. Shoppers are literally standing in stores, smartphones in hand, looking for information to assist with their shopping trip.
But brick-and-mortar retailers have been slow to adopt technology that engages a shopper in-store, and now competitors such as Amazon are beating them to the punch. Despite this, traditional retailers still have the upper hand over Amazon in the physical retail market given Amazon’s lack of experience managing brick-and-mortar locations.
In order to maintain this lead and engage the connected customer, retailers must work to create in-store digital strategies that enhance the shopping experience and reach customers on their mobile devices.
Laying the groundwork for a digitally powered shopping experience
In order to create digital-powered shopping experiences, retailers must first digitally map their physical stores so they know where everything is located, including products and services, all of the time. This provides the critical context needed for accurate product location, indoor navigation, retail store analytics and store fulfillment programs to function.
Retailers that have digitally mapped their stores can show shoppers exact inventory and product locations — down to the aisle and shelf — on their mobile devices and offer navigation that directs customers along the most efficient path through the store based on their shopping lists.
Knowing precise product location also allows retailers to send personalized deals, product suggestions, coupons and other valuable messages based on shopper preferences and location. The more personalized the offers are, the more likely shoppers are to redeem them. If a shopper is standing in the cleaning products aisle and receives a coupon for their favorite brand of laundry detergent, they’re more likely to engage with the offer and take action. In this way, digitally mapping the store can drive not only customer purchases, but also satisfaction and loyalty.
Evoving role of store associates
As smartphones transform the shopping experience, they are also changing the role of the store associate and empowering them with information at their fingertips to service customers. With the rise of the checkout-free model, store associates will shift away from their role as cashiers, instead focusing on serving as a well-informed resource for customers.
Now that shoppers have access to endless information via their smartphones, they enter stores fully expecting sales associates to readily provide prompt answers to the two most important questions, “Do you have this item?” and “Where can I find it?” Arming store associates with technology that provides visibility into product inventory, product location and store metrics allows them to answer these questions and offer informed, personalized product recommendations. This type of service leads to increased revenue and basket size for retailers and improves the overall shopping experience for customers.
Further, as buy-online-pickup-in-store becomes more popular, store associates will also spend more time fulfilling online orders for in-store pickup. Mobile tools that give store associates insight into product inventory and product location are critical for ensuring orders are filled as quickly and efficiently as possible.
In the age of the connected customer, traditional retailers must find a way to leverage the mobile channel to reach shoppers or risk losing to competitors who are embracing a new digitally-focused era of shopping. Global retailers will continue to invest in technologies that improve in-store shopping experiences, and as the world becomes fully connected, those retailers not planning for the future will inevitably be left behind.
Patrick Giusti serves as the chief revenue officer of Point Inside, a company providing digital in-store product location solutions for top retailers to help mobile shoppers search, plan, and complete their in-store shopping trips.
Accessories retailer aims to give stores a local feel
Sunglass Hut is making a strategic move to evolve its brick-and-mortar experience.
The sunglass retailer has added a a cloud-based merchandising system across its stores in Australia and New Zealand that helps each store localize and personalize its product mixes to create a shopping experience that feels custom-tailored.
The Merchandising Cloud solution, from One Door, combines product and promotional visual merchandising with store information, providing each location with a unique, unified, and interactive merchandising plan. The application lets Sunglass Hut capture and maintain an accurate digital model of all store attributes, floor plans, fixtures, and resources.
Armed with this information, the chain can plan and execute campaigns with the speed and agility of digital, while enabling store associates to better personalize the in-store experience.
"The in-store experience is integral to Sunglass Hut," said Kate Grech, the chain’s marketing and e-commerce director. "As our business continues to grow, One Door’s Merchandising Cloud will assist us in delivering a consistent in-store experience, and keep pace with rapidly changing customer needs.”