Affectiva raises $14 million to bring apps, robots emotional intelligence

Affectiva, a startup developing “emotion recognition technology” that can read people’s moods from their facial expressions captured in digital videos, raised $14 million in a Series D round of funding led by Fenox Venture Capital.

According to co-founder Rana el Kaliouby, the Waltham, Mass.-based company wants its technology to become the de facto means of adding emotional intelligence and empathy to any interactive product, and the best way for organizations to attain unvarnished insights about customers, patients or constituents.

She explained that Affectiva uses computer vision and deep learning technology to analyze facial expressions or non-verbal cues in visual content online, but not the language or conversations in a video.

The company’s technology ingests digital images—including video in chat applications, live-streamed or recorded videos, or even GIFs—through simple web cams typically. Its system first categorizes then maps the facial expressions to a number of emotional states, like happy, sad, nervous, interested or surprised.

Over time, the company’s systems can learn to identify more complex emotions. The startup hopes to be able to accurately parse expressions of hope, inspiration and frustration from non-verbal cues in the future, for example. So far, Affectiva has amassed a data repository of 4.25 million videos from people in 75 different countries yielding over 50 billion emotion-related data points.

Assessing facial expressions across gender and international lines proved essential to making “emotion AI” work, el Kaliouby said. Some expressions are unique to different geographies. For example, Affectiva has discovered a “politeness smile,” that is not representative of “happiness,” or “smirking,” which is a prevalent expression among communities in Southeast Asian countries or India. But the politeness smile is not prevalent in the Americas, Africa and Europe.

Early adopters of Affectiva’s emotional intelligence systems have included independent video game studios and brand or advertising teams within large corporations like Unilever, Kellogg’s, Mars and CBS.

Flying Mollusk Studio used Affectiva’s Affdex software development kit to make a psychological thriller genre video game called Nevermind, which becomes more challenging when players feel scared or nervous. If players can master their emotions, and stay calm or calm down, the game levels are easier to master and less surreal.

Advertisers and marketers have also used Affectiva’s software-as-a-service to conduct focus groups, of a kind, where respondents don’t have to describe how they feel as they review an advertisement, program or new product for the first time. Their emotional response can simply be assessed through the software.

The idea is that if a majority of viewers of a film, program or commercial lose interest at a certain point, companies that use Affectiva for testing will know it, and be able to tweak their content after getting “emotional journey” reports back from a focus group.

It’s not hard to imagine bad actors using the power of emotional intelligence to swindle or hurt people. And of course marketing and consumer research is always a bit manipulative.

Affectiva's systems can glean a user's emotional responses by analyzing their facial expressions.

Affectiva’s systems can glean a user’s emotional responses by analyzing their facial expressions.

But el Kaliouby emphasizes that Affectiva doesn’t and won’t develop lie detection features, and the company requires businesses using its technology to get explicit opt-ins and consent from their end users before analyzing their facial expressions.

“It’s like if you are playing poker, you can mask some of your emotions,” she added. “We want users to know exactly what our technology can and can’t do, and to be aware when they are engaging with it.”

Investors in Affectiva expect the company to take “emotion AI” into new corners of tech, especially health, robotics and education, said Fenox Venture Capital’s CEO and General Partner Anis Uzzaman. Fenox has also invested in the personal robot makers, Jibo. It’s easy to envision a potential partnership there.

Uzzaman sees a future where things like Echo, Siri or even Google Maps are empathetic and responsive to a user’s mood.

The firm’s limited partners are all corporations that want an early look at cutting-edge technologies that startups have to offer, and which they can use to improve their own businesses. With this deal, Uzzaman said, Fenox connected Affectiva with the likes of Bandai Namco and Sega Sammy Holdings, toy and video game makers who could use emotion AI in entertainment.

Today the company also announced that el Kaliouby has moved into the role of CEO.

Nicholas Langeveld, Affectiva’s previous CEO, is moving into the role of chairman. During his tenure as CEO, Langeveld led Affectiva’s growth from a company with a handful of early customers to one whose technology is used by 32 Fortune 100 companies and in 75 countries.

He also led Affectiva’s fundraising, attaining $34 million in venture capital to date, including the latest round led by Fenox, and $20 million in earlier venture funding from investors including WPP, Li Ka-shing’s Horizons Ventures and KPCB.