Facebook Moments Uses Facial Recognition To Send Friends Your Photos They’re In

Frustrated with friends who take photos of you, say they’ll pass them on, but never do? Facebook’s newest companion app Moments could retrieve your memories trapped on other people’s phones through facial recognition.

Moments scans your Camera Roll for the all the photos featuring a friend’s face, and bundles them up for one-tap private sharing with that friend. They can contribute to the collaborative private album too, so all your pals from the party or vacation can get each other’s pics.

Moments launches today in the US as a companion app to Facebook and Messenger on iOS and Android (those links will be live soon if not already). If it goes viral, it could be the end of everyone insisting to take that big group photo on their phone too. But with imperfect facial recognition and an unfamiliar sharing style, Moments might fade away or get rolled into the main Facebook app.

You can check out my quick video demo of Moments below:

Leave It To The Artificial Intelligence

I first reported on the development of Moments back in September 2014 as an ultra-private way to share photos. Until now, Facebook’s private sharing methods like manually making Friend Lists to post to or joining Groups have been rigid, clunky, or confusing. Moments wants to make sharing a wider set of content, like photos of friends they’d want to see but not necessarily need posted to Facebook, much quicker and easier.

facebook-college-hackathon-done1Normally, to send pics to friends from a group outing or party, you’d have to track down everyone’s email addresses or phone numbers and either blow up their inboxes with big photos, or blast them out a few at a time. On Facebook you’d have to make a Friend List and share to that, force everyone into a Group, or expose photos to non-friends on an Event wall. It’s a chore, which explains why when someone says “Oh, you don’t need to take this same photo, I’ll send you mine”, they rarely do.

Facebook’s research into what problems were left in photo sharing kept coming back to this. Moments’ Product Manager Will Ruben tells me the pain point kept coming up “loud and clear in the research.”

After several projects from Facebook’s internal hackathons tried to solve it, product manager Will Ruben assembled a small team to build Moments as part of Facebook’s Creative Labs initiative to spin up mobile design experiments.

“One of the difficulties of sharing more privately is that it’s a lot of work to select the audience you want to share with every time”, Ruben tells me. “We looked into bluetooth. We looked into location-based sharing” but Facial recognition is what Ruben’s team decided was the answer. You can read more about what’s under the hood in  Facebook engineering post.

That might sound a tad creepy, especially since Facebook had to temporarily shut down its previous facial recog feature in Europe due to privacy concerns. But it’s basically just automatically detecting which Facebook friends are in your photos — something you could do yourself but that would be tedious. The facial recognition tech comes from Facebook’s artificial intelligence lab and is the same algorithm that powers tag suggestions when you upload pics to the social network.

Facebook Facial Recognition Opt Out

In 2012, contact sharing app Bump released its own attempt at rescuing photos from friends with Flock, which picked who to share with through co-location rather than facial recognition. But it was shut down when Google bought Bump and rolled some of Flock into Google Photos. Ruben says “I’ve used a bunch of these different apps. What we’d done that’s unique is to take all of the friction out of sharing while still keeping you in control.”

If you’re squeamish about being recognized by robots in your friends’ photos, you can opt out here in you Facebook settings in Timeline and Tagging under “How can I manage tags people add and tagging suggestions?

Making Moments

Moments is a separate app but it piggybacks on your Facebook friend list. When you fire it up, it will scan your camera roll for faces and photos taken right before or after, and cluster these into sets. Moments then suggests you sync these sets with the people who appear in them with a right swipe, or dismiss the recommendation with a left swipe. Yes, it’s like Tinder for sharing your photos.

Facebook Moments ExampleRuben walks me though an example. He and some friends went on a recent kayaking trip. He took photos of the landscape, big group pics of everyone, and shots with just one or two of them. Afterwards, Moments detected the faces of everyone, bundled all the photos from the trip together, and suggested he sync them with everyone there. Only those friends could see them, and they’re not posted to Facebook proper unless someone explicitly reshares them there.

When you sync with people from Moments, they’ll get a notification in the app to check them out, save them to their phone, Like their favorites, share the best ones on Facebook or other social networks, and contribute to the private collaborative album. They can then add to the album and tag in more people to give them access.

While Facebook scans all your pics for faces of friends, it doesn’t copy your photos to its servers unless you sync them with someone. I asked why Facebook wouldn’t want these pics to improve its facial recognition algorithm, and Ruben laughed telling me “We have plenty of photos.” Hundreds of billion, in fact.

If you don’t want to share to friends, Moments also offers an internal search engine that will let you type in a name to see all your own photos and ones synced to you by friends that shows them. Eventually if Facebook’s search, machine vision, and image recognition improve, it could also let you surface places or things. Ruben tells me Moments will improve as AI Lab makes progress, saying “You could easily image being able to search your synced photos for dogs or baseball.”

Cleverly, if you sync to someone without Moments, they’ll get pinged in Facebook Messenger with a stylish Ken Burns-style slow pan/zoom animated preview. So while Moments will start with zero users, it’s affiliation with Messenger gives it a potential growth advantage. Moments could also send you a push notification when it’s finished processing a batch of newly taken photos and has suggestions for who should see them.

Fading From Memory

Moments will need all the help it can get. Facebook’s Creative Labs initiative hasn’t had much luck scoring traction with its experiments around new mobile utilities and interfaces..

Snapchat clone Slingshot never quite took hold. Forums app Rooms and collaborative video maker Riff both disappeared from public knowledge within days of launch. I don’t hear celebrities praising Facebook’s Mentions app for VIPs. No-brainer standalone app Facebook Groups made sense since hundreds of millions of people. already use that product. It’s too early to tell the fate of Android-only social caller ID app Hello.

Screen Shot 2015-06-15 at 9.49.37 AM

More frequently, Facebook has learned what people will and won’t use from its Creative Labs projects, and then brought those insights elsewhere.

The biggest success yet was newsreader Paper. While it’s built a relatively small but loyal following, now it’s best known for powering the flashy designs of Facebook’s hosted Instant Articles. Slingshot taught Facebook for Android how to draw on photos. And back in the day, Facebook learned multi-photo uploads and filters from the short-lived Facebook Camera, and brought stickers from Messenger to Facebook comment threads.

Moments-Cartoons

Overall, the utility of Moments is pretty obvious. It could nag friends to send you photos with your face. But as a whole, it feels foreign and imprecise.

Moments repeatedly mis-tagged people, mistaking friends for each other just because they’re the same ethnicity, or grouping photos together that had nothing to do with each other. Left swiping a set seemed daunting, even though it doesn’t delete you photos. Right swiping means anyone you sync with could then share those photos wherever they want. And the stacks of photo sets you’re choosing whether to sync aren’t easy to make decisions about in a glance.

Putting this all inside Facebook’s main app might not feel quite right either, as people might be confused whether they’re privately syncing with a few friends or posting to everyone. But given that most people aren’t taking reams of photos of friends frequently at such a high volume, Moments might not warrant a premier spot on our phone.

Moments is not a photo storage app like Google’s awesome new Google Photos app that offers unlimited free storage. That’s something for everyone. Moments feels like an app built for Facebook employees — social, smartphone-addicted twenty-somethings with dense social graphs in urban areas. Normal people might get by sharing their Moments the old-fashioned way.