Looking for outfits and what to wear is one of the bigger use cases on Pinterest, but as the company looks to try to make the process of getting the right content to users more seamless, it’s been increasingly focusing on its interface with the real world with the camera.
Now the company is going a step further in tapping that big use case by letting Pinterest users tap the camera’s visual search technology with clothes that are already in their closet as they decide what they want to wear today. Instead of searching for clothing options with ripped jeans, users can now point their camera at their pair of ripped jeans. They type in a kind of search — say “outfits for the beach” — and then point their camera to dive deeper into that rabbit hole with a specific product they already own. Pinterest will then execute a visual search on those jeans, and users can tap around and see various kinds of outfits that might interest them.
“The way lenses work, you take a picture of something and we try to match the picture with all the images that are most similar,” visual search product lead Jeff Harris said. “What we’ve done with Lens Your Look is taken a step deeper. We looked within our fashion pins with all the articles of clothing that are represented of that outfit. We’re not looking for whole pins with jean jackets, we’re looking for outfits where a small part is a jean jacket. It’s a good match for the 20 pixels that are a jean jacket. What we can do with combining text and visual search. Text is a clear signal, they want an outfit, that really narrows it.”
The update comes amid an array of other camera-related announcements today, like the introduction of QR codes for Pinterest. Pinterest today is also adding the ability to immediately begin a visual search when users zoom into a part of a pin — like the bag in a photo of an outfit — and go straight to finding other products that are like it.
It’s all part of a move for Pinterest to begin moving the behavior of searching for products or ideas on Pinterest to the real world and capturing that in-the-moment interest that someone has in an idea. And starting off the day trying to decide what you want to wear and just pointing your camera at a jean jacket is a natural example of trying to translate more seamlessly an idea that a user might not be able to naturally articulate in a search.
It also represents a potential boon for advertisers, which may have the opportunity to snag users in a moment where they are already engaged with an idea or product and have shown clear signs of interest. As Pinterest moves a lot of the search behavior to the camera and more visually oriented searches, it can do a better job of capturing the kind of intent-level behavior that people searching on Google might have — but instead of typing in a search, they can point their camera at something.
Pinterest has tried to sell itself to advertisers as a product that can help them trace the behavior of a user from discovering a product or idea all the way down to acting on it, and grabbing them in a moment of intent fills another hole that it might not have. The company is also adding more than 5 million additional shoppable products through its Shop the Look feature, which shows up as a small circle within a pin that takes users to potential products. All this is a way to help get the right products in front of users and get them used to the idea of engaging with objects in the real world on Pinterest.
“The text figures out the candidates and then the vision finds the best match,” Harris said. “It’s a good fit for Pinterest because so many search queries are big open-ended head queries where they can go a whole lot of different directions. You give us that image and it lets you go way more granular. We think that Pinterest has the potential to be the world’s best stylists. You’re combining it with the actual computer vision technology to understand the clothing people have. “