Five weeks after launch, Facebook gave reporters a ‘State Of Graph Search’ today at its Menlo Park HQ. It’s been rolled out from the original 100,000 users to “hundreds of thousands,” and now a news feed story is appearing to lure people’s friends to sign up for the early access wait list.
This morning I noticed Facebook was admitting a slew of wait-listed people to Graph Search. That’s because now when new people gain access, Facebook automatically publishes a news feed story, e.g. “Eric Eldon just got Graph Search.” Click through those last words, and people can sign up for earlier access, too. Facebook tells me this is a viral driver meant to increase waitlist sign-ups. The story frames Graph Search as the cool new feature you should be willing to beg to get access to.
Beyond user counts and the roll out, today’s “Graph Search Whiteboard Session” saw some of Facebook’s lead engineers delving deep into how people are using its new internal search engine, and how it works on the back end.
Graph Search Director Tom Stocky noted that early usage patterns show people using Graph Search for the same things they use the social network for: looking at friends and photos. “But Places is third” said Stocky, an encouraging sign for local businesses. Interests aren’t getting as many queries.
One thing that surprised Facebook was the ways people search for their friends. Rather than asking for something like “photos of my ‘friends'”, users frequently typed in terms like ‘chums’, ‘besties’, ‘buddies’, ‘homies’, and ‘peeps’. At first those stumped Graph Search, but a dedicated natural language team has now built those in so they’re recognized as synonyms.
Next, Facebook is working to scale up Graph Search — both the back-end system (called Unicorn) so it can support more users and the roll-out can continue, and in terms of foreign languages so it can expand beyond English. However, that second part may require it to hire linguists in each language to identify sentence structure and synonyms, or figure out a way to crowdsource this process the same way it did to translate the whole site.
It’s also trying to work out syntax problems so people can run “Or” queries. Right now you can ask for “friends who Live in New York who Like Game Of Thrones and Downton Abbey,” but not “Friends who live in New York who Like U2 or The Rolling Stones” which could help you quickly find someone to go to a rock concert with.
But where I see the real potential of Graph Search is intelligent sorting of results in a way only Facebook could do. For example, if you search for “people who work at TechCrunch” you get all our real employees, but lots of people who just list TechCrunch as their employer as a joke or to look like a techie. Luckily, Graph Search is smart enough to identify which people who “work” at TechCrunch are friends with each other. Since groups of real employees are often friends, it knows which are more likely to be real and to show at the top of the results. Facebook also looks at which people have listed a verified @techcrunch.com email address in their profile as another sign of legitimacy.
Term Frequency-Inverse Document Frequency (TF-IDF), an old method put to new use by Facebook, is helping sort Graph Search results. Ask Facebook Graph Search for “books liked by founders” and you won’t see The Da Vinci Code or The Bible first, though they’re among the most Liked books. Instead you’ll see books disproportionately Liked by founders compared to the general population thanks to TI-IDF. Surfacing The Tipping Point, and Design For Hackers shows how smart Graph Search is.
Facebook’s Mike Curtiss explained TF-IDF to me using the example of “TV shows Liked by my friends.” He says “There’s a tough balance with TF-IDF. If you don’t use it you’d just be promoting the most popular TV shows. But if you can bias it towards the preference of these individuals and not what’s objectively the most popular, you get different results. It’s nice to use that when you can. It’s a good signal. You can show the things with the most Likes, or you can show the things that are very unique to people [you’re asking about], but often you want a balance of those to provide the most interesting results.”
Facebook is trying to leverage its social data to essentially create a custom search engine for every user. Then by sorting results further with TF-IDF, it could build something not just different from Google, but potentially better where they overlap as well.
For more on how Graph Search works straight from the horse’s mouth, watch my interview with former Google employees and the current Facebook Directors of Graph Search, Tom Stocky and Lars Rasmussen.