Facebook has acquired its third company, Malaysian startup Octazen Solutions. Facebook says this is largely a talent acquisition, according to GigaOm. Octazen has a slightly different story on their home page, saying Facebook acquired “most of the company’s assets and to employ those assets in a different direction.”
Either way, it’s leaving some people scratching their heads. Said one senior engineer at a competing company that we spoke to this evening, “Facebook just bought the web’s most talented and creative scrapers that have gotten around everyones rate limits and detection systems.” Said another person we spoke with this evening who is knowledgeable of Octazen’s product, “Facebook is so sanctimonious about protecting their own user data through Facebook Connect, but Octazen has been scraping user data for years off terms of service and then reselling it.” Both sources asked to remain anonymous.
Facebook, for their part, have not yet responded to our request for comment.
What exactly has Octazen been up to? The company is mostly about above-board contact importing from one service to another – signing in to Gmail from Facebook, for example, to import your contacts there and add them as Facebook friends. Much of this is done via OAuth and APIs, but Octazen is known to dive much deeper for data.
One example – Octazen will sometimes collect and store user credentials directly, and sign into large social networks and other sites as if they were the user, say multiple sources. Then they’ll download the address book and social graph. A percentage of your friends on that service might be users of the service (now Facebook) paying Octazen, and you’ll be asked to friend them. But there’s a big question about what happens to the rest of the data as well, and if Octazen is storing a shadow social network in violation of terms of service to recommend user connections down the road. And they may look deeper at data than they should – at email header information, for example, to get a better understanding of who you communicate with the most.
But the most unnerving part of Octazen, say our sources, is the fact that they are very, very good at scraping data at scale without being detected. They may hit a service using lots of different IP addresses, for example, and remain undetected. Octazen could, they say, scrape very public sites like Twitter, where the social graph is on each profile, in a way that Twitter wouldn’t know it’s happening.
In 2007, for example, People were buying and running Octazen scripts to scrape contacts in a very sketchy way: “So we use this toolkit from Octazen to scrape contact lists off of various sites. Our ever eager users (ab)used this feature so much that hotmail blocked us.” The poster found a way to access Hotmail’s API instead of just scraping to get the data, and Octazen responded, saying “Very nice indeed”
Our understanding is that Facebook already uses Octazen to mysteriously determine your long lost friends and suggest that you re-connect with them (leading to scores of emails into our inbox that Facebook is somehow reading emails or otherwise getting data they shouldn’t be).
The big question is why Facebook would need to acquire a company located half way around the world if all they were doing is standard address book imports via OAuth and APIs, or proprietary but well documented protocols like Facebook uses. The implication is that these guys have serious expertise in data gathering at scale that may sometimes be in violation of the terms of service of the sites being harvested.
This is obviously just one side of the possible story, albeit based on hard evidence of Octazen’s shady prior practices and via multiple sources. But until Facebook explains this acquisition in more detail, we don’t have much more to go on.
Update: From Larry Yu at Facebook:
As background, more than 30% of our users currently rely on an email provider other than Hotmail, Yahoo!, AOL or Gmail as their way of authenticating into Facebook, finding friends that are on the site and inviting new friends to join.
This percentage will only increase as we seek to grow around the world. So we brought on the Octazen engineers because they are very good at building contact importers for these alternate services which comprise the “long tail.” This is important because a new user can then ramp up quickly since they can find and invite friends. We intend to have them focus on building these “long-tail” importers and improve our existing solutions.
It is not true that Octazen would collect and store user credentials directly to facilitate contact importing. The majority of their clients installed the software on their own servers, so Octazen would not have access to any user data in those instances.
There was an on-demand service where the contact importing process would run though Octazen’s machines. But for this on-demand service, they only stored the username and a 1-way hash of the user password. This stored data can not be used to log in on behalf of the user. It was only used for audit purposes so they could bill the clients based on successful imports and not attempts.