Playdom’s Gardens of Time took the #1 spot of Facebook’s Most Popular Games of 2011 list. But wait, Zynga’s CityVille was ranked #3 despite peaking at over 100 million monthly active users and 21 million daily active users — far more than Gardens of Time’s peaks of 17 million MAU and 4 million DAU, or its current stats according to AppData. That’s because Facebook’s list was mostly based on vaguely described “recommendations”, and hardly on active users.
It’s almost as if Facebook used its cloudy methodology to keep Zynga from completely dominating the list, as the Mark Pincus machine currently owns all 5 Facebook games with the most DAU and still ended up with 4 of the top 10 spots on Facebook’s list.
[Update 9:25pm PST 12/21/11: Facebook has integrated the feedback from this article and updated its Top Games post to more clearly explain what it means by 'Most Popular'. The methodology note now reads: "This list was compiled by looking at the top games on Facebook with more than 100,000 monthly active users and giving priority to those games with the highest user satisfaction scores. The result is a list of the games that received the most user recommendations in 2011." As long as it's transparent, we're happy to see Facebook finding more ways to feature high quality games, and not just those with the most users.]
A Facebook PR representative denied my theory. She told me that in the methodology, “the top recommended apps (from surveys presented to users in the canvas ticker where they can answer yes/no to, ‘would you recommend this to a friend’?”)…were ranked higher over highest monthly active user numbers. Another way to say it is that these apps were the highest rated among users.”
But why not just say in the post that this was based on ratings, not on active users? The list should have been labeled “Most satisfying games” or “Most recommended games”, not “Most popular games”. Missing the top spot probably won’t help Zynga’s stock, which IPO’d last week. Zynga and Facebook have partnered for years to help Zynga attain traffic goals. However, that relationship may be strained as the gaming company plans to serve games directly to users outside of the Facebook platform.
Facebook could have predicted that people would ask why the #3 game peaked at 80 million more monthly users than the #1 game. If CityVille had a 20% ‘Yes’ rating average and Gardens of Time had an 80% ‘Yes’ average, then fine, the rankings make sense. But if the ratings were even somewhat close, it seems a little fishy that CityVille didn’t take the #1 spot, and Facebook didn’t disclose the actual rating scores or even that they were the core of the methodology.
To really win big on the Facebook platform, you need a substantial marketing budget, existing games to cross-promote from, and sophisticated A/B testing technology. Zynga has all of these and most developers don’t. #1 Playdom and #2 EA (The Sims Social, with peak MAU of 66M MAU and 11M DAU) both do too. Still, Zynga taking the top spot would have really driven home the point that the Facebook gaming business is run by the big boys.
That’s a problem, because attracting the long tail of small and medium developers is essential to retaining gamers who are always looking for something new to play. Facebook went out of its way to feature smaller and non-Silicon Valley developers. It included categories such as Top up-and-comers (Battle Pirates by Kixeye) and most popular games with 50,000-100,000 users (Super Slot Machines). It also cited top sports app TopEleven as having a team of just 30 out of Serbia in an email to us, and noted other develpers from Germany, Japan, and Prague.
There’s nothing wrong with Facebook promoting developer diversity, but it needs to be up front about how this list was calculated. It’s in Facebook’s interest to avoid making it seem like the platform is winner-take-all or like it depends on Zynga. Through obscured methodology, Facebook didn’t have to highlight this by giving CityVille the top spot.
Update 9:23pm PST 12/21/11: Facebook has edited its Top Games post to integrate the feedback from this article. The more transparent explanation of the Most Popular list’s methodology now reads: “This list was compiled by looking at the top games on Facebook with more than 100,000 monthly active users and giving priority to those games with the highest user satisfaction scores. The result is a list of the games that received the most user recommendations in 2011.”