Facebook Completes Rollout Of Haystack To Stem Losses From Massive Photo Uploads

One nugget of information Facebook leaked out to press last week during the Gideon Yu fiasco: the company has been EBITDA profitable for five quarters, but doesn’t expect to generate positive cash flow until 2010. Why the discrepancy? There’s only one answer to that – Facebook is paying out big dollars for something that must be depreciated over time. If they could just write off the expense in full as they paid it they’d be having much bigger losses now that matched cash flow, and they’d hit profitability sooner. But accounting rules let them pay cash now and recognize the expense later on. In early leaked projected financials, there was a $200 million difference in 2008 cash flow and EBITDA profitability (or lack thereof).

What have they been buying? Stuff to serve up all these massive page views, and photos in particular. In our post last October, when Facebook was fishing for dollars in Dubai, we noted some of their expenses, including a massive ongoing outlay for NetApp storage systems that cost $2 million each:

The company is likely spending well over a $1 million per month on electricity alone, say experts we’ve spoken with. Bandwidth is likely another $500,000 or more per month on top of that. The company has earmarked $100 million to buy 50,000 servers this year and next. And sources say they’ve been buying one NetApp 3070 storage system per week just to keep up with all this user generated content. At up to $2 million each, that adds up quickly – we’ve heard estimates that they may have spent as much as $30 million this year alone with the company. And the icing on the cake – earmark another $15 million per year in office and datacenter rent payments.

As we noted in February, Facebook is the largest photo application on the web (forgetting everything else they do). More than 850 million photos uploaded to the site each month, and these things chew up bandwidth and storage like crazy. And it’s even more expensive to serve photos in poorer countries where Facebook is getting all its growth (and little revenue).

Enter Haystack

Haystack is Facebook’s way of substantially lowering the cost of storing and serving photos, and the rollout of the new internal infrastructure was recently completed. See Niall Kennedy for a technical overview of what Haystack is and why it’s so much more efficient than third party solutions they’ve used to date, as well as this 2008 presentation by Jason Sobel.

What isn’t clear is if Haystack will really help Facebook control costs outside of the U.S., particularly in Asia. But it’s a step in the right direction for cost control, and is certainly being factored in Facebook’s estimates of cash flow profitability by next year.