YouTube’s Director of Engineering Explains How 60% Of Videos Are Processed In Under A Minute

You’d be hard pressed to find someone who doesn’t think of YouTube as an Internet giant. After all, each viral hit is emblazoned with a big, bold counter racking up the millions — or tens of millions — of hits it’s received, and you could waste lifetimes clicking on an endless stream of cat videos if you wanted to.

But even then, it’s hard not to be taken aback by some of the statistics the site drops from time to time. Like the fact that it’s technically the Internet’s second biggest search engine, after Google proper. Or a new figure that it’s announcing for the first time: YouTube users are now uploading two hours of video every minute from mobile devices alone. And it gets over 200 million views a day from mobile devices.

A reasonable response to stats like these would be, “How?”

I had a chance to speak with YouTube Director of Engineering Christian Kaiser, who recently left his job as VP of Engineering at Netflix to take the helm at Google’s video giant. Kaiser, who joined YouTube in April, couldn’t get into too many specifics about what goes on behind the scenes. But he did have some broader things to share about YouTube’s approach to dealing with the huge volume of incoming data, and the challenges that lie ahead.

Given YouTube’s massive scale it isn’t a surprise that the company has plenty of custom tools running in the backend to transcode video as quickly as possible (this transcoding system is called Viper). Kaiser says that one key speed improvement that was recently introduced has to do with the order in which videos are processed — the steps used to happen in sequence and are now being run in parallel, which has led to a “huge improvement’ in the time in takes to get a video live.

How big? Today, 60% of all videos go live in under one minute — a year ago, no videos were being processed that quickly.

As for the challenges the team is currently focused on, Kaiser says that the need for quick and seamless playback — which is obviously always a key issue — is greater than ever, especially when it comes to delivering content to televisions. The team is currently implementing adaptive streaming, which changes the bitrate of a video on the fly depending on your connection speed (videos on Hulu, Ooyala, and some other sites already do this). When I asked if this would be coming to HTML5 any time soon (which is the direction YouTube is slowly heading toward), Kaiser said that it’s not implemented anywhere right now, but that it’s working to make it a reality.

Kaiser also discussed one ‘trick’ YouTube uses to quickly serve all those videos around the world. The company has built and deployed hundreds hardware devices worldwide that cache and serve up the most popular YouTube videos, in much the same way that a CDN does.

To close out our brief interview, I asked Kaiser if he thought we still had yet to reach the ‘turning point’, when people are going to start using online services rather than cable as their primary way to consume video content. Kaiser says that we’re already seeing a little bit of this — especially with services like Hulu and Netflix taking off. But he’s not sure there’s going to be a steep inflection point in the switch from cable to IP. Instead, he thinks it may be more gradual (and is already happening).