Facebook revealed the final piece in its scalable data redesign today, one that completes its vision of redefining traditional hardware and software to make it more flexible to meet the needs of Facebook’s scale.
To that end, they announced a new open-source modular switch platform called 6-pack.
The key piece to understand is that Facebook has focused on separating the hardware and software, making each part of the system fully programmable, thereby giving Facebook the flexibility it requires.
Facebook plans to contribute the entire design to the Open Compute project, where engineers will be free to build, configure and design these components as they see fit.
Last June, Facebook began this journey by redesigning the top of rack switch ( the so-called Wedge), and developing a version of Linux called FBoss to run it. This was the first step in separating the hardware from the software, which was key to building the more flexible network required for computing at Facebook scale.
This enabled Facebook to treat the switch like it was a programmable server because it had its own operating system and could operate independently. This was essential because they needed to separate the hardware and software from the network as a whole and allow engineers to deal with each piece as a separate programmable entity.
But they couldn’t stop there. As they built the data center of the future, they also needed to completely rethink network topology, and how information moves in the network. This required creating an entirely new data center fabric that was flexible enough to deal with Facebook’s massive computing requirements without getting bogged down.
As Frederic Lardinois wrote of the new Altoona data center last fall, “With Facebook’s new approach, however, the entire data center runs on a single high-performance network. There are no [traditional] clusters, just server pods that are all connected to each other.”
This led to the final requirement: how to deal with modular switches that weren’t situated at the top of the rack, but within the fabric itself. Today’s announcement solves that final problem by introducing a monster fully open modular switch based on Wedge, that like the top of rack switch, has separated the hardware and software from the network, and given Facebook total flexibility on how to program and run it.
Facebook describes it as follows:
The “6-pack” platform is the core of our new fabric, and it uses “Wedge” as its basic building block. It is a full mesh non-blocking two-stage switch that includes 12 independent switching elements. Each independent element can switch 1.28Tbps. We have two configurations: One configuration exposes 16x40GE ports to the front and 640G (16x40GE) to the back, and the other is used for aggregation and exposes all 1.28T to the back. Each element runs its own operating system on the local server and is completely independent, from the switching aspects to the low-level board control and cooling system.
You may wonder why Facebook went to all this trouble, but it believed it needed to build the entire network from scratch because in its view, commercial offerings were just too limiting for the speed and scale a company like Facebook must have.
“One of the things we saw in the network industry was that it wasn’t open enough,” Matt Corddry, director of hardware engineering at Facebook explained. “It’s difficult to expand and grow our technology with commercial networking equipment. We didn’t have enough visibility and control. A lot of the commercial approaches, they are closed, with software and hardware combined,” he said.
The only option was to redesign the data center from the bottom up, which is what they effectively have done, and this gives them the speed and visibility inside the equipment they need to develop tools much more quickly.
As for putting all these resources on this, then open sourcing it, Corddry says it’s simply part of Facebook’s vision for scalable computing.
“Fundamentally we see part of our mission is to make the world more open and connected. There is no secret mission or hidden agenda, it’s just part of our mission,” he said.