Facebook is planning to open source LogDevice, the company’s custom-built solution for storing logs collected from distributed data centers. The company made the announcement as part of its Scale conference.
Logs are used to track database events. If a server suffers an outage for any reason, companies need a way to debug, perform security audits and ensure consistency between servers. This is particularly important to Facebook, which holds immense amounts of your content across its massive data centers around the world.
LogDevice is capable of recording data regardless of hardware or network issues. If something breaks, it will simply hand-off the task of collecting logs. And when everything turns back on, LogDevice can restore records at between five and 10 gigabytes per second.
If you’re Facebook, and will soon have 10 data centers, you need a system of record to ensure each center is on the same page. And things get extra complicated when you consider the complexity of backups the company needs to do with its data. LogDevice helps when you need to replicate data from these separate data centers.
If you get frustrated and regretfully throw an expensive server across the room, LogDevice will report exactly which records were lost. By separating record sequencing and storage, and randomly assigning records to different storage locations, it enhances resiliency of the entire data center.
Facebook didn’t give an exact date for when it expects to open source LogDevice, but it says it will occur later in 2017.