CoreOS launches Torus, a new open source distributed storage system

CoreOS today announced the launch of Torus, its latest open source project. Just like CoreOS’s other projects, Torus is all about giving startups and enterprises access to the same kind of technologies that web-scale companies like Google already use internally. In the case of Torus, that’s distributed storage.

The idea behind Torus is to give developers access to a reliable and scalable storage system for applications that have been deployed on containers using the Google-incubated Kubernetes container management service.

“Persistent storage in container cluster infrastructure is one of the most interesting current problems in computing,” CoreOS’s Barak Michener writes in today’s announcement. “Where do we store the voluminous stream of data that microservices produce and consume, especially when immutable, discretely contained executable code is such a powerful pattern?”

The CoreOS team argues that existing storage solutions weren’t designed to work with container clusters. They were meant for small clusters of large machines while the modern approach focused on more large clusters that run relatively small machines. And while container deployments are also about being able to quickly start up and shut down containers as needed, most developers want a persistent storage system that can then feed the applications that run on these containers. “Ensuring persistent storage for these container microservices as they are started, stopped, upgraded, and migrated between nodes in the cluster is not as simple as providing a backing store for a single server running a group of monolithic applications, or even a number of virtual machines,” Michener writes.

Torus uses a key-value database to store and retrieve files. CoreOS argues that the system can scale up to hundreds of nodes. For now, this early version of Torus exposes files as block-oriented storage through a Network Block Device. Because the system is meant to be extensible, CoreOS expects that others will also build the necessary tools to support object storage systems on top of Torus.

Torus joins CoreOS’s other open source projects like its eponymous Linux distribution, the rkt container engine and flannel networking tools. These — together with numerous other tools — power the company’s commercial offerings like the Tectonic container management system and Quay service for building, storing and distributing software containers.