The Matrix Of Hell And Two Open-Source Projects For The Emerging Agnostic Cloud

Docker, an app container service from the co-founder at DotCloud, and Salt, an open DevOps platform from the founder of SaltStack, were mentioned this past week at OSCON as two of the most exciting new open-source efforts.

Complexity comes with the cloud and its fit with enterprise data centers. The Docker team calls this new world of services and devices the matrix of hell. The Salt folks see salvation in speed — perhaps to save us all from the hell that comes with heavyweight systems that require extensive resources and are slow due to being built when distributed systems were not as common as they are today.

Both projects are tied to the deeper complexity that comes now with what new DotCloud CEO Ben Golub and Co-Founder Solomon Hykes describe as a world that resembles a matrix, represented by rows of endless number of available services and columns that represent any number of devices where applications run. DotCloud supports the Docker open-source project.

Their emergence also represents the new reality about what can be described as the “agnostic cloud.” Sure, there’s a belief structure about cloud but there is no almighty allegiance to its power. Instead, there is an agnostic movement to make on-premise and cloud services accessible through a universe of providers and open-source services that run anywhere — be it a private data center or a public cloud service.

Docker

Docker automates the deployment of apps as a lightweight Linux container. The container can be built and tested on a laptop and synced to run anywhere. It can run on virtual machines, bare-metal servers, OpenStack clusters, public instances or any combination of on-premise and cloud offerings.

Docker does not port the virtual machine nor the operating system, which makes sense when considering that the infrastructure itself is becoming the operating system. The compute, storage and networking is already in place on a cloud service — the application just goes there to run.

The service avoids the issue that comes with moving virtual machines, which are not designed to move between clouds. So instead of moving the VM, Docker moves the code between the VMs. Most of the security is managed by the Linux kernel.

Hykes said in an interview last week that developers particularly like the capabilities to continually test and integrate app containers. This makes for simpler and faster methods for building applications that can run anywhere. For example, developers are using Docker to build next-generation platform as a service (PaaS) offerings. It’s a noteworthy development. Most PaaS providers have historically provided monolithic platforms to do as much as possible. With Docker, platforms can be built that leverage the services of different providers to create lightweight environments for building and delivering apps.

For more technical descriptions about Docker, there are some good resources here, here and here.

Salt

Salt is a new open DevOps platform built for speed. It is designed to use generic high-speed communication to move data out to nodes by doing parallel data processing. Generic commands get sent to the nodes with feedback coming back very quickly. Harvard University used it for their supercomputer clusters. Jobs that once took 15 minutes now take five seconds.

According to the SaltStack website, Salt can be scaled to tens of thousands of servers through a communications bus that orchestrates, does remote execution and configuration management as well as other tasks.

Salt is being used as a replacement for Chef and Puppet, the two leading DevOps platforms. It is now used by LinkedIn and Rackspace. Here’s an excerpt from a good analysis by Sebastian Kreutzberger, CEO of RhodeCode, an open source software configuration and management platform for Git and Mercurial:

Salt is like a mix of Chef/Puppet (defining states) and an easy way to communicate with machines directly (like with an MQ). The big difference to Chef is the architecture: the slave (called minion) does not pull for changes every bunch of minutes, which can cause weirdness, but has a standing connection to the master which allows instant changes and commands.

Noted often about Salt is its documentation, which has helped the community further develop the platform. Here’s an introduction to Salt by its creator Thomas Hatch:

Conclusion

The cloud and on-premise systems are starting to merge into one cohesive universe. OpenStack serves as a way to make data-center environments more elastic. Cloud services like Amazon Web Services represent the public cloud infrastructure. The PaaS providers are becoming environments for serving apps to these different infrastructures. These agnostic providers, such as Cloud Foundry, do not serve one cloud. They help developers serve multiple cloud environments.

The same is true for services like CloudMunch, which offers a continuous integration platform that can move code between different cloud services. CloudMunch Founder Pradeep Prabhu said this new universal world has three main characteristics:

  • There must be the choice to use any developer or operations tools with any PaaS for any IaaS/cloud or on-premise/private cloud.
  • It has to be workload centric. Whatever makes best sense for a given workload including tooling, patterns and practices and infrastructure/cloud for delivering the best results/roi for that workload.
  • It is the ability to define a customizable software delivery progression with all the checks and balances for both application code and infrastructure code with no lock-in to any tool, methodology or cloud.

Similar principles apply to Docker, which treats the app container as the way to deliver apps to the cloud or any other infrastructure. Salt also fits into this universal mentality.

The new world is not about universal control and beliefs in all-mighty systems. Open-source efforts like Docker and Salt are popular because they fit into this more flexible and agnostic view of the cloud and data center universe.

Image credit: Wikipedia