At NSA, The Cloud Is About Big Data And Moving Beyond IT

The National Security Agency (NSA) cloud is about big data and creating unicorns. And it all started when some geeks stole two servers. It makes no sense, according to conventional thinking, but these are unconventional times, and the cloud that NSA built had to be thought through differently, too.

NSA’s goal is to unify data and use it to do analysis, said Nathanael Burton, a computer scientist with the security agency in a keynote address today at the OpenStack Summit in Portland. But with its old infrastructure, the data was spread across different systems that did not work together.

Today, OpenStack is running across the NSA and has drastically changed the way the agency works with data. So much so that Burton said it is now becoming an advisor across the international intelligence community.

The NSA is a government agency, and it can take weeks — even months — to get an idea approved and resources committed. This was squelching innovation or, in Burton’s words, making people wonder, “why bother?”

Burton had heard about OpenStack and decided to attend its February 2011 conference in Santa Clara. He came back impassioned and decided to steal two servers instead of going through the arduous process of getting IT approval. (Of course, this was in a lab environment so “stealing” two servers is used loosely.) In two weeks they had a pilot of OpenStack up and running on the Cactus release.

“We started to see our first unicorns,” Burton said.

Burton said they integrated Puppet and other tools and now can do installations in 20 minutes. The system has been hardened at all layers of the infrastructure. He said “SSL is everywhere.”

They created a free tier for people to test it. People started to adopt it. Its applications became apparent. Suddenly there were “lots of unicorns and rainbows.”

OpenStack is now in production. They did a silent launch. It has auto-account creation and it’s self-service. The installation pretty much has helped NSA move beyond the complications of going through IT.

“Self-provisioning is good because most people at the NSA are introverts,” he said.

The system, like many scaled-out infrastructures, is designed for failure, so it will always keep running. A node may fail but the distributed environment keeps running, moving the workload elsewhere.

Burton’s keynote illustrated how viable and secure OpenStack is. If the agency charged with protecting U.S. security systems uses it in production, then it has to be good enough for any enterprise environment, right?