Capital One open sources Cloud Custodian AWS resource management tool

Capital One is a huge organization with lots of compliance issues related to being a financial services company. It also happens to be an Amazon Web Services customer and it needed a tool to set rules and policies in an efficient way around AWS usage.

Last July it started developing the tool that would become Cloud Custodian; today it announced at an AWS event in Chicago that it was making that tool available as open source on GitHub.

“Cloud Custodian is a rules engine that lets us define policies to be well-managed in AWS. You can [determine] a number infrastructure resources and every organization has a set of policies to be achieved around those resources,” Kapil Thangavelu, technical fellow and primary developer of the Cloud Custodian project told TechCrunch.

This ability to define policies in an organized manner resulted in a 25 percent reduction in use of AWS resources, which of course translates into big dollar savings for a company the size of Capital One. Prior to developing Cloud Custodian, the company would create scripts for each individual requirement with no central oversight. Cloud Custodian gives Capital One a central place to create, monitor and manage policies it lacked before or that required multiple tools.

Cloud Custodian dashboard from Capital One.

Cloud Custodian dashboard. Photo Credit: Capital One

The tool is possible because of a couple of newer services offered by AWS. First of all it takes advantage of the CloudWatch Events (CWE), which was released in January and provides a way to monitor events in a much more efficient manner than was possible before this service. Instead of constantly polling the API whether there was an action or not, with CWE you get real-time notifications only when there is an event of interest to you.

The other breakthrough was the Lambda service announced last year at AWS re:invent, which based on the event trigger in CWE can launch a set of resources for a given set of rules for a set period of time whenever it matches a rule. By combining  CloudWatch Events and Lambda, Capital One was able to create a super-efficient rules engine that became Cloud Custodian.

Instead of maintaining servers for an infrequent event, the company can pinpoint the exact resources it needs, only when it needs them for a given event. The beauty is that they go away when the event is over. When you’re not running a server constantly, whether you need it or not, that’s going to reduce your resource overhead pretty significantly right there.

And giving administrators more visibility into the rules gives more visibility and control over the entire AWS cloud infrastructure, ensuring that every instance is in compliance and that you’re not paying for resources you aren’t using.

The company decided to open source Cloud Custodian because they take advantage of open-source software and they felt like it was the right thing to do give back. Secondly, and more pragmatically, perhaps, by open sourcing the tool they now have a community of people working on it instead of just them, which can improve it and take the maintenance burden off of them.

Capital One hopes Cloud Custodian takes off as an open-source project and develops a loyal following. The announcement today is the first step as it tries to build a community around the tool.