Massachusetts Open Cloud Project Hopes To Create Ad-Hoc Infrastructure Marketplace

Today, the cloud infrastructure market is dominated by several big companies — Amazon, Google and Microsoft — but a public/business/academia partnership called the Massachusetts Open Cloud project is hoping to change that by creating an open computing marketplace where you can negotiate whatever services you need from multiple infrastructure vendors.

Peter Desnoyers, a professor at Northeastern University who helped launch the project, explained that while companies like Amazon offer useful services, they have limitations.

First of all, from an academic perspective, they have a closed system. That means their internal team has access to the system for research purposes, but anyone outside the company like academics who want to study the system and present papers are shut out. While they can go to company conferences and hear employees present papers, they can’t get deep inside the system and that’s a real problem for him and his fellow academics.

The other is that Amazon and other IaaS vendors offer what he calls the “Henry Ford” approach to IaaS. You can have any color you want as long it’s black. In other words, they have certain products they have packaged together. The trouble with this approach though, Desnoyers explained, is that people often have very specialized requirements, and the way Amazon designs its products shuts those people out or makes it prohibitively expensive if they need specialized services.

Desnoyers says that the project hopes to create a marketplace where multiple vendors can come together and offer their services in an ad-hoc kind of way, so you might get your compute power from one vendor, your storage from a second and your memory from a third. The vendors seem like to this approach and include industry heavyweights Cisco, Juniper, Intel, Red Hat and others.

The colleges involved include Harvard, MIT, UMass Amherst, Boston University and Northeastern.

The Commonwealth of Massachusetts is also involved and the project will be housed at the Massachusetts Green High Performance Computing Center in Holyoke, Mass.

Vendors will contribute equipment and engineering talent and the goal of the project is to create a commercial project based on open source tools.

One vendor involved with the MOC project is Red Hat, and Jan Mark Holtzer, who is senior consulting engineer for the CTO office at Red Hat says his company can learn a lot from a project like this.

“For us I would see the key opportunities we see around MOC is operational access, understanding large scale cloud infrastructure, and growing skills [around these areas]. We will rotate resources from support and consulting organizations so they can get first hand experience.”

Holtzer says the initial use case for the project probably involves getting vast computing resources for a short period of time to meet a specific need. “Clearly currently the initial use case we see and MOC sees is probably driven by [high performance computing] and MOC would give customers the capability of harvesting large amount of resources and then releasing them quickly,” he said.

He says, however, before it becomes a viable commercial entity for vendors like Red Hat, he sees potential as an incubation space for innovation where participants can experiment with different business models and Service Level Agreements (SLAs).

But perhaps the biggest advantage of being involved in a project like this from a vendor perspective is very similar to the academic one. They can get real data about how large-scale systems like this work. “Probably the very interesting use case is the ability to get the operational data from such a large scale environment. A lot of cloud services are black boxes. We work with these vendors, but we don’t have the ability to get as much information from inside a large scale infrastructure,” he said.

Holtzer added that there is a huge advantage in making the MOC project operational data transparent and visible.

The fact is there are lots of cloud infrastructure options available out there, but no open marketplace where people can negotiate pricing and access different pieces of the infrastructure. A project like this is at least a starting point for offering a more open way of selling infrastructure services moving forward.

For now it’s experimental, but if it works, it has the potential to change the way enterprise customers interact with and deal with IaaS vendors and that’s significant in itself.