By David Nalley, Apache CloudStack Committer
The cloud API wars have continued unabated in 2013, with many flare ups. Apache CloudStack hasn’t been a huge partisan in these wars; but we still consider APIs very important.
What is this ‘Apache CloudStack you speak of?
You haven’t heard about Apache CloudStack? Perhaps it’s not surprising then that its often been called the ‘best kept secret in cloud computing’. Apache CloudStack is both project and product; it’s a top level project at the inimitable Apache Software Foundation with a purpose of producing software for deploying public and private Infrastructure-as-a-Service clouds. This means you can pool storage, network, and compute resources and allow users to provision and consume resources on-demand; while at the same time ensuring isolation between users’ resources. Despite not having a huge hype following there are a number of prolific deployments that have been publicly acknowledged.
What are APIs so important to Apache CloudStack or any other IaaS platform? APIs form the core communication method between consumers and the platform itself. While virtually every IaaS platform has some type of nice web GUI to interact with, it’s just not efficient. Moreover, such UIs typically are designed for humans. What if your monitoring system or continuous integration system needs to interact with your IaaS platform? In short, APIs are how work actually gets done in a IaaS cloud platform.
Apache CloudStack GUI: Pretty, but not where the real work gets done!
APIs are great, right, so no problems? Well, except that there are a plethora of APIs out there; and while automation and your own sanity demand that you have an use an API, which one is best? Does it matter?
It’s important to realize that when most people talk about cloud APIs they generally are referring to user APIs. E.g. the APIs users make use of to provision, deprovision, or otherwise interact with the cloud platform. There is a separate (sometimes hidden) administrative API that the cloud provider uses for controlling the cloud itself.
It’s important to realize that when most people talk about cloud APIs they generally are referring to user APIs. …There is a separate (sometimes hidden) administrative API that the cloud provider uses for controlling the cloud itself.
The Standards
Naturally most folks want to use some ‘standard’. The IT industry certainly has a glut of standardization bodies, so finding and adopting a cloud API standard should be a no brainer, right? Sadly it’s a bit more complicated than that. There are a couple of cloud API standards; most notably CIMI (Cloud Infrastructure Management Interface) from the folks at DMTF and OCCI (Open Cloud Computing Interface) from the Open Grid Forum.
There are really two problems with these standards. The first problem is that neither has massive adoption. Which in turn means that the number of libraries and third-party tools that support these APIs is pretty small. In many ways this defeats the purpose of using a single, widely adopted, standard.
The second issue is that because of their wide applicability the APIs are pretty generic. Turning on or off a virtual machine is easy, but if you want to use the latest coolest feature of a specific IaaS platform it might not be exposed at all by CIMI or OCCI. In some ways this is a good thing; it enforces adherence to core functionality that should work on virtually any platform. This means you mitigate some of the risk of lock-in to a specific platform. On the other hand it might mean artificially limiting ourselves from cool, innovative technology.
In some ways this is a good thing; it enforces adherence to core functionality that should work on virtually any platform. […] On the other hand it might mean artificially limiting ourselves from cool, innovative technology.
The de facto standard
As in most emerging technology, there are de facto standards in the cloud API arena. In the IaaS realm that is the Amazon Web Services API. Virtually every cloud-y tool or library will at least work with the AWS API. The corollary to this is that virtually every IaaS has some measure of AWSAPI compatibility.
Perhaps that makes it the real standard. It certainly has the adoption and ecosystem around it. But its sadly no panacea and does have a few difficulties. Unlike CIMI and OCCI which are designed to be more generic by nature, the AWSAPI is really designed to be implementation specific – to Amazon’s implementation. The challenge here is that in addition to ingesting requests and parroting back the proper commands, there are plenty of assumptions around how the cloud is implemented; and those underlying semantics mean you must either mimic them or you have some potential problems with your fidelity to the API. That means you have a lot of technology limitations; and for better or worse are set to track Amazon.
What’s the Apache CloudStack take on all of the APIs?
In short, the project recognises that there are valid reasons to use a real or de facto standard and we see the advantages and limitations that come with that. We encourage the development of those ‘standards integrations’ even as we continue forward development of our native APIs.
Apache CloudStack tends to be agnostic; we don’t have a preferred hypervisor, or a preferred storage technology, or networking topology; and that agnosticism is just as present in the API realm. CloudStack maintains and develops its own user and administrative APIs that are easily consumable, but we also have a native translation layer for the Amazon EC2 API that covers about 60% of the classic EC2 API set; because we recognize that there are so many tools written around EC2.
There’s also plenty of work going on close to the project from community members who value the API diversity. So you can find both OCCI implementations for CloudStack as well as a Google Compute Engine translation layer.
In short, the project recognizes that there are valid reasons to use a real or de facto standard and we see the advantages and limitations that come with that. We encourage the development of those ‘standards integrations’ even as we continue forward development of our native APIs.
David Nalley @ke4qqq, [email protected]] is a committer and member of the Project Management Committee for Apache CloudStack and is employed in the Citrix Open Source Business Office. David is also a Thought Leader for Compare the Cloud and you can see his profile here.
