The idea of running a private cloud, hosted or on premise, for a single company is rapidly becoming a commonplace one. More and more businesses are learning of cloud computing and seeing that running their own cloud platform is both feasible and potentially valuable to the business. But do to a general lack of cloud knowledge it is becoming more and more common that clouds are recommended when they do not suit the needs of the business at all, instead being mistaken for traditional virtualization management systems.
A cloud is a special type of virtualization platform and fills a unique niche. Cloud computing takes traditional virtualization and layers it with automated scaling and provisioning that allows for rapid, horizontal scaling of applications. This is not a normal business need. Cloud also lends itself, and is often tied to, self-service of resource provisioning but this alone does not make something a cloud nor justify the move to a cloud platform, but could be an added incentive. What makes cloud interesting is the ability to provide self-service portals to end users and the ability for applications to self-provision themselves. These are the critical aspects that set a cloud platform apart from traditional virtualization.
What a cloud does not imply are features such as simplified whole-domain system management from a single pane of glass, large scale consolidation, easy migration between hardware systems, rapid provisioning of new systems, virtualization, high availability, resource over-commitment, etc. These features are all available in other ways, primarily through or on top of standard platform virtualization (VMware vSphere, Microsoft’s HyperV, Xen, et. al.) It is not that these features cannot be made available in a private cloud, but the features are not aspects of the cloud but rather of the underlying virtualization platform. The cloud layer is above these and simply passes through the benefits of the underlying layers.
Often cloud is approached because of a misunderstanding that many of the features commonly associated with private clouds are not available in some other, simpler form. This is rarely the case. Normal virtualization platforms, most commonly VMware’s vSphere and Microsoft’s HyperV, offer all of these options. They can be used to make robust clusters of physical servers, managed from a single interface, with incredibly high reliability and rapid provisioning of new systems that require minimal specialty knowledge from the IT department and maintain traditional business workflows. Most times, when I am speaking with businesses that believe that they may be interested in pursuing the ownership of their own cloud, the features that they really want are not cloud features at all.
The term “cloud” has simply become so popular recently that people begin to assume that important features for nearly everyone must be attributed to it to explain the sudden surge in importance, but this is simply not the case. Cloud remains, and will remain, a predominantly niche solution appropriate for only a very small number of companies to own themselves. The use of public clouds or the use of hosted services delivered from cloud platforms will become, and indeed has already become, nearly ubiquitous But ownership of a private cloud for the use of a single company is a long way from being a business need for most businesses or business units and in many cases, I suspect, never will become so.
Private clouds shine in two key areas. The first is a business who needs a large number of temporary or ad hoc systems “spun up” on a regular basis. This often occurs with large development teams and application testing groups, especially if these groups target multiple operating systems. The ability to rapidly provision temporary testing systems or lab systems can be very advantageous and the nature of cloud computing to easily expose provisioning tools that allow business customers to create, manage and destroy their own system instances with, we would expect, built-in charge back mechanisms can be very beneficial to corporate efficiency as the interaction between the IT department and the end users becomes nearly frictionless for this transaction. Responsibility for maintaining the cloud as a whole can easily be segregated from the responsibilities of maintaining individual systems. Seldom used in this manner for production workloads, this allows a self-service approach that many business units desperately seek today. Impractical on a small scale due to the overhead of creating and maintaining the cloud platform itself but on a large scale can be hugely productive. In addition to technical advantages, this aspect of cloud computing can serve as a model for thinking of IT as an internal service provider and departments as customers. We have long discussed IT and other business units in these terms but we rarely truly think of them in this way.
The second area where cloud computing really comes into its own and the one for which the concept was developed originally is to handle auto provisioning for horizontally scaling applications. That is, application workloads that are able to increase in their capacity handling ability by spawning new instances for themselves. On a small scale, many web applications, due to their stateless nature, do this within a single system by spawning new thread workers to handle additional connections. An Apache web server might start with eight listeners ready to service requests but as those threads become exhausted it automatically starts new threads to handle additional incoming connections so that it is able to scale within the confines of a single server. To expand on this concept, applied to cloud computing, that same application sensing thread exhaustion approaching on a system-wide level (or based on other metrics such as a lack of free memory or a loss of performance) would use an API exposed from the cloud computing platform to signal the cloud management system to provision a new copy of the system that was calling it – essentially cloning itself on the fly. In a matter of seconds, a new virtual server, identical to the first, would be up and running and joining its parent in servicing incoming requests. This child or clone system would likewise spawn new threads internally, as needed, and then if it too sensed exhaustion would call the cloud platform to create yet another new system to handle even more threads. In this way the application can grow itself almost infinitely (within the hardware limits of the entire cloud platform) as needed, on the fly, automatically. Then, as individual systems become idle, workloads die down, one at a time a system can signal that it is no longer needed to the cloud management system and the system will be powered off and destroyed as it was simply a stateless clone, freeing system capacity for other applications and workloads that may need to take advantage of the spare capacity.
As we can see, cloud computing is massively powerful, especially with the bulk of today’s public and enterprise applications being written in a stateless manner in order to take advantage of web protocols and end user interfaces. Web applications are especially adept at leveraging cloud computing’s scalability model and most large scale web applications leverage this elastic expanding and contracting of capacity today. Many new NoSQL models are beginning to emerge that signal that databases, in addition to application front end processing nodes, may soon benefit from similar models on a large scale. This can certainly be leveraged for internal applications as well as publicly facing ones, however internal applications rarely need to scale beyond a single system and so it is quite rare to find private clouds being leveraged in quite this way.
The dangers around cloud computing come in the form of additional complexity above and beyond normal virtualization. There is the potential for complex storage needed to support the platform and more layers to learn and maintain. Cloud computing’s ability to rapidly create and destroy systems can make it tempting for users to attempt to use cloud resources as if they were persistent systems, which they can be made to be, which can result in data loss from users receiving behavior very different from what is traditional and expected. Possibly the biggest cloud concern is a human one and that is the increased likelihood of experiencing uncontrolled system sprawl as end users wildly spin up more and more new systems which, as they are created by end users and not IT, are probably not tightly controlled and monitored leaving systems in a rogue, and oft forgotten state. This can lead to a maintenance and security nightmare as systems go unpatched and uncared for increasing risk and draining resources. And most worrisome is the possibility that systems will be created and forgotten and potentially exist without proper licensing. Tracking and reporting on auto provisioned systems carries process risk caused by the huge shift in how systems are created. IT departments are accustomed to the heavy licensing processes necessary to maintain compliance but with cloud computing there is a potential for this process to be exposed to the business units in a way for which they are not at all equipped to handle. There are accommodations for the licensing needs of cloud computing, but this is extra complexity and management that must be addressed. Allowing systems to exist without direct IT department oversight clearly carries risk of a potentially unforeseen nature.
Private cloud ownership brings many exciting possibilities, but it is clear that these benefits and opportunities are not for everyone. They cater to larger businesses, to those with good process control, to companies running especially adapted applications that are capable of taking advantage of the system-level elasticity of the resources and those needing large scale ad hoc system creation and destruction provided, as a service, for end users to self-provision. Most large enterprises will find limited use for cloud computing in house. Smaller organizations will rarely find cloud computing to be advantageous in the near future, if ever.