Information Week’s Plug into the Cloud blog featured an article by Editor John Foley. In the article, John mentions that Gartner predicts that the cloud services market will jump to $56 billion this year. In that same post, he mentions that Gartner predicts those same cloud services will reach $150 billion by 2013.

John opens the piece by pointing out the skepticism and ambiguity as to what cloud services really are. Many in the IT and business community claim that cloud services are not a new paradigm at all and that the term is nothing more than a buzzword generating a bunch of short term hype around the same old ideas and existing technologies that we in the IT community already utilize.

Perhaps those pessimists are somewhat correct in their assertions, but I have a more positive perspective on the current cloud services market. I have first hand knowledge that the concept of cloud services and dynamic computing environments really do work and offer the payback that many articles and white papers claim. I have had the privilege of being directly involved with such an effort for the last two and a half years. Granted, my involvement has been on coming up with the strategy for effectively monitoring the availability and performance of the entire platform and working with others to present the key metrics in a meaningful way to support teams and end-users. This has put me in a unique position to get to look under the covers of each of the technology components that make up a cloud services infrastructure and see how all the components work together to generate great value for the business.

Sure, the concept involves the same old routers and switches that we have been using for decades now, albeit with an enhanced feature set, and the same old Intel based architecture for much of the dynamic Grid and Fabric environments (software solutions that are usually components of a SOA). Again, those Intel based systems have matured quite a bit from where the 80s and 90s, but nonetheless, they are Intel based PCs that are collectively used to run massive computationally intensive jobs, on a much less expensive platform than a mainframe. Not to mention these cloud services environments usually Provide the ability to harvest unused CPU cycles from user desktops as well as disaster recovery infrastructure for better utilization of a firm’s IT Investment. The details behind these concepts are the scope of another blog post sometime in the future.

What is different with cloud services and the innovative firms involved with the concept? Many of the principals of these firms have lived through the disjointed and often complex sprawl of the distributed computing model of years past. I am referring to the approach of deploying hordes and hordes of servers into a data center support a single application, or maybe two. There was no good documentation on how the system was architected or deployed or even how it was connected in the data center. Managing changes to these deployed systems was also a challenge, at best, and detecting the impact of change to the business was next to impossible.

All the while, these systems are consuming power and dispersing heat into the data center, whether they are utilized or not. These same systems were usually monitored for availability and maybe very limited performance data, but that was about it. There was a complete lack of transparency in terms of host-to-host or application to application dependencies within a data center or across data centers. Furthermore, the monitoring data that was provided was not being instrumented, correlated, or displayed to the end user in the context of the business they were supporting or in such a manner that the data was meaningful. That is to say, the data provided no data to those that depended on it the most, the end consumer and user of the application and those supporting it.

A cloud services approach forces an organization to thoughtfully plan for and deploy an infrastructure to support many different business applications. The organization must do this in much the same way that a restaurant publishes a menu. In other words, the various reusable components of the environment must be published in such a way that application developers and other IT teams know that they exist. Those teams also need to know how to utilize those services and what charges may exist for utilizing those services. This approach lends itself to the concept of the Information Technology Information Library (ITIL) and one of the critical components of ITIL, the Service Catalog. Back to the restaurant analogy, those teams need a menu to choose from, much as a diner does in a restaurant.

Typically, these infrastructures are well documented and rigorous processes are put in place to ensure that the architecture and design of the system is maintained. In the new cloud services paradigm, one needs to take a new approach to application availability and performance management as well. The tools used in the past may still work, in some cases, but enhanced instrumentation and documentation of those systems is warranted due to their dynamic and componentized nature. There continues to be new software based entrants to this space that can be deployed to augment current investments in legacy monitoring technologies. These tools add automation to the alert and performance metrics that are coming from these systems, and enable analysts to present this data in a way that is actionable and meaningful to the business user as well as the teams supporting the day-to-day operations of the application. Concepts such as root cause now have real meaning, as alert and performance metrics can be integrated into a system to statistically determine what an application’s performance baseline is, and automatically alert teams to an outlier event. These metrics can then be compared against business metrics to determine if there is a correlation to business volumes and IT performance.

Call it what you will, the concept of cloud services is showing great promise. The combination of well-documented infrastructures, combined with the renewed vigor around process and application performance instrumentation is exactly what the consumers of IT Services have been looking for within corporate America. Business leaders are fed up with the lack of transparency in the IT footprint. Cloud services, and the processes and policies that are implemented to enable them to be successful in meeting the needs of the business, provide the level of transparency and, more importantly, key performance indicators, that business leaders can use in growing their business and decreasing their time to market with new products and services. The IT community is constantly challenged to do more with less, now more than ever. With the right discipline, and planning, cloud services will continue to evolve and prove that with upfront planning and forethought, the platform can deliver greater cost savings and improved reliability and performance over traditional distribute computing models.

The key to these cloud services environments is top evolve them to a utility model. The management and measurement of the performance of these environments needs to be done much like a utility. Much like we depend on dial tone to always work, cloud services need to be franchised in that the processes and technology supporting the clouding services should be based on repeatable processes much like the network jack at your office, when you plug in a new application to the utility, it just works. With the right planning and talent at your firm, it will do just that.