Is the CMDB Irrelevant in a Virtual and Cloud Based World?

Configuration Management Databases (CMDB’s) have been a linchpin of the offerings from the enterprise systems management vendors like CA, IBM, BMC and HP. These products have been marketed as the foundation of both the ITIL framework for management processes, and the Business Service Management frameworks offered by these vendors. While these offerings occupy very important parts of the product strategies from the various vendors who offer them, it is also the case that CMDB’s are enormously expensive to purchase and implement – and due to the time required to implement them have a long time to value for the customer. For these reasons, relatively few enterprise customers have implemented CMDB’s.

Now, along comes virtualization and cloud computing. This article explores what these new developments will have upon the role of the CMDB. The key question is whether or not the CMDB will now become much more important due to the changes created by virtualization and the cloud, or whether the CMDB will be rendered irrelevant by these very changes.

CMDB’s – An Overview

ITIL defines a CMDB as follows:

“A database used to store Configuration Records throughout their Lifecycle. The Configuration Management System maintains one or more CMDBs, and each CMDB stores Attributes of CIs, and Relationships with other CIs.”

A configuration management database (CMDB) is often implemented using standard database technology and typically persists CI lifecycle data as records (or configuration records) in that database. Configuration records are managed according to some data or information model of the IT environment. One of the goals of this specification is to expedite the federated implementation of multiple CMDBs in a single configuration management system.

ITIL defines (in part) a configuration management system as follows:

“A set of tools and databases that are used to manage an IT Service Provider’s Configuration data. The CMS also includes information about Incidents, Problems, Known Errors, Changes and Releases; and may contain data about employees, Suppliers, locations, Business Units, Customers and Users.”

A configuration management system is presumed to be a federation of CMDBs and other management data repositories. The federated CMDB described in this specification is a good match with the database requirements of a configuration management system.

CMDB Overview1

The federated CMDB could support the following scenarios. (However, the scenarios that a federated CMDB supports are left entirely to the discretion of each implementation.):

  • Maintain an accurate picture of IT inventory from a combination of asset information (finance) and deployment/configuration information
  • Reflect changes to IT resources, including asset and licensing data, across all repositories and data sources
  • Compare expected configuration versus actual configuration

Enable version awareness, such as in the following examples:

  • Coordinate planned configuration changes
  • Track change history

Relate configuration and asset data to other data and data sources, such as incident, problem, and service levels. The following are some examples:

  • Integration of change management and incident management with monitoring information
  • SLA incident analysis, by using the service desk and incident

Federated CMDB

Virtualization and Cloud Driven Configuration Management Requirements

A Federated CMDB is supposed to be the place where the data about the assets and their relationship to each other (the configuration of these assets) is stored. It is not necessary that all of this data be stored in one CMDB, but the notion of a federated CMDB means that that CMDB at least knows where all of the data is so that you can query the master CMDB and it can go get the information that is not within that single database.

Virtualization and the Cloud introduce several new challenges to the idea of CMDB as currently specified and offered by the various CMDB vendors. Those challenges are:

  1. A whole new class of data gets created by the virtualization platform – specifically how the virtualization platform itself is configured in support of the guests and the applications that run on the guest.
  2. A whole new set of relationships between the elements in this data get created – specifically new relationships between hosts, hypervisors, guests, virtual networks and virtual storage get created that existing CMDB’s were not built to handle.
  3. New information gets created at a very rapid rate. Hundreds of new guests can get provisioned in time periods much too short to allow for the traditional Extract, Transform and Load processes that feed CMDB’s to be able to keep up.
  4. The environment can change at a rate that existing CMDB’s cannot keep up with. Something as simple as vMotion events can create thousands of configuration changes in a few minutes, something that the entire CMDB architecture is simply not designed to keep up with.
  5. Having portions of IT assets running in a public cloud introduces significant data collection challenges. Leading edge APM vendors like New Relic and AppDynamics have produced APM products that allow these products to collect the data that they need in a cloud friendly way. However, we are still a long way away from having a generic ability to collect the configuration data underlying a cloud based IT infrastructure – notwithstanding the fact that many current cloud vendors would not make this data available to their customers in the first place.
  6. The scope of the CMDB needs to expand beyond just asset and configuration data and incorporate Infrastructure Performance, Applications Performance and Service assurance information in order to be relevant in the virtualization and cloud based worlds.

For the reasons listed above, it is doubtful the today’s enterprise CMDB solutions will make the leap forward into these new dynamic and distributed virtualized and cloud based environments. Rather it is likely that we will again see what we have seen every time a new wave of innovation has swept through IT. New solutions will be built and delivered that address the specific requirements of the new wave. These new solutions will for the most part come from new venture funded startups and perhaps from the virtualization platform companies themselves.

Who and What Will Replace the CMDB?

The first issue that needs to be addressed when it comes to how to create relevant CMDB functionality for virtualized and cloud based environments is where the data is going to come from. The issue with this data is that there are more elements, more relationships and a higher rate of change and therefore transaction rate than existing CMDB solutions are designed to keep up with. These means that a CMDB capability that is relevant to the virtualization and cloud computing worlds must get updated in real time, and be continuously accurate. This in turn means that any attempt to poll the underlying environment (as is done with all existing CMDB’s) is doomed to failure as it will be impossible to poll the environment frequently enough for the configuration management solution to keep up in real time with ongoing configuration changes.

Since VMware vSphere is the market leading virtualization platform that creates these new problems, it is only logical that we look first to VMware both in terms of how it is making data available and how it is using that data itself. VMware has broken significant new ground with the VMsafe API, which is being used to various degrees by vendors in the VMware ecosystem. Some examples of good progress in this area are:

  • Reflex Systems was the first third party vendor to get their driver certified for the VMsafe.net. This allows Reflex to see everything that flows through the hypervisor (which is everything that the hosts and guest are doing). Reflex also can do deep packet inspection to identify the applications running in the guests, and can report configuration changes along with changes in resource utilization patterns. All of this is stored in Reflex’s own database which is accessible by the Reflex VQL query language.
  • ManageIQ has built an extremely robust virtualization management platform, that is based upon deep and broad configuration change detection and configuration policy enforcement as its core functionality. ManageIQ also collects resource utilization data and can cross correlate this data with configuration change data. All of the information collected by ManageIQ is stored in ManageIQ’s own database.
  • Hyper9 uses a search based technology to be able to collect, trend and report on an incredibly wide range of configuration items and to combine these items with views of resource utilization in the virtualized environment. All of the information collected by Hyper9 is stored in Hyper9’s own database.
  • Akorri combines a leadership position in Infrastructure Performance Management with the ability to detect and accept configuration events in the vSphere environment. Akorri is also the only vendor that can cross-correlate Infrastructure Response Time with configuration issues down to the spindle level in the storage arrays. All of Akorri’s information is stored in Akorri’s own database.
  • Xangati provides a very deep view of the local and wide area network based upon data collected via Netflow and from virtual appliances on the VMware hosts. This includes network configuration data as well as data about what is communicating with what, and how long it is taking. This data is stored in a Xangati database.
  • Netutive has just announced a new Performance Management Database (PMDB). The PMDB takes the performance information (response times and all of the related infrastructure metrics) that Netuitive collects from the hundreds of other monitoring products that Netuitive interfaces with and stores that information in one central performance oriented database. The PMDB also interfaces with popular CMDB’s. This allows analysis of performance data across servers, guests, resource pools, applications, regions, business units or services.

Conclusions

The CMDB’s that were designed and architected for static physical systems appear to be unwieldy, too difficult to keep up to date, and not real-time enough to make the transition into the virtualized and cloud based world.  Virtualized environments change too fast for existing CMDB’s to keep up, and the notion of keeping a CMDB up to date as assets are moved into and out of public clouds seems hopelessly beyond the intended original use case of a CMDB.

Given that traditional CMDB’s are now looking like a legacy technology, what can and should take its place? The starting point needs to be a new architecture that is based upon a data model that appropriately includes the correct elements of these new distributed and dynamic virtualized systems, with an ability to keep this data store up to date in near real time. It is also likely that the right answer will incorporate not just configuration data, but also true performance data that includes metrics like Infrastructure Response Time and Applications Response Time. In other words, the CMDB needs to make a transition from being configuration focused to being focused upon how configuration impacts the delivery of virtualized and cloud based Business Services. This new datastore will therefore need to tie directly into existing Infrastructure Performance efforts, Applications Performance efforts and next generation Service Assurance efforts. In the near term enterprises should look to new and creative vendors who are blending deep configuration management of virtualized and cloud based environments, with leading edge performance analysis capabilities like the ones mentioned above and in the posts linked to in the article. Long term a new standard definition for something like Netuitive’s PMDB may replace the currently archaic CMDB.

Share this Article:

The following two tabs change content below.
Bernd Harzog
Bernd Harzog is the Analyst at The Virtualization Practice for Performance and Capacity Management and IT as a Service (Private Cloud). Bernd is also the CEO and founder of APM Experts a company that provides strategic marketing services to vendors in the virtualization performance management, and application performance management markets. Prior to these two companies, Bernd was the CEO of RTO Software, the VP Products at Netuitive, a General Manager at Xcellenet, and Research Director for Systems Software at Gartner Group. Bernd has an MBA in Marketing from the University of Chicago.

Related Posts:

Leave a Reply

11 Comments on "Is the CMDB Irrelevant in a Virtual and Cloud Based World?"


Sort by:   newest | oldest | most voted
Member
mwronski
6 years 3 months ago
I completely agree that many existing CMDB products were not designed to keep up with the pace and dynamic nature of virtualization and cloud computing. This is one of the reasons for our design decisions for the Reflex VMC. I would like to point out that Reflex VMC does far more than you gave us credit for above. In addition to the security functions described, we cover much of the challenges you lay out in points 1-4 above. The vProfile and vWatch features of Reflex VMC provide comprehensive configuration tracking, reporting, and audit of the virtualization environment (host config, virtual… Read more »
Guest
6 years 3 months ago
I tend to believe the challenge with current products and projects around CMDB is that there is too much emphasis on the DB and thus often times “fixed” to the current known environment as input and output requirements. The secret is to design and leverage technology that is open, secure, policy driven externally (as this will change) and takes into consideration a heterogenious enviromnent intelligently. The one thing we can count on is change in incoming technology and outgoing views/uses. The CMDB is an enabler to Intelligent Workload Management in a complex, heterogenious and changing environment. Data needs to be… Read more »
Mike Norman
Member
6 years 3 months ago
There are three strands runing through Bernd’s post here. First, the relevance of application performance data in the context of CMDB, and actually ITIL really isn’t focussed there. Arguably it could be, but whereas change management or service availability are well-informed by the contents of a CMDB, poor service performance is much harder to track to root-cause. Second, the relevance of dynamic guest/platform mapping data. CMDB architectures aren’t built to deal with this, but perhaps the CMDB can black-box the virtual infrastructure in just the same way as it black-boxes threading in the CPU. Third strand is the overall value… Read more »
trackback
Virtualized Info » On the role of CMDBs in virtualization and cloud computing
6 years 3 months ago

[…] Harzog, founder and CEO of APMExperts.com, recently published an interesting article about the role of Configuration Management Databases (CMDBs) in a world of virtualization and cloud […]

Guest
6 years 2 months ago

Interesting article. I wrote something similar on my blog around how the CMDB is not a prerequisite for APM, as some would have you think: http://blog.compuware.com/post/how-does-the-cmdb-relate-to-application-peformance

While companies might want to look to a CMDB for other reasons, to do it in order to measure and improve on the performance of their applications as delivered to their end users is not a reason.

Guest
Rahul
5 years 11 months ago
This has been a question on my mind too, the difficulties of maintaining CMDB in a virtualized env where one might not even be aware of the underlying infrastructure is real or virtualized. CM & CMDBs need to take a holistic look at software development and service delivery. A (probably) silly question : why is configuration management and CMDB always taken in context of ITIL, which apparently only is related to service. A software development project that is a retail software rather than a service still needs the ability to track changes to requirements, code and other dependencies , system… Read more »
Member
5 years 11 months ago
It is not a silly question. The whole relationship between configuration management, service management, and ITIL is, in my mind, thown into question by virtualization. It is quite possible that ITIL needs to either die or substantially change as a result of virtualization. As you point out ITIL misses the key impacts of development decisions on configuration. ITIL was not designed to cope with Agile Development and its rapid rate of change, nor was ITIL designed to cope with the 1000’s of configuration changes per second that can occur in a dynamic data center. I have a friend at a… Read more »
Guest
5 years 6 months ago
If you follow ITIL’s definition of a CMDB then your infrastructure should theoretically disappear and the need for vendor driven Configuration Management solutions should also disappear. However, if you adhere to the reality that there are many different forms of configurations that must be managed (build, distribution, installation, instantiation, initialization, execution, rollback, etc.) then you realize that most of the big vendor CMDBs don’t come close to solving the real problem of what Configuration Management means. The big vendors, like CA, IBM, BMC and HP, “never” addressed the needs of Software Configuration Management, let alone human and organizational configuration management.… Read more »
Guest
Kalyan Kumar
3 years 6 months ago
We are getting too much caught on the word CMDB.. it is all about a metadata repository which will contain Core Data and point to data sources outside. also the consumer of the CMDB will determine how the data is used or even updated… the core is clearly delink the diffference between CI (Config Item), Asset (Financial Value) and MO (Managed Object). all there can be there for a single Device/Instance/app etc or any one may apply… the key is to have the right data model. also it seems there there is too much of confusion around Event Management DB… Read more »
Guest
iwantvarun
2 years 5 months ago

CMDB and cloud have no relation.

Data Compilers are the future for CMDBs, for many reasons. Unlike transaction based database oriented CMDBs, Data Compiled CMDBs…

– allow for snapshots of the enterprise at any point in time, handling history very well,
– allow for recompiling, at will, to accommodate rapid changes,
– allow for models to be changed with little impact on work and on the enterprise,
– allow for multiple instances and portability (because the outputs run on laptops),
– and cost a fraction of the price to own and support.

Guest
9 months 21 days ago

We have talked to our customers, and all of them need a CMDB. They need a place to “keep” details for assets, licenses, documents etc.

When you using hybrid, its nor easy to track and keep the overview.

-Mads

wpDiscuz