Configuration Management Databases (CMDB’s) have been a linchpin of the offerings from the enterprise systems management vendors like CA, IBM, BMC and HP. These products have been marketed as the foundation of both the ITIL framework for management processes, and the Business Service Management frameworks offered by these vendors. While these offerings occupy very important parts of the product strategies from the various vendors who offer them, it is also the case that CMDB’s are enormously expensive to purchase and implement – and due to the time required to implement them have a long time to value for the customer. For these reasons, relatively few enterprise customers have implemented CMDB’s.

Now, along comes virtualization and cloud computing. This article explores what these new developments will have upon the role of the CMDB. The key question is whether or not the CMDB will now become much more important due to the changes created by virtualization and the cloud, or whether the CMDB will be rendered irrelevant by these very changes.

CMDB’s – An Overview

ITIL defines a CMDB as follows:

“A database used to store Configuration Records throughout their Lifecycle. The Configuration Management System maintains one or more CMDBs, and each CMDB stores Attributes of CIs, and Relationships with other CIs.”

A configuration management database (CMDB) is often implemented using standard database technology and typically persists CI lifecycle data as records (or configuration records) in that database. Configuration records are managed according to some data or information model of the IT environment. One of the goals of this specification is to expedite the federated implementation of multiple CMDBs in a single configuration management system.

ITIL defines (in part) a configuration management system as follows:

“A set of tools and databases that are used to manage an IT Service Provider’s Configuration data. The CMS also includes information about Incidents, Problems, Known Errors, Changes and Releases; and may contain data about employees, Suppliers, locations, Business Units, Customers and Users.”

A configuration management system is presumed to be a federation of CMDBs and other management data repositories. The federated CMDB described in this specification is a good match with the database requirements of a configuration management system.

The federated CMDB could support the following scenarios. (However, the scenarios that a federated CMDB supports are left entirely to the discretion of each implementation.):

  • Maintain an accurate picture of IT inventory from a combination of asset information (finance) and deployment/configuration information
  • Reflect changes to IT resources, including asset and licensing data, across all repositories and data sources
  • Compare expected configuration versus actual configuration

Enable version awareness, such as in the following examples:

  • Coordinate planned configuration changes
  • Track change history

Relate configuration and asset data to other data and data sources, such as incident, problem, and service levels. The following are some examples:

  • Integration of change management and incident management with monitoring information
  • SLA incident analysis, by using the service desk and incident


Virtualization and Cloud Driven Configuration Management Requirements

A Federated CMDB is supposed to be the place where the data about the assets and their relationship to each other (the configuration of these assets) is stored. It is not necessary that all of this data be stored in one CMDB, but the notion of a federated CMDB means that that CMDB at least knows where all of the data is so that you can query the master CMDB and it can go get the information that is not within that single database.

Virtualization and the Cloud introduce several new challenges to the idea of CMDB as currently specified and offered by the various CMDB vendors. Those challenges are:

  1. A whole new class of data gets created by the virtualization platform – specifically how the virtualization platform itself is configured in support of the guests and the applications that run on the guest.
  2. A whole new set of relationships between the elements in this data get created – specifically new relationships between hosts, hypervisors, guests, virtual networks and virtual storage get created that existing CMDB’s were not built to handle.
  3. New information gets created at a very rapid rate. Hundreds of new guests can get provisioned in time periods much too short to allow for the traditional Extract, Transform and Load processes that feed CMDB’s to be able to keep up.
  4. The environment can change at a rate that existing CMDB’s cannot keep up with. Something as simple as vMotion events can create thousands of configuration changes in a few minutes, something that the entire CMDB architecture is simply not designed to keep up with.
  5. Having portions of IT assets running in a public cloud introduces significant data collection challenges. Leading edge APM vendors like New Relic and AppDynamics have produced APM products that allow these products to collect the data that they need in a cloud friendly way. However, we are still a long way away from having a generic ability to collect the configuration data underlying a cloud based IT infrastructure – notwithstanding the fact that many current cloud vendors would not make this data available to their customers in the first place.
  6. The scope of the CMDB needs to expand beyond just asset and configuration data and incorporate Infrastructure Performance, Applications Performance and Service assurance information in order to be relevant in the virtualization and cloud based worlds.

For the reasons listed above, it is doubtful the today’s enterprise CMDB solutions will make the leap forward into these new dynamic and distributed virtualized and cloud based environments. Rather it is likely that we will again see what we have seen every time a new wave of innovation has swept through IT. New solutions will be built and delivered that address the specific requirements of the new wave. These new solutions will for the most part come from new venture funded startups and perhaps from the virtualization platform companies themselves.

Who and What Will Replace the CMDB?

The first issue that needs to be addressed when it comes to how to create relevant CMDB functionality for virtualized and cloud based environments is where the data is going to come from. The issue with this data is that there are more elements, more relationships and a higher rate of change and therefore transaction rate than existing CMDB solutions are designed to keep up with. These means that a CMDB capability that is relevant to the virtualization and cloud computing worlds must get updated in real time, and be continuously accurate. This in turn means that any attempt to poll the underlying environment (as is done with all existing CMDB’s) is doomed to failure as it will be impossible to poll the environment frequently enough for the configuration management solution to keep up in real time with ongoing configuration changes.

Since VMware vSphere is the market leading virtualization platform that creates these new problems, it is only logical that we look first to VMware both in terms of how it is making data available and how it is using that data itself. VMware has broken significant new ground with the VMsafe API, which is being used to various degrees by vendors in the VMware ecosystem. Some examples of good progress in this area are:

  • Reflex Systems was the first third party vendor to get their driver certified for the VMsafe.net. This allows Reflex to see everything that flows through the hypervisor (which is everything that the hosts and guest are doing). Reflex also can do deep packet inspection to identify the applications running in the guests, and can report configuration changes along with changes in resource utilization patterns. All of this is stored in Reflex’s own database which is accessible by the Reflex VQL query language.
  • ManageIQ has built an extremely robust virtualization management platform, that is based upon deep and broad configuration change detection and configuration policy enforcement as its core functionality. ManageIQ also collects resource utilization data and can cross correlate this data with configuration change data. All of the information collected by ManageIQ is stored in ManageIQ’s own database.
  • Hyper9 uses a search based technology to be able to collect, trend and report on an incredibly wide range of configuration items and to combine these items with views of resource utilization in the virtualized environment. All of the information collected by Hyper9 is stored in Hyper9′s own database.
  • Akorri combines a leadership position in Infrastructure Performance Management with the ability to detect and accept configuration events in the vSphere environment. Akorri is also the only vendor that can cross-correlate Infrastructure Response Time with configuration issues down to the spindle level in the storage arrays. All of Akorri’s information is stored in Akorri’s own database.
  • Xangati provides a very deep view of the local and wide area network based upon data collected via Netflow and from virtual appliances on the VMware hosts. This includes network configuration data as well as data about what is communicating with what, and how long it is taking. This data is stored in a Xangati database.
  • Netutive has just announced a new Performance Management Database (PMDB). The PMDB takes the performance information (response times and all of the related infrastructure metrics) that Netuitive collects from the hundreds of other monitoring products that Netuitive interfaces with and stores that information in one central performance oriented database. The PMDB also interfaces with popular CMDB’s. This allows analysis of performance data across servers, guests, resource pools, applications, regions, business units or services.

Conclusions

The CMDB’s that were designed and architected for static physical systems appear to be unwieldy, too difficult to keep up to date, and not real-time enough to make the transition into the virtualized and cloud based world.  Virtualized environments change too fast for existing CMDB’s to keep up, and the notion of keeping a CMDB up to date as assets are moved into and out of public clouds seems hopelessly beyond the intended original use case of a CMDB.

Given that traditional CMDB’s are now looking like a legacy technology, what can and should take its place? The starting point needs to be a new architecture that is based upon a data model that appropriately includes the correct elements of these new distributed and dynamic virtualized systems, with an ability to keep this data store up to date in near real time. It is also likely that the right answer will incorporate not just configuration data, but also true performance data that includes metrics like Infrastructure Response Time and Applications Response Time. In other words, the CMDB needs to make a transition from being configuration focused to being focused upon how configuration impacts the delivery of virtualized and cloud based Business Services. This new datastore will therefore need to tie directly into existing Infrastructure Performance efforts, Applications Performance efforts and next generation Service Assurance efforts. In the near term enterprises should look to new and creative vendors who are blending deep configuration management of virtualized and cloud based environments, with leading edge performance analysis capabilities like the ones mentioned above and in the posts linked to in the article. Long term a new standard definition for something like Netuitive’s PMDB may replace the currently archaic CMDB.

Share this Article:

Share Button
Bernd Harzog (324 Posts)

Bernd Harzog is the Analyst at The Virtualization Practice for Performance and Capacity Management and IT as a Service (Private Cloud).

Bernd is also the CEO and founder of APM Experts a company that provides strategic marketing services to vendors in the virtualization performance management, and application performance management markets.

Prior to these two companies, Bernd was the CEO of RTO Software, the VP Products at Netuitive, a General Manager at Xcellenet, and Research Director for Systems Software at Gartner Group. Bernd has an MBA in Marketing from the University of Chicago.

Connect with Bernd Harzog:


Related Posts:

10 comments for “Is the CMDB Irrelevant in a Virtual and Cloud Based World?

  1. June 2, 2010 at 10:06 AM

    I completely agree that many existing CMDB products were not designed to keep up with the pace and dynamic nature of virtualization and cloud computing. This is one of the reasons for our design decisions for the Reflex VMC.

    I would like to point out that Reflex VMC does far more than you gave us credit for above. In addition to the security functions described, we cover much of the challenges you lay out in points 1-4 above. The vProfile and vWatch features of Reflex VMC provide comprehensive configuration tracking, reporting, and audit of the virtualization environment (host config, virtual networks, storage) as well as dynamic activity like self service provisioning and vMotion. We even have hooks to pull software asset inventory (including patch data) out of the guests and templates in a non-invasive way.

    All of this data can be queried or browsed using our VQL language as well as configured to push the data into a more traditional CMDB system.

    The system was designed from the ground up to function under these load and demands that you mention as challenges.

  2. June 4, 2010 at 1:04 PM

    I tend to believe the challenge with current products and projects around CMDB is that there is too much emphasis on the DB and thus often times “fixed” to the current known environment as input and output requirements. The secret is to design and leverage technology that is open, secure, policy driven externally (as this will change) and takes into consideration a heterogenious enviromnent intelligently. The one thing we can count on is change in incoming technology and outgoing views/uses.

    The CMDB is an enabler to Intelligent Workload Management in a complex, heterogenious and changing environment. Data needs to be referenced where it resides (not collected) and related to become meaningful information in a view that enables IT to Acknowledge – Assess – Act in market conditions in market time. This must be done agnostic to current technology definitions and restrictions and today’s requirements for use as they will both change rapidly.

    It’s not about the database or the technology, it’s the relationships of data, the information it provides in time to intelligently manage the workloads in alignment with the business.

  3. Mike
    June 7, 2010 at 8:06 AM

    There are three strands runing through Bernd’s post here.

    First, the relevance of application performance data in the context of CMDB, and actually ITIL really isn’t focussed there. Arguably it could be, but whereas change management or service availability are well-informed by the contents of a CMDB, poor service performance is much harder to track to root-cause.

    Second, the relevance of dynamic guest/platform mapping data. CMDB architectures aren’t built to deal with this, but perhaps the CMDB can black-box the virtual infrastructure in just the same way as it black-boxes threading in the CPU.

    Third strand is the overall value (or otherwise) of CMDB in any infrastructure – virtual or physical. The CMDB value proposition hasn’t changed much if you exclude performance data and the physical-to-virtual mapping, because they were never really in scope. People struggle with CMDB because of the complexity of mapping their environments, and because when you’ve finished all you have is a map.

    However, when you get close to ITIL, you realise there is something there that transcends the detail of CMDB implementation. A Service-to-Technology approach that too often gets replaced by a Technology-first approach in IT shops.

  4. July 1, 2010 at 3:05 PM

    Interesting article. I wrote something similar on my blog around how the CMDB is not a prerequisite for APM, as some would have you think: http://blog.compuware.com/post/how-does-the-cmdb-relate-to-application-peformance

    While companies might want to look to a CMDB for other reasons, to do it in order to measure and improve on the performance of their applications as delivered to their end users is not a reason.

  5. Rahul
    October 6, 2010 at 6:03 AM

    This has been a question on my mind too, the difficulties of maintaining CMDB in a virtualized env where one might not even be aware of the underlying infrastructure is real or virtualized. CM & CMDBs need to take a holistic look at software development and service delivery.

    A (probably) silly question : why is configuration management and CMDB always taken in context of ITIL, which apparently only is related to service. A software development project that is a retail software rather than a service still needs the ability to track changes to requirements, code and other dependencies , system software , patches , databases and infrastructure. I don’t get the logic of tying CM to service aspects like incident management, problem management etc, CM is relevant to development as well.

  6. Bharzog
    October 6, 2010 at 9:54 AM

    It is not a silly question. The whole relationship between configuration management, service management, and ITIL is, in my mind, thown into question by virtualization. It is quite possible that ITIL needs to either die or substantially change as a result of virtualization. As you point out ITIL misses the key impacts of development decisions on configuration. ITIL was not designed to cope with Agile Development and its rapid rate of change, nor was ITIL designed to cope with the 1000′s of configuration changes per second that can occur in a dynamic data center. I have a friend at a large enterprise who is in charge of their VMware environment and they have decided simply not to tell the ITIL Change Control committee about VMotion and DRS since they believe that the mentality of the folks on that committee will not be able to embrace automatic and dynamic changes.

  7. March 15, 2011 at 10:27 PM

    If you follow ITIL’s definition of a CMDB then your infrastructure should theoretically disappear and the need for vendor driven Configuration Management solutions should also disappear. However, if you adhere to the reality that there are many different forms of configurations that must be managed (build, distribution, installation, instantiation, initialization, execution, rollback, etc.) then you realize that most of the big vendor CMDBs don’t come close to solving the real problem of what Configuration Management means.

    The big vendors, like CA, IBM, BMC and HP, “never” addressed the needs of Software Configuration Management, let alone human and organizational configuration management. These needs will never go away, whether you use a cloud or not.

    The material in this post focus on ITIL’s view of Configuration Management. IT Leaders know and understand that there is far more to Configuration Management than what ITIL suggests.

    Thanks for the article.

    Theresa

  8. Kalyan Kumar
    March 25, 2013 at 1:39 AM

    We are getting too much caught on the word CMDB.. it is all about a metadata repository which will contain Core Data and point to data sources outside. also the consumer of the CMDB will determine how the data is used or even updated… the core is clearly delink the diffference between CI (Config Item), Asset (Financial Value) and MO (Managed Object). all there can be there for a single Device/Instance/app etc or any one may apply… the key is to have the right data model.

    also it seems there there is too much of confusion around Event Management DB or Ops Mgmt DB viz the CMDB.. the realtime state of Managed Objects is what is stored in the OMDB or Event DB.. these objects would be releated or linked to a CI in CMDB…

    I agree on Discovery tools completely… they r just a waste of effort.. the key is to havea a strong Integration Layer in the CMDB which can integrate to externral sources via ETL, REST API, Message Bus etc so that depending upon the level of dynamic envirohment one has, it can connect and leverage the Native sources rather trying to duplicate them..

    things have to be made simple and using the Least Common Denominator approach one can achieve the outcome…

  9. April 6, 2014 at 4:01 PM

    CMDB and cloud have no relation.

    Data Compilers are the future for CMDBs, for many reasons. Unlike transaction based database oriented CMDBs, Data Compiled CMDBs…

    - allow for snapshots of the enterprise at any point in time, handling history very well,
    - allow for recompiling, at will, to accommodate rapid changes,
    - allow for models to be changed with little impact on work and on the enterprise,
    - allow for multiple instances and portability (because the outputs run on laptops),
    - and cost a fraction of the price to own and support.

Leave a Reply

Your email address will not be published. Required fields are marked *


8 × = seventy two