Is the Third Party Market for Virtualization Performance Management Alive or Dead?

Now the VMware is shipping AppSpeed and owns Hyperic, the question has to be asked? Should companies looking for performance and capacity management tools for their VMware environments look just to VMware to meet these needs, or should they also continue to look at third party solutions?

In order to address these questions, we need to first understand what the new VMware products provide, and what gaps exist in their functionality. First let’s take a look at AppSpeed. AppSpeed is a compelling solution for the measurement and management of the performance of web based applications (that may contain Java or .Net middle tiers) and that also talk to back end SQL or Oracle database servers. AppSpeed is a virtual appliance that sits on a virtual mirror port on the VMware vSwitch, and AppSpeed has specific protocol decoders for web, .Net, J2EE, SQL Server and Oracle protocols. As a result of these specific protocols decoders, AppSpeed can discover transactions in these protocols and time them between the servers that comprise an applications system. AppSpeed is therefore very useful for understanding the performance of an application that is built to these protocols. However, AppSpeed does have some limitations:

  • AppSpeed is a VMware vSphere specific product. It is integrated with vSphere vCenter Server (Virtual Center). It does not work with prior versions of VMware, nor does it work with other virtualization platforms. It also does not work unless substantial portions of the application system are virtualized (if the traffic is not flowing through a vSphere host, AppSpeed is not going to see it).
  • AppSpeed is not a resource and availability monitor (Hyperic HQ is, more on that below). AppSpeed is therefore not going to be terribly useful for monitoring tasks that require deep monitoring of resource utilization data (like Capacity Planning, Capacity Management, and Availability Management).
  • AppSpeed is not an Infrastructure Performance Monitor. Specifically, it is does not monitor the health and performance of the virtual infrastructure (from the VM’s to the spindles in the SAN)
  • AppSpeed does do a great job of monitoring applications performance (applications response time) for the applications that it supports (see above for the list and the caveats), but it is not a general monitor of response time for all applications. This is not a slam against AppSpeed, since there is at the moment no general monitor of applications response time available that works across all applications types irrespectively of how they are architected. However, this is an important issue, since owners of the virtual infrastructure want a tool that supports all applications, whereas an applications architecture specific tool like AppSpeed is more likely to appeal to team that support individual applications on the virtual infrastructure.
  • AppSpeed is not, nor does it claim to be an end-to-end transactions performance monitor that traces individual transactions through the entire stack and layers of a complex applications system.

Hyperic HQ (recently acquired as a part of the SpringSource acquisition) is a broad scale, heterogeneous resource and availability monitoring solution. Please see this post for a detailed review of the features and capabilities of Hyperic. Hyperic HQ monitors a broad spectrum of physical, virtual, open source and vendor proprietary server resources. It has been proven to scale up into very large environments. However, Hyperic HQ (at least at the moment) has the following limitations:

  • While Hyperic is now owned by VMware, this has not been the case for very long, and vendors that have been focused upon adding value to VMware have a much greater degree of depth and breadth of integration with VMware than does Hyperic at this moment in time.
  • Hyperic HQ is missing the virtualization specific reporting and analytics that are present in mature VMware monitoring solutions.
  • While Hyperic is an infrastructure monitoring solution, it approaches the problem from a resource utilization perspective, not a response time perspective. Due to how pervasively resources are shared in a VMware resource pool, this is not the most effective method to determine infrastructure performance. Approaches that time how long it takes the infrastructure to respond to requests for work are better suited to this task.

If we combine AppSpeed and Hyperic HQ in an environment, and combine the limitations listed above, the following picture emerges for enterprises and mid-size businesses looking for first class monitoring solutions:

  • Due to the fact that few mission critical applications have been virtualized to date, most companies have focused upon acquiring monitoring solutions that focus upon availability and performance management for their virtual environments. Market leading products like VizionCore vFoglight and Veeam Monitor that have deep VMware integration and large installed bases have, at the moment a substantial advantage over AppSpeed and HQ from a resource monitoring perspective. Enterprises looking to buy a resource and availability monitoring solution right now will find that VizionCore vFoglight and the Veeam nWorks Connectors for SCOM and HP OpenView offer compelling and leading edge functionality. Mid-sized companies looking for a VMware foscused solution should look Veeam Monitor which is the market leader in this segment.  Mid-sized companies looking for a broad scale resource and availablity monitoring solution that also includes good VMware support should look at up.time 5 from uptime Software. All of these solutions have substantial VMware specific functionality that is not present in Hyperic HQ yet. Companies looking for a capacity planning solution will find that vKernel has a more robust set of functionality in this regard than is present in the VMware products. However this is certain to change over time as VMware strongly influences the development pipeline at Hyperic.
  • Understanding the performance of the virtual infrastructure (the service level that the infrastructure provides to applications and users) is probably the single most important issue for administrators of VMware production environments. For a variety of reasons (articulated in the Performance and Capacity Management White Paper available for download on the White Papers page), measuring the response time of the infrastructure is a better way to understand infrastructure performance than is measuring resource utilization in the infrastructure. Vendors like Akorri and Virtual Instruments who have serious intellectual property and technical capabilities in this area remain highly differentiated from the VMware offerings, and are a great choice for enterprises looking to understand virtual infrastructure performance. Akorri BalancePoint is the only solution in this space that can map I/O flows from the VM’s to the spindles in the arrays, and Virtual Instruments NetWisdom is the only solution in this space that can comprehensively and deterministically monitor the Fiber Channel SAN while scaling up to the largest and most sophisticated environments.
  • Teams that support important applications on a virtual infrastructure will need a virtualization aware applications performance management tool. AppSpeed is an excellent virtualization aware APM tool. If your application(s) of interest use the protocols supported by AppSpeed (primarily HTTP and the SQL Server and Oracle database protocols), then you should strongly consider AppSpeed. However there are many choices when it comes to APM tools. For example the full Quest Foglight product has support for a broader set of applications than does AppSpeed and contains deeper root cause diagnostics than does AppSpeed. BlueStripe FactFinder is a fine application response time monitoring product that works across multiple virtualization platforms, and supports applications on physical infrastructures as well. CA, IBM, and HP all offer products that do application management for applications built to J2EE and .Net and many enterprises already own these solutions.
  • If you have business critical transactional applications that you are or are planning to run on a virtual infrastructure (and these applications are written to J2EE or .Net) then you need a virtualization aware Transaction Performance Management solution. dynaTrace, Optier, and Quest (Foglight) all offer such solutions. The advantage of Transaction Performance Management over APM is that with TPM the product traces the performance of each individual transaction end-to-end in the server tiers of the applications system. This is critical for virtualized business critical applications as the dynamic nature of these systems can cause transaction paths to move around as VM’s are moved around.
  • End User Experience Management (understanding response time from the perspective of the actual end users) is a separate and rapidly growing category of monitoring solutions. Vendors like Knoa (GEM) and Aternity (FPI) have solutions based upon agents that run on end users’s desktops that monitor the actual response times experienced by the actual end users within their applications. These solutions can be the source of valuable “outside-in” data about the performance of applications systems as they get virtualized (an objective response time perspective that is not impacted when server tiers get virtualized), and can also be of great value in VDI projects. Since the VMware solutions do not monitor from the end users’ perspective at all these solutions are not competitive with what VMware offers in any respect.
  • Desktop virtualization is moving into production in many enterprises, and as this occurs, enterprises are starting to realize that as soon as a users’ OS and applications move from their PC to a VM in the data center that the virtualization team is now much more responsible for the performance of that desktop and those desktop applications than was the case previously. There are an emerging set of solutions that focus upon these problems from vendors like LiquidWare Labs, eG Innovations, and Lakeside Software. These solutions all contain VDI specific functionality that is not present in the monitoring offerings from VMware, and should therefore be strongly considered when VDI pilots move into production.

In summary, monitoring VMware is no longer a green field opportunity for third party monitoring vendors. VMware has the products to compete for this business today, and these products will only grow stronger over time. VMware’s products will, of course, only become more VMware specific over time as well. Vendors will need to focus their strategies and their positioning so that their value add to the VMware offerings is clear to each group of constituents. Vendor that offer just a resource monitoring solution or simple integration with Virtual Center API’s will shortly find that the VMware solutions supersede their features and capabilities.

Enterprises who believe that they will end up with more than one enterprise level virtualization platform (those who believe that Microsoft will get there sooner or later), should think about the trade-offs of a virtualization platform independent solution vs. a VMware specific solution (note that AppSpeed is vSphere specific today, whereas Hyperic HQ is not specific to any platform today). Enterprises who are making near term purchasing decisions for resource monitoring solutions should evaluate the functionality of the products listed above relative to what is present in the VMware offerings today and come to the appropriate conclusion as to the best fit for their needs. However, all enterprises should carefully evaluate what their real needs are regarding infrastructure performance management, applications performance management, transactions performance management, and VDI focused solutions. Very differentiated and focused solutions exist in each of these categories that are a better fit for these requirements than are the current solutions from VMware.

Posted in IT as a ServiceTagged , , , , , , , ,