Bernd Harzog is the Analyst at The Virtualization Practice for Performance and Capacity Management and IT as a Service (Private Cloud).
Bernd is also the CEO and founder of APM Experts a company that provides strategic marketing services to vendors in the virtualization performance management, and application performance management markets.
Prior to these two companies, Bernd was the CEO of RTO Software, the VP Products at Netuitive, a General Manager at Xcellenet, and Research Director for Systems Software at Gartner Group. Bernd has an MBA in Marketing from the University of Chicago.
On Thursday April 26 VMware announced that it has acquired Cetas, an early stage startup focused upon making access to advanced big data analytics much easier and cheaper. The obvious goal of this is that if you make something easier and cheaper, more of it gets consumed, which then allows more people to benefit from it. 25 years ago, mobile phones were expensive, the size of shoe boxes, and few people could afford to buy them and bother to use them. We all know how ubiquitous mobile phones are now, and this is entirely due to the democratization and commoditization of mobile phone access.
What Does Cetas Do?
Cetas makes it easy to apply advanced self-learning complex event processing technology to random sets of data. Furthermore it is built from the ground up to handle “big data” which means that it is designed to handle large data sets, large amounts of rapidly arriving data, and data that arrives at high rates of frequency (at or near real time rates). VMware thinks that Cetas is good for three primary uses cases shown in the diagram below.
There are two very interesting problems that VMware could potentially address with Cetas. The first is that doing analytics at cloud scale (think of trying to analyze data about every virtual server at Amazon at the same time) is clearly a big data problem, and a challenging problem purely on the front of making the analytics work and be easy to use with data sets of that size.
The second has to do with Operational Performance and Application Performance data. Right now VMware collects data from its hypervisor at 20 second intervals and rolls that up into 5 minute intervals for access via the vSphere API. These intervals are too long, and the rollups obscure too much data, but until now VMware has not had any way to analyze the data to make it more useful. Cetas therefore can potentially solve problems that apparently the Integrien technology that VMware purchased a couple of years ago is not suited to address.
How is Cetas Deployed
Cetas is available as a cloud resident service (analytics as a service), or as an on-premise solution.
When we look back five years from now, we will probably conclude that the Cetas acquisition was one of the most significant acquisitions that VMware did. The Cetas technology is going to bring real time self-learning analytics to several layers of VMware’s management offerings over time. As soon as VMware gets into the business of producing and analyzing real time, continuous and deterministic management data the final nail will be driven in the legacy management solutions that sample and operate at 5 minute intervals.
On 4/12/2012, EMC in conjunction with technology partners VMware, Microsoft, Cisco, Brocade, Citrix and Intel announced EMC VSPEX. EMC VSPEX is an specification framework that allows multiple vendors to participate in providing “standard” building blocks for virtualized data centers and private clouds – targeted at accounts who need less than 250 virtualized servers and/or 2000 virtualized users (VDI) and delivered exclusively through EMC/VMware/Cicso/Microsoft/Citrix partners. The goal of this initiative is to provide the mid-market with cost effective, pre-certified, pre-integrated solution sets and leverage standard building blocks from leading vendors. In other words this is VCE/vBlock for smaller companies delivered through the distributor and VAR channels of the respective vendors. Continue reading EMC VSPEX vs. vBlock, or NetApp FlexPod. Can VMware VARs “Refuse” the Offer?→
Ever since VMware announced the virtual memory based pricing (the VTAX), many organizations have investigated the question of whether they should cease to rely solely upon VMware vSphere as their virtualization platform. These investigations have been motivated by a desire on the part of upper management in many organizations to protect the organization from what they perceive as the potential for increased licensing costs, if the organization pushes density far enough to have to purchase extra vSphere licenses just to address the configured virtual memory limits in each vSphere license.
As a result of these licensing concerns, financial managers have said to their technical staff (the team that runs vSphere), “Look we are not switching now, we are not even adding something else now, but if we do decide to add something else, please investigate the alternatives and tell us what we would pick as our second virtualization platform”. This is of course a very common sense and highly recommended approach in a situation like this. It is essential to know what your alternatives are, even if you do not want to act upon one of them right now.
As organizations have embarked upon these investigations that they have found the following things:
Since the last time that many organizations have looked (the technical staff at most enterprises is very happy with vSphere and has had no motivation to look until they were prodded to do so by their financial managers), competing products from Microsoft (Hyper-V), Red Hat (KVM) and Citrix (XenServer) have gotten much better. VMware vSphere still retains a lead in functionality, performance, and scalability, but that lead is narrowing.
The competing products are less expensive. In some cases much less expensive, and in some cases less expensive enough to matter, but not so much so as to be an overwhelming consideration.
The Virtualization Management Stack Challenge
As soon as organizations come up with a second potential virtualization platform, they then encounter an even more difficult problem that choosing the second virtualization platform. The problem is how to manage two virtualization platforms instead of one. This is a really important consideration because the last thing you want to do is make a mess of managing your virtual environment so that it is just as difficult and expensive to manage as the physical environment that you are moving away from. The primary cause of the mess on the physical side was that most organizations built their management stacks organically over time, and bought overlapping and redundant products for each of the physical environments and the software that ran on them. So the last thing you want to do is repeat this mistake and build two management stacks if you have two virtualization platforms.
Approaches to Cross-Virtualization Platform Management
There are three approaches that you can take to having one cross platform virtualization management strategy:
Assemble as management stack from best of breed third party solutions that support both of your virtualization platforms. For example, performance and capacity monitoring vendors like VKernel (now part of Quest Software), SolarWinds, VMTurbo, and Zenoss all support multiple virtualization platforms. Application Performance Management vendors like AppDynamics, New Relic, BlueStripe, Correlsense, and ExtraHop Networks tend to be agnostic of the underlying virtualization platform. Most backup vendors like Veeam support more than one hypervisor as well.
Standardize upon Microsoft SCOM as your management console, and then use plug-ins to SCOM (management packs) to support other virtualization platforms. For example you can use the Veeam nworks product to manage your VMware environment from within Micrsosoft SCOM. You can use the BlueStripe plugin to SCOM to manage all of the applications deployed across both VMware and Hyper-V virtualization platforms.
Use Hotlink, to extend the reach of vCenter to Hyper-V, KVM, and XenServer. The balance of this article will focus upon this approach.
Hotlink SuperVISOR for VMware 1.5
Hotlink is announcing today a new version of its SuperVisor solution. Hotlink 1.5 allows you to manage vSphere, Microsoft Hyper-V, Red Hat KVM, and Citrix XenServer all from within you vCenter console. Additionally, Hotlink 1.5 adds the following extremely useful functinality:
Snapshot Manager – Administrators can create, utilize, and manage cross-platform snapshots inside the VMware vCenter console – providing a single point of management for heterogeneous virtual machines
Template Manager – Users are able to create and deploy a single template across all target hypervisors, eliminating the time-consuming and inefficient process of building and maintaining platform-specific virtual machine templates
Homogenous Live Migration – VMware vCenter is now extended to provide live migration (e.g. vMotion) of Hyper-V, XenServer, and KVM virtual machines within homogeneous clusters – enabling the robust VMware management capabilities to be utilized for cross-platform, critical workloads
A screen shot of Hotlink SuperVISOR 1.5 is shown below. Notice that all four supported virtualization platforms appear in one tree control on the left side of your vCenter.
Microsoft SCOM vs. vCenter/Hotlink
One of the huge differences between the Microsoft virtualization management strategy and the VMware virtualization management strategy is that Microsoft has an open management pack architecture that lets anyone add value to SCOM, and Microsoft will partner with vendors that add value to SCOM at a sales and marketing level, while VMware has neither the open and extensible partnering architecture, nor the willingness to partner with management vendors who add value to vSphere.
Hotlink is now in the interesting position of filling this gap for VMware even if VMware does not want it filled. The reason for this is that there are a large number of third party products that use the data from vCenter as the source of the information upon which they build their management functionality. Hotlink now adds data from Hyper-V, KVM, and XenServer environments to the data that vCenter provides to the entire VMware management ecosystem. Therefore Hotlink has effectively made the entire VMware management ecosystem into a cross virtualization platform play. Needless to say, this is wonderful for people who love vCenter. Interestingly enough it also opens the door to be able to use VMware’s own management products like vCenter Operations across other virtualization platforms as vCenter Operations also relies upon the vCenter API data.
Tiering Virtualization Platforms
You tier your storage for very good reasons. Those reasons have to do with matching the cost and the performance of the storage with the requirements of the workloads using the storage. The same thing makes sense when it comes to virtualization platforms as long as you do not drive yourself crazy trying to manage the resulting complexity. The great news with Hotlink is that Hotlink allows you to, for example, add Microsoft Hyper-V to your VMware vSphere environment, and keep using the same management products.
The graph below shows the result of such an approach at one Hotlink customer. Notice that the total number of hosts rises significantly over a 6 year period. However, the number of vSphere hosts goes down as they get reserved for only the most mission critical workloads, and all of the tactical workloads go to Hyper-V.
The economics of this are also very interesting. In the graph below, the top line is what the customer would have spent had they deployed all of the incremental workloads on vSphere. Since the customer in this example is a large Microsoft customer, Hyper-V is essentially free to this customer. So as workloads are deployed on Hyper-V instead, savings in the millions of dollars a year result.
If you are going to have more than one virtualization platform, you should not have more than one virtualization management stack. Hotlink SuperVISOR lets you use the market leading virtualization console (vCenter) to manage other virtualization platforms like Hyper-V, KVM, and XenServer. Furthermore it extends the vCenter API data used by third party management vendors with data from these other platforms, turning Hotlink into the management data broker for the virtualization management industry. Finally Hotlink allows for easily managed tiering of virtualization platforms and the fitting of platforms to their appropriate price/performance use cases
Cloud services are for the most part extremely attractive because they let the provider (vendor) of the cloud service focus upon what they do best (manage some set of layers in the hardware and software infrastructure), and let you focus upon what you do best (typically concern yourself with your applications and your data). Great examples of cloud services that promote agility include simple IaaS services like those available from Amazon or Rackspace, a PaaS service from Engine Yard or Heroku (now part of SalesForce.com), a SaaS service like SalesForce.com, or a storage service like DropBox. For a lot of use cases for for a lot of customers, these services are the computing equivalent of “Look Ma, no hands” in the sense that they deliver what you need with little effort on your part. Continue reading Can Your Cloud Impede Your Agility?→
One of the areas where VMware vSphere has had a large advantage over Microsoft Hyper-V has been in the architecture and capabilities of the VMware vSphere Virtual Switch. The most important of these capabilities has been the ability to designate one or more ports on the switch as promiscuous or mirror ports. These ports make a read only copy of all of the data flowing through the switch available to whatever virtual machine is sitting on that port. This is an essential capability for many security products and also performance management solutions like those from ExtraHop Networks that rely upon deep packet inspection to work. Continue reading The Windows Server 8 Hyper-V 3 Extensible Switch→
There have recently been a spate of articles and blogs that are attempting to create a contest between “Network Performance Management” tools and “Application Performance Management” tools. This includes a Network Computing survey that finds fault with APM solutions, and a SOA World Magazine comparison that tries to compare the two types of solutions. This is silly and unproductive. It is far more productive to approach this problem from the perspectives of what your needs and applications look like.