Splunk, the provider of the leading software platform for real-time operational intelligence, today announced it has acquired Cloudmeter, Inc., a provider of network data capture technologies. The addition of Cloudmeter will enhance the ability of Splunk customers to analyze machine data directly from their networks and correlate it with other machine-generated data to gain insights across Splunk’s core use cases in application and infrastructure management, IT operations, security, and business analytics.
Implications of Splunk Acquiring Cloudmeter
To understand what is possible and what is not possible with Cloudmeter, we have to start by understanding what data Cloudmeter can capture, and how it captures it. Cloudmeter is based on an agent that is a shim in the network stack of the operating system. So, Cloudmeter sees all of the network traffic that flows through the TCP/IP stack in a Windows or Linux PC. The existing Cloudmeter product was used to analyze HTTP traffic for things of interest to the an e-commerce business.
This method of collecting data has several advantages:
- Data collection works the same for every application, no matter whether it purchased or custom developed, or whether it runs on Windows or Linux.
- The data collection follows the operating system. This means that it works the same way if the operating system is deployed on physical hardware, is virtualized, or runs in a public cloud.
The areas of concern regarding collecting this data are:
- You have to install and maintain the Cloudmeter agent on every operating system instance from which you want data.
- You have to tell the Cloudmeter agent exactly what data you want. This means you have to first know what data you want, and then you need to make sure that you are not asking for so much data that the act of collecting creates a performance problem for the server on which the Cloudmeter agent is running.
- If you tell Cloudmeter to collect too much data, then this could become expensive, as Splunk charges by the amount of data that is ingested every day. By way of a simple example, if a web server is talking to one Chrome browser, that browser will be sending 500 bytes of data per second to the web server while in a quiet state, which is 43.2MB of data per user per day. One hundred concurrent users is not unusual, which brings the figure up to 4.32GB of data per day for one server. One hundred servers would put you at 432 GB per day, which would require one of the more expensive Splunk licenses, just for this one stream of data.
The points above mean that, absent Splunk’s building a solution of some kind around Cloudmeter, Cloudmeter will become a very interesting Swiss Army Knife of network data collection for a set of Splunk customers. That set will be the customers who build their own applications, understand them so well that they know what data to look for in the network between the components of their applications, and know how to ask for just that set of data so as to not suddenly make their Splunk license twice or five times as expensive as it was before.
Implications for Splunk Performance Management Partners
Splunk has a variety of important partnerships with third-party performance management vendors. The implications for these vendors are:
- AppDynamics – AppDynamics collects a completely different set of data than does Cloudmeter. AppDynamics collects data about custom-developed applications from the run time (Java, .NET, PHP, etc.) of the application. This data is extremely useful to the developers of these applications, as it contains the details as to where problems like in code. Those details are not available in the Cloudmeter data, and therefore there is no overlap between what AppDynamics does and what Cloudmeter can do.
- ExtraHop – ExtraHop has hardware appliances that sit on mirror/span ports of hardware switches. These appliances have the processing power to collect and parse data coming from those switches as it arrives, so both the overhead of collecting the data and the overhead of processing the data is offloaded from servers and switches to the appliances.
A crucial distinction applies to both AppDynamics and ExtraHop with respect to Cloudmeter. AppDyamics and ExtraHop have built complete performance management solutions targeting, respectively, the developer who has to support their custom application in production and the operations staff who have to support every application in production. Cloudmeter is by no stretch of the imagination a performance management solution at this point in time. It is merely a method for collecting some data that, when combined with other data, can be used to understand performance. But it will be left up to either the customer or Splunk (over time if they choose to do so) to build the intelligence that collects data from individual servers into an understanding of the performance of an application system.
The Strategic Issues for Splunk
With this acquisition, Splunk has created two strategic issues for itself. The first is that it must now decide if it is going to build a performance management product that competes with the likes of ExtraHop, AppEnsure, AppFirst, BlueStripe, and Correlsense. With its Splunk App for VMware, Splunk has already signaled that it intends to solve the Operations Management use case on top of its own platform. Competing with ExtraHop, AppEnsure, AppFirst, BlueStripe and Correlsense would commit Splunk to competing in the Operations-focused segment of the Performance Management business—something that to date has required the attention of an entire company of engineers and investors to do well.
The second issue comes down to pricing. An ExtraHop appliance can operate at a line rate of 20 GB/second. Collecting the same quantity of data with CloudMeter agents would create 1,728 GB (1.728 Terabytes) of data per day, which would then require a very expensive Splunk license. Therefore if Splunk wants to go down the road of adding the largest data set on the planet to its data store, Splunk is going to have to revisit it pricing.
Splunk has acquired Cloudmeter, giving Splunk the ability to add custom-configured slices of network data to its data store. There are likely numerous use cases for Splunk that will be enhanced by this capability, which will solidify Splunk’s position as the leader in the new ecosystem-based management software business.