Performance Management covers monitoring the physical infrastructure, the virtual infrastructure and applications for end-to-end performance and service levels. It covers Application Performance Management, Infrastructure Performance Management, Operations Management, Capacity Planning, and Capacity Management. (Read More)(Read Less)
Environments covered include Virtualization Performance Management, Software Defined Data Center Performance Management, and Cloud Performance Management. Key issues include ensuring the performance of virtualized and cloud based data centers, ensuring the performance of software defined data centers (SDDC performance management), ensuring virtualized application performance, cloud application performance, and SDDC application performance. Key vendors covered include VMware, AppDynamics, AppEnsure, AppFirst, AppNeta, Astute Networks, Aternity, BlueStripe, Boundary, Cirba, CloudPhysics, Correlsense, Compuware, Dell, Embotics, ExtraHop, GigaMon, Hotlink, HP, Intigua, ManageEngine, New Relic, Prelert, Puppet Labs, Riverbed, Splunk, Tintri, Virtual Instruments, Virtustream, VMTurbo, Xangati, and Zenoss.
Back when APM got started, it was used to monitor complex applications that ran on relatively few servers and changed once a year or even less frequently. Now applications are distributed across thousands or even tens of thousands of servers, and they change daily. This requires management vendors to collect more types of data, and to collect this data more frequently which turns APM into a big data problem.
The AppDynamics Big Data Release
This week, AppDynamics announced its Summer 2014 release with a host of major new features. The most interesting of these features is that AppDynamics has decided to put its metric data into an open-source big data back end—Hadoop. This has several implications for the management software industry:
Like ExtraHop, which has just announced that it has “set [its] data free,” AppDynamics is now taking a leadership position in letting customers use their data for any use case imaginable by putting that data in an open-source database.
This sharpens the distinction between “commodity data” and “valuable data.” Commodity data is data that is collected by operating systems and devices and made freely available via management APIs, like the vSphere API, WMI, SNMP, and SMIS. Valuable data, like that which AppDynamics collects (detailed interactions of transactions with their application run times across an N-tier system), can only be collected the “hard way,” which is through world-class instrumentation designed with great care by people who really know what they are doing.
If ExtraHop and AppDynamics are willing to set their “valuable” data free, then what justification is there for a vendor that just collects commodity operating system or network statistics to lock its data up in a vendor-proprietary data store?
You can read more about the new AppDynamics release at the links below:
Splunk acquired Cloudmeter back in December 2013. Splunk App for Stream is the result of this acquisition. It gives Splunk customers the ability to parse network data and add that data to their Splunk datastores.
The Splunk App for Stream
The Splunk App for Stream consists of two components. An agent sits inside of the network stack of the operating system (Windows or Linux). All network traffic for that operating system instance passes through this agent, and it can capture any portion of that traffic and forward it to the Splunk datastore. The second component is a user interface that allows the user to specify the application from which to collect data and the fields within that stream for that application to capture. This is crucial to avoid overloading the Splunk datastore with the most voluminous type of data (wire data) and to avoid overrunning the license limits on the Splunk installation. As Leena Joshi, Splunk’s senior director of solutions marketing, explained:
“The Splunk App for Stream, the first product delivered from our acquisition of Cloudmeter last year, is a new approach that magnifies the Operational Intelligence organizations can gain with Splunk software…Unlike traditional and appliance-based solutions, which are difficult to deploy, especially in public cloud infrastructures, the Splunk App for Stream can be added to gain immediate wire data access on-premises or in public, private or hybrid cloud infrastructures. It opens up for our customers a whole new class of data sets to correlate for additional IT, security and business insights.”
The Application Performance Management, IT operations management, and security use cases for Splunk App for Stream are summarized as follows:
Where (and Where Not) to Use the Splunk App for Stream
The amount of wire data and Splunk’s pricing per amount of data ingested per day will make it prohibitively expensive to just dump all of the wire data from your hundreds or thousands of servers directly into your Splunk datastore. The good news is that Splunk gives you a very fine-grained way to control this with the user interface for Stream. However, the need and the ability to control the amount of data you ask App for Stream to collect and send to the datastore drives the use cases for this app. For example:
If you have a very small number of custom-developed applications that are critical to your business, and you know enough about them (since you built them) to know what data fields to expect on the wire, you can configure App for Stream to capture only the critical fields related to those critical applications. If you have hundreds or thousands of applications that are a mixture of purchased and custom-developed applications, then you need an AA-IPM solution, like those profiled in “Who’s Who in Application Performance Management for the SDDC and Cloud.”
If you are in IT Operations, App for Stream could be a valuable complement to Splunk’s App for VMware and the Apps for Citrix. If you know specific things represent problems in the network, you can set up App for Stream to look for them ahead of time, instead of running a trace and looking through a mountain of data after the fact.
The same holds true for security. If you know ahead of time what kind of an event on the network is associated with a security threat, you can set up App for Stream to find these for you instead of waiting for the event to happen and then doing a search.
This announcement also signals an important shift in strategy for Splunk. Prior to App for Stream, Splunk only collected data from management interfaces like syslog, SNMP, WMI, vSphere API, etc. Now Splunk has taken the extra step of collecting unique and valuable data that only vendors who specialize in this type of data collection provide. One can only speculate as to where this will lead.
Links to more information about Splunk App for Stream:
The Splunk App for Stream adds configurable slices of wire data to the Splunk datastore. This is a valuable additional source of data, but it is not on its own a complete network-based application performance, IT operations management, or security solution.
These days, just about every management product you might buy comes with one or more databases. This makes it extremely difficult to engage in analysis and correlation across datastores, which is one of the main reasons why management of IT operations is such a headache. ExtraHop has taken a bold step toward getting this problem fixed.
Over at readwrite.com, Matt Asay published a blog post entitled “In A World Of Open Source Big Data, Splunk Should Not Exist.” He then does a pretty good job of debunking his own thesis and explaining why customers continue to pay Splunk big bucks to do what it does. However, since there is so much noise around the question of open-source big data tools as alternatives to Splunk, this question deserves further exploration.
Attending Gigaom Structure was an exercise in getting fire-hosed with the leading edge innovation that public cloud providers are bringing to their customers worldwide. These innovations not only will have a profound effect on public cloud computing, but also will ultimately impact data center architectures, costs, and benefits worldwide.