Splunk acquired Cloudmeter back in December 2013. Splunk App for Stream is the result of this acquisition. It gives Splunk customers the ability to parse network data and add that data to their Splunk datastores.
The Splunk App for Stream
The Splunk App for Stream consists of two components. An agent sits inside of the network stack of the operating system (Windows or Linux). All network traffic for that operating system instance passes through this agent, and it can capture any portion of that traffic and forward it to the Splunk datastore. The second component is a user interface that allows the user to specify the application from which to collect data and the fields within that stream for that application to capture. This is crucial to avoid overloading the Splunk datastore with the most voluminous type of data (wire data) and to avoid overrunning the license limits on the Splunk installation. As Leena Joshi, Splunk’s senior director of solutions marketing, explained:
“The Splunk App for Stream, the first product delivered from our acquisition of Cloudmeter last year, is a new approach that magnifies the Operational Intelligence organizations can gain with Splunk software…Unlike traditional and appliance-based solutions, which are difficult to deploy, especially in public cloud infrastructures, the Splunk App for Stream can be added to gain immediate wire data access on-premises or in public, private or hybrid cloud infrastructures. It opens up for our customers a whole new class of data sets to correlate for additional IT, security and business insights.”
The Application Performance Management, IT operations management, and security use cases for Splunk App for Stream are summarized as follows:
Where (and Where Not) to Use the Splunk App for Stream
The amount of wire data and Splunk’s pricing per amount of data ingested per day will make it prohibitively expensive to just dump all of the wire data from your hundreds or thousands of servers directly into your Splunk datastore. The good news is that Splunk gives you a very fine-grained way to control this with the user interface for Stream. However, the need and the ability to control the amount of data you ask App for Stream to collect and send to the datastore drives the use cases for this app. For example:
- If you have a very small number of custom-developed applications that are critical to your business, and you know enough about them (since you built them) to know what data fields to expect on the wire, you can configure App for Stream to capture only the critical fields related to those critical applications. If you have hundreds or thousands of applications that are a mixture of purchased and custom-developed applications, then you need an AA-IPM solution, like those profiled in “Who’s Who in Application Performance Management for the SDDC and Cloud.”
- If you are in IT Operations, App for Stream could be a valuable complement to Splunk’s App for VMware and the Apps for Citrix. If you know specific things represent problems in the network, you can set up App for Stream to look for them ahead of time, instead of running a trace and looking through a mountain of data after the fact.
- The same holds true for security. If you know ahead of time what kind of an event on the network is associated with a security threat, you can set up App for Stream to find these for you instead of waiting for the event to happen and then doing a search.
This announcement also signals an important shift in strategy for Splunk. Prior to App for Stream, Splunk only collected data from management interfaces like syslog, SNMP, WMI, vSphere API, etc. Now Splunk has taken the extra step of collecting unique and valuable data that only vendors who specialize in this type of data collection provide. One can only speculate as to where this will lead.
Links to more information about Splunk App for Stream:
- Splunk App for Stream guide on SlideShare
- Splunk App for Stream overview
- Splunk App for Stream fact sheet
- Splunk App for Stream press release
The Splunk App for Stream adds configurable slices of wire data to the Splunk datastore. This is a valuable additional source of data, but it is not on its own a complete network-based application performance, IT operations management, or security solution.