I have written many times about the need for application-centric data protection and data-centric security. But what these both require is that our data protection, security, management, and networking are data-aware. We use applications, but we thrive on data. The more data we have, the more chance we can make use of it, which has resulted in big data tools and big data extensions, even to hypervisors. We talk constantly about moving data closer to processing, with flash and other techniques at the storage layer. But we have not grown other aspects of our systems to be data-aware. It is time this changed. Continue reading Data-Aware Services: Oh, the Places We Could Go!
Every time we as an industry come up with a wonderful innovation, we tend to deploy that innovation before we have the ability to manage it in production. This occurred with the first round of data center virtualization—and resulted in an entirely new category of operations management solutions. But these new solutions did not arrive until several years after CPU and memory virtualization had become widely adopted. Gigamon and VMware seem determined to break this cycle with their joint announcement addressing the question of NSX visibility.
It is that time of year again, when we see all the new toys, tools, ideas, and processes that make up the show called VMworld. This year, quite a few changes in virtualization security will be discussed by VMware and other organizations that work with virtual and cloud environments. One of the key messages will be that everyone needs to stop treating virtualization security as something unique and different. Instead of this type of treatment, we have been seeing the extension of existing tools and techniques into virtual and cloud environments. Virtualization and cloud security is a natural progression of all organizational security.
Continue reading Virtualization Security at VMworld
On the August 7 Virtualization Security podcast, we discussed how people in virtualization, security, compliance, data protection, storage, and networking—and everyone else in IT—should form their own organizational communities to improve overall communication and establish easy access to experts in those fields. This thought came out of a conversation I had with @jtroyer about whether or not IT should be a community instead of seeing its various components as silos. Even to this day, we are seeing more silos and fewer communities. The lines have just been drawn differently. Continue reading Building Your Own IT Community
Splunk acquired Cloudmeter back in December 2013. Splunk App for Stream is the result of this acquisition. It gives Splunk customers the ability to parse network data and add that data to their Splunk datastores.
The Splunk App for Stream
The Splunk App for Stream consists of two components. An agent sits inside of the network stack of the operating system (Windows or Linux). All network traffic for that operating system instance passes through this agent, and it can capture any portion of that traffic and forward it to the Splunk datastore. The second component is a user interface that allows the user to specify the application from which to collect data and the fields within that stream for that application to capture. This is crucial to avoid overloading the Splunk datastore with the most voluminous type of data (wire data) and to avoid overrunning the license limits on the Splunk installation. As Leena Joshi, Splunk’s senior director of solutions marketing, explained:
“The Splunk App for Stream, the first product delivered from our acquisition of Cloudmeter last year, is a new approach that magnifies the Operational Intelligence organizations can gain with Splunk software…Unlike traditional and appliance-based solutions, which are difficult to deploy, especially in public cloud infrastructures, the Splunk App for Stream can be added to gain immediate wire data access on-premises or in public, private or hybrid cloud infrastructures. It opens up for our customers a whole new class of data sets to correlate for additional IT, security and business insights.”
The Application Performance Management, IT operations management, and security use cases for Splunk App for Stream are summarized as follows:
Where (and Where Not) to Use the Splunk App for Stream
The amount of wire data and Splunk’s pricing per amount of data ingested per day will make it prohibitively expensive to just dump all of the wire data from your hundreds or thousands of servers directly into your Splunk datastore. The good news is that Splunk gives you a very fine-grained way to control this with the user interface for Stream. However, the need and the ability to control the amount of data you ask App for Stream to collect and send to the datastore drives the use cases for this app. For example:
- If you have a very small number of custom-developed applications that are critical to your business, and you know enough about them (since you built them) to know what data fields to expect on the wire, you can configure App for Stream to capture only the critical fields related to those critical applications. If you have hundreds or thousands of applications that are a mixture of purchased and custom-developed applications, then you need an AA-IPM solution, like those profiled in “Who’s Who in Application Performance Management for the SDDC and Cloud.”
- If you are in IT Operations, App for Stream could be a valuable complement to Splunk’s App for VMware and the Apps for Citrix. If you know specific things represent problems in the network, you can set up App for Stream to look for them ahead of time, instead of running a trace and looking through a mountain of data after the fact.
- The same holds true for security. If you know ahead of time what kind of an event on the network is associated with a security threat, you can set up App for Stream to find these for you instead of waiting for the event to happen and then doing a search.
This announcement also signals an important shift in strategy for Splunk. Prior to App for Stream, Splunk only collected data from management interfaces like syslog, SNMP, WMI, vSphere API, etc. Now Splunk has taken the extra step of collecting unique and valuable data that only vendors who specialize in this type of data collection provide. One can only speculate as to where this will lead.
Links to more information about Splunk App for Stream:
- Splunk App for Stream guide on SlideShare
- Splunk App for Stream overview
- Splunk App for Stream fact sheet
- Splunk App for Stream press release
The Splunk App for Stream adds configurable slices of wire data to the Splunk datastore. This is a valuable additional source of data, but it is not on its own a complete network-based application performance, IT operations management, or security solution.
Over at readwrite.com, Matt Asay published a blog post entitled “In A World Of Open Source Big Data, Splunk Should Not Exist.” He then does a pretty good job of debunking his own thesis and explaining why customers continue to pay Splunk big bucks to do what it does. However, since there is so much noise around the question of open-source big data tools as alternatives to Splunk, this question deserves further exploration.