In the world of managing systems, networks, servers, operating systems, virtualization, application performance, and end user experience, the “single pane of glass” has been a holy grail. Many organizations have spent so much time and money pursuing this goal that the pursuit itself has turned into a single glass of pain.
The “Single Pane of Glass” Holy Grail
Many IT organizations have spent years and fortunes pursuing the notion of a “single pane of glass,” aided and abetted by vendors’ marketing departments who promised them just that. There have been only a couple of problems with this goal:
- Most organizations believed IBM, BMC, HP, and CA when they promised their customers that their management frameworks would in fact deliver such a “single pane of glass.” The only problem was that these management frameworks were themselves Franken-Monitors, assembled through acquisition and with their component products never fully integrated into the whole. Ask any framework vendor how many different databases and consoles the product is comprised of, and you will get the ugly truth.
- While the frameworks looked comprehensive on paper, they never comprised a best-of-class set of components. Furthermore, as innovations occurred, such as new languages, new types of networks, new types of storage devices, data center virtualization, hybrid and public clouds, and, finally, Agile Development and DevOps, these frameworks fell behind, never to catch up.
- This caused customers to fill in the gaps in the frameworks with point tools that addressed the new technologies and processes in their environments. This resulted in most customers’ having the worst of all possible worlds; a framework that was out of date and many (between 50 and 200) different point tools.
- The quest for the single pane of glass started as customers attempted to implement a “Manager of Managers.” The idea here was to feed the alerts from each point tool into a master console, which would aggregate the alerts, attempt to sort them by priority, and try to provide insight as to the root cause of the alert.
- But since the thresholds for the alerts were set by many different human beings using many different criteria for what was normal and what was not normal, the quality of the alert stream was suspect. This resulted in IT Operations teams wasting countless hours trying to figure out which alerts to pay attention to.
The Single Glass of Pain
The mess articulated above has led to a situation in which no one in IT Operations really knows what is going on, what real problems are occurring, what events are being surfaced that some tool claims are problems but are really false alarms, and what is the best way fix problems once it is determined that they really are problems.
A Glass of Cold Beer Instead
To address this issue, management of performance, availability, and reliability needs to be re-thought and re-implemented:
- Start with high-quality, real-time, deterministic, and comprehensive data. Most of the management data that is collected by frameworks and commodity management tools from standard management interfaces is already averaged before it is collected, it is not collected frequently enough, and it does not represent the real state of the item from which the data is collected.
- Invest in vendors that do the hard work to collect the real data that represents the real state of what they claim to manage.
- Forget about a single pane of glass. It is never going to happen. There are too many different use cases for management data for it to be possible to satisfy all of these different constituents with one console.
- Understand that management data is a multivendor big data problem. No single vendor is ever going to be able to collect all of the data you need. Instead of integrating consoles and integrating alerts, you should integrate the data.
Who Is Doing This Right?
Splunk got this party started by creating a big data back end and a partner program that encourages third-party vendors to put their data into Splunk. But Splunk has recently started showing signs that it intends to compete with some of its partners. For example, the Splunk App for Stream was positioned by Splunk as a direct competitor to Splunk partner ExtraHop.
ExtraHop has reacted by delivering an Open Data Stream architecture wherein, initially, ExtraHop supports open source data stores like MongoDB and Elasticsearch. ExtraHop and AppDynamics have also just recently announced the bidirectional nature of their two valuable sets of data.
Data integration will replace console integration and alert integration in the management software industry. This is the only path by which customers will be able to replace the current glass of pain with something that actually works.