On Premise vs. Monitoring as a Service – Considerations and Tradeoffs

We all pretty much know that we can buy Infrastructure as a Service (IaaS), Development/Run time Platforms as a Service (PaaS), Software as a Service (Saas), Security as a Service, Cloud Storage as a Service, among other things – but we can also buy monitoring as a service. We can buy monitoring at both the infrastructure level and the application level as a service. This is an intriguing idea, and one that is rapidly gaining traction. However Monitoring as a Service (MaaS) carries with it some unique benefits, but it also carries with it some trade-offs especially when evaluated against on-premise solutions.

The Benefits of Monitoring as a Service

The main benefit of monitoring as a service has to do with the ease with which the solution can be set up and managed. Typically all you have to do is create an account on the web site of the MaaS vendor, download the agent that collects the data of interest, install that agent, and ensure that either or both ports 80 and 443 are open and you are off and running. However there are several other benefits of this approach that are worth considering:

  • Yes, MaaS is easy to set up, but it is also much easier to maintain than an on premise solution. You do not have to either set up or maintain the application server, the database server or the storage required to support an on-premise solution. If you are an application developer and what you want to do is monitor your application and what you do not want to do is install an entire computer and storage system to monitor your application, MaaS can be an ideal solution. Furthermore, the entire maintenance of the back end of the system is done by the MaaS vendor. They manage the database. They size the servers. They manage and size the storage. They are responsible for keeping it running, and most importantly, they are responsible for upgrading it when new versions come out.
  • New versions brings up the topic of how rapidly the vendor can evolve the product to meet changes in customer needs or simply add innovative new functionality. SaaS vendors can evolve their applications much more quickly than can vendors who have to upgrade 1000’s of customers.  The same is true for MaaS vendors. The MaaS vendor can upgrade the one instance of their back end, and all of a sudden 1000’s of customers have been upgraded through no effort on the customer’s part. Most SaaS and MaaS vendors practice Agile Development and can release new functionality to all of their customers on a monthly or even weekly basis.
  • Distributed applications are easily handled. If your applications are distributed across multiple data centers that you own, and you want them all on one pane of glass, you may find that it is a lot easier to point management agents in N data centers back at one hosted back end, then it is to point those agent at a management system internal to one of your data centers. The reason for this is that most companies find it to be relatively easy to open outbound only ports from inside their data center (in many cases they are already all open), than they find opening an inbound port in data center A so that agents in data center B can talk to the management system in data center A.
  • MaaS just makes a lot of sense for public cloud resident applications. If your application (or a part of it) runs in a public cloud, you probably do not want to install a management system in that cloud just to monitor your application. There are two reasons for this. The first is that you want to pay the public cloud vendor to run your applications, not systems that manage your applications. The second is that if the cloud vendor is having an issue that is hurting your application, you do not want that same issue to hurt the ability of your management system to detect that issue and help you diagnose it (or help you yell at your cloud vendor).
  • MaaS might be the right fit for your private cloud as well. Most MaaS solutions are designed for the agent to come up and register with the back end, with little to no back end configuration required. If you have a private cloud you can just embed the agents in your templates and master images, and when they are activated, monitoring of those applications can be automatically instantiated.
  • You can access your monitoring system from anywhere and from any device with browser. This might seem like an obvious point, but if the back end of your system is inside of your firewall, you are going to need to either open a port to get to it, or use a remote access solution like Citrix XenApp/Desktop or Microsoft RDS to get to it. If you use MaaS as long as you have a browser that can get out to the public internet, you will have access to your monitoring system.

Considerations and Trade-Offs with Monitoring as a Service

There is no such thing as the perfect solution for all needs, and so like everything else, MaaS comes with a set of caveats that you need to carefully consider before you choose it:

  • The security of your monitoring data. With a MaaS solution the data about your applications (their performance, reliability, and the issues with them) will travel over the public Internet (encrypted of course), and be stored in someone else’s database, and on someone else’s disks. This data is therefore inherently more accessible by “other people” than it would be if you implemented an on-premise solution and kept the data locked up in your own data center. If this is a concern then get the security policies and the admin policies that your MaaS vendor uses and assess them for robustness and compliance with your standards.
  • The security of your application data. It is possible to configure certain MaaS solutions so that they see user and application data and transmit that data back to the back end hosted by the MaaS vendor. In this case if your application is subject to PCI it is entirely possible to configure your MaaS solution (if it operates at the application level) so as to make you non-compliant with PCI. The obvious steps to take here are to configure your MaaS solution so as to mask that data, and to only give the right to make configuration changes to how you collect data to a few, carefully chosen and trusted people.
  • The reliability of the monitoring system. The bottom line with MaaS, as is the case with all internet delivered services is that you are relying upon the vendor of the service to have their act together when it comes to managing the uptime and availability of the system. Most software vendors are pretty good at developing and testing software. Unless a software vendor decides to invest in the skill, they are not going to be good at data center operations, and making sure that something is available 24×7. Part of your process of evaluating a MaaS vendor needs to be assessing their staff and their process for running the back end systems that you will be relying upon.
  • Real time data. This is an area that can completely disqualify a MaaS solution. If you need real time (meaning really real time, not the 5 minutes or even 5 seconds that some people claim is real time) data about the performance of your infrastructure or your application, MaaS may not be for you, or it may not be the right approach for the particular problem at hand. For example if you are need a solution like Virtual Instruments to see every transaction on your SAN or a solution like ExtraHop to see every transaction on your IP network, and you want anomalies reported to you instantly (in real time), MaaS is just not a fit.
  • Performance sizing of the monitoring solution. Some monitoring solutions allow you to set how frequently you collect data and how you roll it up. Many MaaS vendors have just one instance of their back end with a common set of settings for every customer. Some MaaS vendors host a unique back end for every customer and can do some customization in terms of sizing to support more rigorous data collection requirements. But it you need real time – see the above bullet – MaaS is probably not for you.
  • Customization of the monitoring solution. Some on-premise monitoring solutions can be heavily customized. It is fair to say that the most customizable MaaS solution is not nearly as customizable as most on-premise monitoring solutions.

Who’s Who in Monitoring as a Service

While there are many offerings in the Monitoring as a Service space, these offerings are by no means all of the same. They broadly split into offerings focused upon either applications or infrastructure, and then again split on whether they support a few kinds of custom developed applications and provide deep code diagnostics, or support a far broader range of applications at the expense of not providing deep code diagnostics (because deep code diagnostics are not terribly useful for purchased applications).  Five leading Monitoring as a Service vendors are profiled below. New Relic is the market leader in this group with over 20,000 organizations using their service (includes those using the free solution).

Vendor Scope of the Solution Target User  Network and Server Hardware Monitoring Server Resource Monitoring Virtualization (vSphere) Monitoring Application Monitoring End User Experience Monitoring Discovers Application Topology  Measures End-to-End Response Time 
AppDynamics Monitoring business services and transactions in production Java and .NET applications with deep code diagnostics Development teams supporting custom applications in production Red X green check Red X green check green check green check green check
AppFirst Monitor any Windows or Linux application. Applications Operations teams supporting a broad mix of applications in production Red X green check Red X green check Red X green check green check
CopperEgg Monitor the resource consumption metrics for any Windows or Linux based application Applications Operations teams supporting a broad mix of applications in production green check green check Red X Red X Red X Red X Red X
LogicMonitor Monitor a broad array of physical and virtual infrastructure with one one site probe that agentlessly uses SNMP and WMI to collect data Applications Operations teams supporting a broad mix of applications in production green check green check green check Red X Red X Red X Red X
 New Relic Monitoring production Java, .NET, Ruby, PHP, and Python applications with deep code diagnostics Development teams supporting custom applications in production Red X green check Red X green check green check green check green check


Monitoring as a Service is an attractive solution for many companies and situations – primarily those where the service provides the needed features and the customer wants to focus upon running their environment or their application instead of spending time and money maintaining the back end of a monitoring system. However, MaaS solutions carry with them caveats that need to be carefully evaluated as a part of making a MaaS decision.

Posted in IT as a ServiceTagged , , , , , ,