CliQr Addresses Cloud Price/Performance with Free Cloud Benchmarking Service

For the entire (brief) history of the cloud computing business there has been too much focus upon infrastructure (IaaS) and platform (PaaS) and not enough focus upon applications. CliQr is one of a set of important new cloud management vendors whose offerings focus upon deploying actual applications in on-premise and cloud environments. Other vendors with this focus include ServiceMesh, Cloud Sidekick, Elasticbox, and, of course, VMware with vFabric Application Director.

The CliQr Cloud Price Performance Benchmarking Service.

Today, CliQr has announced a free cloud price/performance benchmarking service that allows enterprises to see how various types of applications perform on various clouds with various kinds of instance configurations. Included in this benchmarking service is what it would cost to run that application on that cloud using the configuration of the cloud services chosen for the benchmark.

“The reality is that not all clouds were created equal,” said Gaurav Manglik, CEO and co-founder of CliQr Technologies. “All types of infrastructure, operation, and geographic variances can affect an application’s performance on a specific cloud environment. On most clouds time is money and because of this diversity, simply using a spreadsheet to assess price-performance is not possible without understanding the application’s actual comparative performance on each cloud. How fast does it run? What does it cost? It doesn’t get any more straightforward, and it couldn’t be more important. With today’s announcement, users can select from a series of reference applications and sample data sets covering a variety of use cases including web applications, Hadoop, batch, and cluster. Users can select from one or more clouds and cloud environments to deploy and/or perform a benchmark of these reference applications at no cost. Users can decide to on-board and benchmark their own application(s) across a broader collection of CliQr-supported clouds including Amazon Web Services, HP Cloud Services, Microsoft Azure, Rackspace, IBM Smart Cloud Enterprise, OpenStack, CloudStack, VMware, and Google Compute Engine.”

 Price/Performance Optimization in CliQr CloudCenter

CliQr takes the concept of price/performance even further in its CloudCenter products. In these products, you can choose how you want your application deployed from the best price/performance, the best performance, or the best price. The CloudCenter product then uses its knowledge of your application and its knowledge of the performance and pricing plans of the various clouds to make an optimal placement decision.


The Importance of Cloud Price/Performance

In order for cloud computing to take off, two new important pieces of ground need to be broken. The first is that the performance of applications needs to be redefined away from resource utilization to response time, throughput, or job completion time (the version of response time relevant to a batch job). CliQr is able to measure throughput and job completion time itself, but it is reliant upon the customer to have instrumented the application with an APM solution to get response time. Hopefully the role of application response time in understanding cloud price/performance will be an impetus for more customers to instrument more applications with APM solutions that can measure response time and throughput.

Once you understand the response time and throughput (or job completion time) characteristics of your application, combining these metrics with what it costs to achieve various levels of response time and throughput has the promise to revolutionize many disciplines in IT. For example:

  • Capacity management can and should cease to be a question of resource utilization. If you are getting the response time and throughput that you need at the least possible cost you are not out of capacity.
  • IT spending and budgeting can be completely revolutionized. Right now huge amounts of money are spent in the hope of achieving certain levels of performance, but no one ever takes the time to look at the trade-offs between how much money is spent and what levels of response time and throughput are attained.
  • The business will be able to objectively shop around for the best execution environment for their applications (security considerations aside). This will put enormous pressure on IT to measure their costs and to be able to charge back on terms comparable to external (cloud based) execution environments.
  • Cloud computing will become quite a bit less opaque. Right now cloud vendors do a great job of hiding what goes on under the covers. There is simply no way that Amazon is going to tell you how congested one of the physical servers, physical switches, or storage devices that are handling your workloads are. But once you get to the point of being able to measure application response time, throughput, and cost, the customer gets the upper hand, because the customer can then say to Amazon, “I do not care what you do under the covers – just meet my response time and throughput goals at the advertised cost or else I am taking my business elsewhere.”
  • Needless to say, the ability of products like CloudCenter, Cloud Sidekick, ElasticBox, ServiceMesh, and VMware vFabric Application Director to move workloads between private, hybrid, and public clouds will be essential to the customer gaining the upper hand with respect to the cloud vendors. As more customers embrace these kinds of solutions, putting your application on one vendor’s cloud will cease to be like putting your application in that vendor’s jailhouse.
  • Transparency as to cloud/performance will ultimately create a market for cloud services in which customers (enterprises) can rationally shop for cloud services on the basis of price, performance, and hopefully soon security as well. Again, internal IT organizations will come under tremendous pressure to make their internal data centers competitive with the cloud services that can be transparently evaluated on price, performance, and throughput.
  • If response time and throughput are going to be two of the canonical metrics (cost being the other one) by which application placement decisions are going to be made, then this will drive the creation of a whole new segment of the APM industry. The new segment will consist of vendors whose products can actually measure response time and throughput for every application the customer owns. The early innovators in these new segment of the APM industry are AppEnsure, AppFirst, AppNeta, BlueStripe, Boundary, Correlsense, ExtraHop Networks, and INETCO. Less than 5% of the business critical applications in the world are instrumented for response time and throughput today, and these vendors stand the best chance of changing that.


Application response time and throughput are the right ways to measure application performance in virtualized and cloud based environments. When this information is combined with cost in a transparent manner across clouds, customers will be empowered to shop for cloud services on an intelligent and well-informed basis. This will shift power into the hands of the owners of applications, as they will then be able to demand performance, throughput, and cost transparency from all providers of execution services.

Posted in IT as a Service, SDDC & Hybrid CloudTagged , , ,