Tag Archives: Application Performance Management

New Application Performance Management for Virtualization and Clouds White Paper Released

PerformanceManagementThe Virtualization Practice has released a major update to its Application Performance Management for Virtualization and Cloud White paper. This paper covers both Application Performance Management for custom developed applications (DevOps), and Application Performance Management for every application (purchased and custom developed) in production (AppOpps). Continue reading New Application Performance Management for Virtualization and Clouds White Paper Released

APM for Agile and DevOps: 7 practical use cases

APM for AgileApplication Performance Management (APM) solutions are historically monolithic systems used by IT operations for monitoring the performance of production applications. But this trend is changing quickly. A combination of agile and DevOps methods combined with cloud computing and a new generation of DevOps focused APM products are adding value throughout the agile development lifecycle – far beyond application support.

Legacy APM tools are generally expensive, complex, require lots of configuration to get working and are fragile when application ecosystems change. They were designed for applications running on physical hardware that rarely change – the antithesis of agile development in the cloud. These new breed of APM products including AppDynamics, New Relic, dynaTrace, Foglight and VMware vFabric APM amongst others still do what their predecessors did, but now they’re increasingly being embraced by developers, testers and DevOps team members. These people are using APM to add value during analysis, design, development and testing phases too.

This article highlights specific ways agile teams are using APM outside IT operations, starting from a project’s kickoff.

#1 – Kickoff

APM Setup Checklist

As a part of starting a new project or product, many agile teams do a Sprint 0. This Sprint is primarily focused on technical tasks and planning so that new feature development can begin in Sprint 1. While the product team is focused on writing the initial user stories, the technical team can focus on hooking up APM to their application while integration is still simple. This, along with other stories for continuous integration, code management can be done.

These steps might be largely manual at first or if they occur rarely. However, if setting up new applications or projects is a common than these can be largely automated. Tools like Puppet can script the installation of the controller and/or agents as a part of environment setup. Agent configuration file(s) can be packaged with application frameworks so new applications are automatically configured to communicate with the controller.

If you’re developing in a PaaS solution such as Heroku or Azure, integrating a new application with APM is even simpler. Just enable in your PaaS management dashboard and you’re set.

#2 – Analysis

Most agile teams capture functional requirements in the user story format: As a <Persona>, I would like to <Do Something>, so that I can <Achieve Some Result>

NFR for Performance

Along with each user story are a set of acceptance criteria that defines specifies when a user story is “done”. Often acceptance criteria may include non-functional requirements related to performance, scalability and resiliency. Teams often struggle with how to test these during development, and area where APM can help.

Defining a non-functional requirement can be as simple as as the example above. The method alone clearly stakes which APM metric from which dashboard, under what amount of load and for what percentage of users. This clarity of requirement not only sets expectations with internal stakeholders including architects and developers, it’s also something that can be easily integrated into a business-facing dashboard built in the APM product during development and available after release.

#3 – Design & Development

During development APM is a supplemental tool to the team, not something necessarily used everyday but a very handy tool for running down issues, such as why a particular test failed or why page load times are slow for single users. They can also identify poor design decisions and raise them to the surface, something that occurred on a recent project I audited.

Two companies were charged with building a new customer service web site for a client. One focused on the CMS and web application. The other on creating web services to expose data in legacy systems to customers. During design the legacy team insisted on creating fine grained web services because it was the least expensive option and easiest to do. The project manager agreed and teams moved forward. The web team’s user story requirements called for a year’s worth of transaction data to be displayed on the web page. The web team  implemented the story, making the right API calls, and the functional tests passed.

Soon thereafter developers and testers started complaining about the slow load times. They hooked up their APM tool and discovered the offending page was making 117 web service calls to get the required data to load the page. Although each call was less than 400 milliseconds, the sheer number made the performance horrible. The web team brought this information to the legacy team’s attention, showed them the APM data. Then they quickly worked out a single API call that took additional parameters but made the integration much simpler and faster. Once each team refactored their code the page load time dropped dramatically.

Solving integration problems like these are a part of a developer’s life. Without the right tooling, some of these problems may take days or weeks to resolve and potentially result in a whole bunch of hand-written diagnostic code.

The new breed of APM tools get this reality and are increasingly focusing their attention on developers and not just operations. In addition to free developer versions of their products, APM vendors are forming partnerships with PaaS vendors (such as New Relic with Heroku and AppDynamics with Azure) to make integrating monitoring into your applications very simple.

#4 – Functional Testing

Agile teams leverage automated testing as a normal part of their sprints. While developers focus on automated unit testing, quality assurance typically focuses on integration and acceptance testing. During each sprint APM is hooked up to the application in the test environment(s). If one of the automated tests fail at a specific time, the corresponding snapshot of the failed transactions at that same time can be captured from APM to enable further analysis. Tools like Splunk can also help here as well, enabling developers and testers to collaboratively solve issues uncovered by testing – especially tough ones such as bugs that are only reproducible under certain conditions.

#5 – Performance Testing

This is one of the most popular uses of APM as it helps architects, developers and testers answer questions such as: what really happens to the application under load and can the application support the customer demand?

Modern APM’s are essentially application profilers that have such a low overhead they can run all the time without negatively impacting an application’s performance. Gone are the days of hooking up an application profiler, running tests and having your results skewed greatly because of their invasive overhead. Today’s APM tools give developers the same drill down capabilities – such as identifying the problem line of code or SQL statement – that profilers traditionally provided but without the overhead and extra setup.

What this means is that during performance tests, a team can in real-time watch the application’s performance under load and diagnose issues on the fly. They can also save off results for post-analysis. APM’s are useful for clearly identifying bottlenecks and limitations on scalability. They bring these issues to the attention of the team who are in the best position to fix them, whether this be tweaking a configuration parameter or refactoring code. They’re also useful for recording previous performance test runs so teams can do comparisons between releases to look for any subtle trends.

It’s no secret that faster applications generate more revenue and better customer experience. Amazon notes that every 100 millisecond drop in response times yields a 1% sales decline (that’s a $200M+ potential revenue impact). Google notes that a 500 millisecond drop in response times results in 20% less search traffic. This means the new APM tools are delivering value not only by reducing resolution times (costs), but also improving performance (revenues) – a great position for any product.

#6 – Production Deployment

APM is particularly helpful during new releases to production, which can be nerve-racking events themselves. Most vendors have a way to indicate a change to production such that post-release metrics (such as response time) can be compared with pre-release metrics. Should something look amiss or there’s a performance problem identified, the decision can be made to quickly rollback and investigate. Data from the APM tool can be used as a part of this analysis to figure out what went wrong before attempting the next release.

This same basic process can be applied to teams practicing continuous deployment. But instead of relying on humans to do the release analysis, this is automated so only exceptions are raised to the attention of humans, otherwise the same post-launch checks are all done via automated tests and validated in part using APM. Should issues arise, workflow scripts can be created to send issues to the organizations incident management system. A table of popular DevOps competent APM tools for use in dynamic and cloud based environments is below.

DevOps Focused APM Tools

Vendor/Product Product Focus Deployment Method Data Collection Method Supported App Types Application Topology Discovery Cloud Ready “Zero- Config” Deep Code Diagnostics
AppDynamics Monitor custom developed Java and .NET applications across internal and external (cloud) deployments On Premise/SaaS Agent inside of the Java JVM or the .NET CLR Java/.NET green check green check green check green check
dynaTrace (Compuware) Monitoring of complex enteprise applicatons that are based on Java or .NET but which may include complex enterprise middleware like IBM MQ and CICS On Premise Agent inside of the Java JVM or the .NET CLR Java/.NET, Websphere Message Broker CICS, C/C++

green check

green check green check green check
New Relic RPM Monitor custom developed Java, .NET, Ruby, Python, and PHP applications across internal and external (cloud) deployments SaaS Agent inside of the Java JVM, NET CLR, or the PHP/Python runtime Ruby/Java/ .NET/PHP/Python green check green check

green check

green check
Quest Foglight Monitor custom developed Java and .NET applications and trace transactions across all physical and virtual tiers of the application On-Premise Agent inside of the Java JVM or the .NET CLR Java/.NET green check  green check green check
VMware vFabric APM Monitor custom developed Java applications in production. Strong integration with the rest of the VMware product line including automated remediation and scaling. On Premise Mirror port on the vSphere vSwitch and an agent inside the Java JVM HTTP/Java/.NET/SQL

green check

green check

#7 – Support

This is the traditional use case for APM and still the most popular: helping operations teams reduce incident resolutions times. This may be in real-time as an incident occurs or during post-incident analysis looking for clues as to what went wrong. Often times this includes looking into slow or failed transactions to identify root causes. I’ve known teams to use APM to discover the 5am database back-up job is causing application performance to degrade.

From SLA management to operational dashboards, these newer APM tools still support their core operations administrator and help-desk engineer. But with increased simplicity and more intuitive user interfaces, these new APM adding value beyond their traditional support role.


A new breed of DevOps focused APM tools is moving performance management outside the domain of operations. With features to support analysts, architects, developers, testers and DevOps APM is at home in all phases of agile development.


News: AppDynamics and New Relic Prepare to go Big Time in the APM Market

PerformanceManagementAppDynamics has just raised $50m and New Relic has just raised $80m, both in preparation for going public. The legacy APM vendors are about to have a really serious problem. These funding rounds prove that some of the smartest investors in the world now believe that virtualization, cloud computing, new languages, and dynamic run time environments combine to create both a brand new set of requirements for a relevant management stack and the opportunity for a brand new set of vendors to be both the platforms for that management stack and the foundations of that new management stack.

Continue reading News: AppDynamics and New Relic Prepare to go Big Time in the APM Market

Virtualizing Business Critical Applications – Managing Applications Performance

PerformanceManagementThe most frequently encountered barrier to virtualizing business critical applications is a “concern” on the part of the application owners that the applications will “not run as well”, or “not perform as well” in a shared and dynamic virtual environment as it does in a dedicated physical environment. Depending upon who has what political power, these concerns can stop the project to virtualize these applications dead in its tracks. Continue reading Virtualizing Business Critical Applications – Managing Applications Performance

The Third Generation of APM

PerformanceManagementRecently it has become abundantly clear that there is great turmoil in the business of Application Performance Management (APM). CA has sponsored a study by IDG research services  that concluded that “Most Enterprises are Approaching APM SaaS Cautiously“. Separately, Information Week asked “What’s Killing APM”, and concluded that “App performance management is seen as less important than it was two years ago, partly because vendors haven’t kept up”. These problems are being caused by first and second generation APM solutions and are driving the market for third generation APM solutions. Continue reading The Third Generation of APM

Your VMworld 2012 Virtualization Management Sponsors and Exhibitors Short List

VMworld2012150x27VMworld 2012 is upon us and one of our tasks is to figure out which vendor’s booths to go see. With over 230 booths to choose from this is a daunting task. If you are interested in finding creative new solutions to your management, monitoring, deployment, security, data protection, and desktop management problems, this list will help you.

Virtualization Management Categories Defined

Here are the definitions of the eight virtualization management categories profiled below:

  • Application Performance Management (APM) – APM is about the end-to-end and hop-by-hop (across application tiers) measurement of response time and the diagnostics required to pinpoint degradations in response time (or flat out failed transactions) in the applications themselves or in the supporting infrastructure. APM tools come in two varieties. Developer focused tools help developers (or DevOps teams) support custom developed applications in production by quickly identifying and diagnosing application code problems in production. Operations focused tools  support every application in the environment (custom developed and purchased), and focus their diagnostics more on infrastructure issues that are impacting application performance.
  • Operations Management – Operations Management is a broad category of products that are used to support the day-to-day performance, capacity and configuration management tasks that face virtualization administrators. While all of these products support vSphere, some support other hypervisors as well.
  • Infrastructure Performance Management – IPM is APM for the infrastructure. It is all about the end-to-end and hop-by-hop latency of the infrastructure in support of the workloads running on the infrastructure. The thesis of this category is that in a virtual environment you cannot infer the performance of the infrastructure from resource utilization metrics, you have to measure it directly and continuously.
  • Automated Server and Image Management – This category has come into its own this year. The focus is upon allowing you to automatically manage what runs on your servers (physical, virtual or cloud), update them at scale, and keep them consistent. Think of this category as BladeLogic Version 2.0.
  • Cloud Management – Cloud Management is about building clouds on your vSphere infrastructure, and extending those clouds to other hypervisors, as well as to public cloud infrastructures.
  • Virtualization Security – Virtualization Security is about protecting the infrastructure, the systems software, the middleware, the applications, and all data from unauthorized use or attacks.
  • Virtualization Backup and Data Protection – Backup and Data Protection ensure that your data is always available for you (and no on else), irrespective of what failures or disasters have occurred in or to your IT environment.
  • Desktop Virtualization – Desktop Virtualization is about using virtualization as a catalyst to combine the benefits of user flexibility and centralized management.

Your VMworld 2012 Short List

Vendor Product Category What Makes Them Special New at VMworld 2012 Booth #
Application Performance Management
Aternity Frontline Performance Intelligence Platform Application Performance Management Provides a unified view of end user experience management across physical, virtual, and mobile environments Management of View end user experience through the correlation of PC-over-IP® (PCoIP®) latency with the actual business transaction response times and the VM image’s health. 329
ExtraHop Networks Application Delivery Assurance
Application Performance Management Performance Management of every application in the environment across virtual and physical tiers with no agents User application performance geo-mapping, solution bundles, and flex grids for virtualization management. Subscription pricing for virtual appliances. 2335
New Relic RPM Application Performance Management SaaS delivered APM for Java, .NET, Ruby, PHP and Python applications 529
VMware vFabric APM Application Performance Management Support for every application through a virtual appliance on a mirror port, and support for deep dive Java diagnostics via a Java agent
 Operations Management
Astute Networks ViSX Operations Management Simple to deploy network attached Flash Storage optimized for VMware and deployable by the VMWare Adminstrator ViSX G4 High-Performance VM Storage Appliance 435
Cirba Cirba Operations Management Sophisticated predictive analytics for controlling capacity, resource allocations and placements. 623
CloudPhysics CloudPhysics Operations Management The CloudPhysics platform helps you make the best decisions in the face of continual change. We make the invisible visible – revealing otherwise hidden structures and relationships in your datacenter. Launch of the CloudPhysics service. Fusion-io 2201
Hotlink SuperVISOR Operations Management Extends VMware vCenter to fully manage Hyper-V, KVM, XenServer, CouldStack and Amazon EC2 – no additional console required. Management of EC2 and CloudStack from VMware vCenter 2422
Intigua  Intigua Operations Management Virtualizes your Management Agents so that their impact is minimized and so that they are easily managed at scale 428
ManageEngine  IT360 Operations Management Agentless monitoring for applications and servers along with single pane of glass view across physical, virtual and cloud IT infrastructure. Privileged password management for VMware vSphere, capacity planning reports and automated resource optimization for virtualization software. 2340
PHD Virtual  Virtual Monitor Operations Management End to end virtual infrastructure monitoring for vSphere and XenServer & XenDesktop environments. 314
Reflex Systems Virtualization Management Center Operations Management Real-time, streaming data analysis including customizable analytics, charts and graphs that give enterprises extremely granular visibility and pure data fidelity for managing virtual infrastructure.  Reflex and ExtraHop partner to drive IT automation in virtualized datacenters and cloud infrastructure by delivering integrated infrastructure and application context to performance management. 517
Splunk Splunk App for VMware Operations Management Unlock the value of vSphere data and get granular performance management and analysis, as well as security reporting and monitoring, complete operational visibility and capacity analytics. The new Splunk App for VMware  1909
VKernel vOps Manager Operations Management Provides control over virtual environments and cloud deployments with analytics, advice and automation Performance impact modeling for environment changes, Automated environment change implementation, VM issue diagnosis screen for all problems affecting a VM, VMware vCloud Director integration 607
VMTurbo Operations Manager Operations Management Ability to guarantee resource allocations based upon workload priorities

Integrated configuration management with OpenStack


VMware vCenter Operations Operations Management Automated operations management solution with self-learning analytics, application awareness and integrated performance, capacity and configuration management for virtual and cloud infrastructure. Full transaction tracing and Closed loop integration from provisioning to monitoring to auto scaling (In Beta)  CAP1
Zenoss Service Dynamics Operations Management A single product that delivers end-to-end service assurance for real-world, hybrid IT that spans physical, virtual and cloud-based infrastructure.


 Infrastructure Performance Management
Gigamon GigaVUE Fabric Manager Infrastructure Performance Management Gigamon empowers infrastructure architects, managers and operators with visibility into the physical and virtual networks without affecting the performance or stability of the production environment. GigaVUE-FM provides the platform to manage distributed GigaVUE-VM virtual fabric nodes on virtual and physical infrastructures. 1716
Virtual Instruments VirtualWisdom  Infrastructure Performance Management
Real time, Deterministic and Comprehensive Health and Latency Metrics from the VM to the LUN  
Xangati Xangati Management Dashboard (XMD) Suite Infrastructure Performance Management Live and continuous tracking of infrastructure performance health (server+hypervisor, storage and network). Problems identified within first hour of install. StormTracker: Industry’s first solution to track and anticipate transient, performance storms that fly under the radar of conventional monitoring solutions.  2413
 IT Automation
Puppet Labs Puppet Automated Server and Image Management IT automation software that gives system administrators the power to easily automate repetitive tasks, quickly deploy critical applications, and proactively manage infrastructure changes, on-premise or in the cloud. The Puppet integration service for VMware vFabric Application Director provides customers with the productivity and agility benefits of software-defined infrastructure.  2323
Opscode Chef Automated Server and Image Management Chef is an open-source systems integration framework built specifically for automating the cloud. Integration of Opscode Chef with VMware vCloud Director as well as vCloud integrations with Terremark and Bluelock, enabling users to deploy and automate vCloud infrastructure in private/public clouds.  2431
rPath  Cloud Engine Automated Server and Image Management Automate the assembly, provisioning and change of OS platforms, middleware platforms and entire application stacks to deploy, not in weeks, but in minutes Unveiling of ECAF model removes chaos from cloud transformation & plugs IT pros into power of cloud with positive business results. Pragmatic approach embeds need for standardization & guides journey.  2441
 Cloud Management
Cloupia Unified Infrastructure Controller Cloud Management A multi-hypervisor and multi-cloud provisioning and management solution that provides physical and virtual infrastructure control, management and monitoring Unified Converged Infrastructure Orchestration Solutions – FlexPod & VSPEX 100
DynamicOps Cloud Suite Cloud Management Management of private, hybrid, and public clouds across multiple hypervisors and virtual/physical infrastructures Just got acquired by VMware 512
Embotics V-Commander Cloud Management Private Cloud In a Box – Deployed in Less than One Hour IT as a Service functionality: Request multiple virtual or physical assets from a single entry in the service catalog. 616
ManageIQ  EVM Suite Cloud Management Enterprise cloud management solutions that enable IT services to be easily provisioned and managed across physical, virtual, private and public cloud infrastructures.  Major new release announcement with support for hybrid clouds including leading public clouds like Amazon and Rackspace. 506
Virtustream XStream Cloud Management A cloud for business critical, performance critical and mission critical applications like SAP xStream software GA for enterprises, service providers and governments: private, public and hybrid cloud management for existing hardware/virtualization with security and legacy application SLAs NetApp Booth 1402
Virtualization Security
 AFORE Solutions CloudLink Virtualization Security AFORE’s CloudLink® provides encryption for active Cloud Storage and Virtual Hosted Applications. CloudLink® protects mission critical data while managing security compliance and service performance. AFORE’s announces: CloudLink® 2.0 – Secure Virtual Storage Appliance; CloudLink CX Encryption of End-User Data in VDI and Virtual Hosted Apps; CloudLink Achieved Secured by RSA® Partner Certification 2428
HyTrust HyTrust Virtualization Security HyTrust empowers organizations to virtualize more by delivering enterprise-class controls for access, accountability, and visibility to their existing virtualization infrastructure. 2129
Trend Micro Deep Security &
Secure Cloud 
Virtualization Security Trend Micro Incorporated, creates a world safe for exchanging digital information with Internet content security and threat management solutions. 1123
Symantec Critical System Protection
Virtualization Security Critical System Protection w/vSphere integration. CSP adds an multi-layer security to critical systems such as VMware vCenter as well as enforcement of security hardening of vSphere.  Deep Security 9 offers better protection, performance & ROI for virtual servers & desktops & with our new Trend Ready program you can confidently pick a Deep Security compatible cloud service provider. 909
Backup and Data Protection
Quantum  vmPro Virtualization Backup Quantum addresses data protection and big data management, providing specialized storage solutions for physical, virtual and cloud environments to maximize the value of data. 2017
Veeam Veeam Backup & Replication Virtualization Backup VM admins like Veeam because it’s fast, reliable and easy to use. Eliminates headaches of traditional backup, including agents, backup job failures and multiple backups. 100% virtualization-focused. Restore VMs from SAN snapshots. Fast: < 2 min. Flexible: Individual VMs, guest files or Microsoft Exchange items. Free: worry-free, agent-free and literally free (included in upcoming 6.5 release). 1709
Zerto Virtual Replication Virtualization Backup Zerto, VMworld Best of Show 2011 winner, provides the only hypervisor-based business continuity/disaster recovery solution for business-critical application protection in virtual & cloud environments. Zerto Virtual Replication 2.0 is Live – featuring multi-site replication and secure multi-tenancy. Plus, over 35 Cloud Providers begin offering cloud Disaster Recovery services powered by Zerto. 624
Desktop Virtualization
Appsense  User Virtualization Platform Desktop Virtualization AppSense reduces IT complexity and enables enterprise consumerization with independent management of the user experience across all mobile devices and desktops. 301
Citrix XenApp, XenDesktop, & XenClient Enterprise Desktop Virtualization With cloud, collaboration, networking and virtualization technologies, Citrix powers mobile workstyles and cloud services, making complex enterprise IT simpler and accessible for 260,000 organizations 735
Desktone Desktone Cloud Desktop Virtualization Desktone delivers the industry’s only multi-tenant desktop virtualization platform for service providers to offer Desktops as a Service (DaaS). Dell/Wyse, NaviSite, Quest Systems & NetApp will be demoing DaaS built on the Desktone platform. In addition, Desktone will discuss support for VMware View 5 user experience features including PCoIP. 2330
Liquidware Labs Stratusphere FIT, IXD, UX & ProfileUnity Desktop Virtualization Liquidware Labs™ offers an integrated set of solutions to support the complete desktop transformation lifecycle for migration to next-generation “anytime, anywhere, any device” desktops. ProfileUnity with FlexApp combines virtualized deployment of department and user applications with full-featured user virtualization for ultra-manageable desktops with best TCO. 2536


We wish you safe travels to and from VMworld 2012 and a great show. The one certainty is that the virtualization and cloud landscapes will be different after VMware and all of the vendors in the ecosystem make their announcements next week. VMware’s new Software Defined Data Center strategy is going to usher in a set of changes as profound as those precipitated by virtualization itself – and that entire journey lies in front of us.