Christofer Hoff (@Beaker) and I had a short discussion on twitter the other day about the VMware Cloud Director (vCD) security guidance. We both felt it was a bit lite and missed the point of Secure Multi Tenancy. However, I feel even more strongly that people will implement what is in the vCD Guidance, vBlock Security Guidance, and the vSphere Hardening Guidance, and in effect have a completely insecure cloud. These three guides look at the problem as if they were singular entities and not as a whole.

This realization tied to Chad Sakac’s recent discussion on the 9/22 VMware Communities podcast leads me to believe that ‘good enough’ is no longer ‘good enough’ from a security perspective.  Chad discussed that there need only be the vCloud Director administrator and the vSphere administrator to do the daily heavy lifting. That there would no longer be the need for a security, network, storage, and system specific administrators. In other words, OPEX savings.Any C-Level reading or listening to this will think they can just let go their high paid experts in non-virtualization or cloud technologies. I think this will lead to having an insecure environment. I recently found and discussed several attacks and virtualization security with others in the field. Out of this discussion comes the following:

  • It is now possible to use side-channel attacks against non-virtualization systems to actually gain low-level access to virtualization systems using a previously known but thought to be closed attack vector
  • The virtualization administrators do not have the security expertise to ensure these non-virtualization subsystems are protected.
  • Following the guides will not produce Secure Multi-Tenant public and possibly private cloud deployments.

Mike Foley (@mikefoley), Brad Hedlund (@bradhedlund) and I continued this discussion over the last week end on twitter. The issue at hand is that the vCloud Director is being marketed as producing a Secure Multi-Tenant cloud and that low-level access to hardware is no longer a security concern.

While I do agree that vCloud Director is a step in the right direction, I do not believe it provides Secure Multi-Tenancy except in a single instance. Outside this narrow view of the cloud, vCloud Director is no better or worse than anything else out there. What is the narrow view of the cloud?

vCloud Directory provides Secure Multi-Tenancy for private clouds hosted within the enterprise with only a single classification level.

Let us review what is required to for Secure Multi-Tenancy:

  1. One Tenant cannot see another Tenant’s data (confidentiality).
  2. One Tenant cannot impact the availability of another Tenant’s data (availability).
  3. One Tenant cannot modify another Tenant’s data (integrity).
  4. The Cloud Administrators cannot see, modify, or impact the availability of ANY Tenant’s data.
  5. The Virtualization Administrators cannot see, modify, or impact the availability of ANY Tenant’s data.
  6. The Hardware Administrators cannot see, modify, or impact the availability of ANY Tenant’s data.
  7. The Tenant can dial in the level of security they require.

vCloud Director provides a solution for 1-3 but not for 4-6 and 7 is not yet possible. By their very nature Cloud, Virtualization, and hardware administrators have low-level access to any hypervisor. At this time they can see, modify, take, and destroy data within the Cloud without the Tenants even knowing. It is at this time impossible to prevent them from doing this. While there are tools that will audit for these actions, there are still ways an administrator can bypass any of the existing controls provided outside of the hypervisor.

Granted tools like HyTrust, Catbird, Reflex Systems, RSA Envision, and others provide audit capability, but they are not preventatives at this time. Yes, root password vaulting is important, as is centralized access control, ala HyTrus,t and real-time continuous auditing provided by Catbird. Yet, if these systems can be bypassed we now have a security issue. These tools are improving every day and eventually they will cover all avenues. And yes I believe a combination of these tools are necessary to cloud security. They are only a step in the proper direction for solving 4-6 above. Implementing #7 will take a bit more work.

Because we do not have solid answers for 4-7, any cloud, virtualization, or hardware administrator needs to have classification level to view any of the data residing within the cloud as they can do that now. In government speak these classifications are the clearance levels and outside the government, these are the confidentiality levels. In either case, your administrators still need the classification level to see any data within the system.

This last element is why I believe vCloud Director is great for private clouds hosted within the Enterprise and not with a cloud provider. Current Cloud security boils down to how well you TRUST the administrative team and organizations TRUST those they hire more than they TRUST those they do not. Unless, there are reasons to TRUST those cloud hosting facilities such as bonded administrators and those who have the appropriate clearance levels, etc. This means that Cloud providers must provide guarantees and conditions within their agreements that protect their administrators and at the same time the organizations who use their clouds.

We apply the vCloud Director security Guidance, vSphere Hardening Guides, and others to secure the parts of the environment, but when you put it all together, we do not secure everything. The most recent spate of attacks against non-virtualization subsystems to gain access to virtualization systems definitely implies The Sum of the Parts does not imply security.  We still need our security experts to vet the systems, check them after patches are applied, and to look at all subsystems not just the virtualization ones.

While the vCloud Administrator can handle the heavy lifting and day to day administration of virtual machines, behind the scenes needs to exist a team of security, network, storage, and system administrators that continually audit, provide the service catalog, and solve problems before they become disasters.  These ‘experts’ need to work together to form a pro-active team that when the problems do occur, and they will, they can be solved quickly with the proper incident response no matter the time of day.

The security, network, storage, and system specific experts need to be continuously available to work all security issues within the environment.

Share this Article:

Share Button
Edward Haletky (376 Posts)

Edward L. Haletky, aka Texiwill, is the author of VMware vSphere(TM) and Virtual Infrastructure Security: Securing the Virtual Environment as well as VMware ESX and ESXi in the Enterprise: Planning Deployment of Virtualization Servers, 2nd Edition. Edward owns AstroArch Consulting, Inc., providing virtualization, security, network consulting and development and The Virtualization Practice where he is also an Analyst. Edward is the Moderator and Host of the Virtualization Security Podcast as well as a guru and moderator for the VMware Communities Forums, providing answers to security and configuration questions. Edward is working on new books on Virtualization.

[All Papers/Publications...]

Connect with Edward Haletky:


Related Posts:

4 comments for “Sum of the Parts… Not equal to the Whole

  1. September 29, 2010 at 9:32 AM

    “The Cloud Administrators cannot see, modify, or impact the availability of ANY Tenant’s data.
    The Virtualization Administrators cannot see, modify, or impact the availability of ANY Tenant’s data.
    The Hardware Administrators cannot see, modify, or impact the availability of ANY Tenant’s data.
    The Tenant can dial in the level of security they require.”

    Are these really completely feasible? I mean the point of security is to protect from the hands of those not responsible while compliance and governance is to audit/protect those that are responsible, right?

    I agree with most of the other points especially this nice concise list:

    “One Tenant cannot see another Tenant’s data (confidentiality).
    One Tenant cannot impact the availability of another Tenant’s data (availability).
    One Tenant cannot modify another Tenant’s data (integrity).”

    .nick

  2. September 29, 2010 at 12:37 PM

    Hello Nick,

    I think they are feasible with a few other changes to the environment, such as using AES&I accelerators within Westmere chips tied to TPM/TXT (virtual versions) to do VMDK/RDM encryption as well as memory encryption. It is all about dialing a level of security for those who want that level of security. Given encrypted virtual disks and memory, it would be close to impossible for an administrator to see, modify, steal the data. Given these, then the administrator is not a concern, and if the management networks were hacked, the chance of stealing data that is useful goes way down.

    Just let me dial that level of security and I would be much happier. BTW, all this has to happen within the hypervisor not OUTSIDE the hypervisor so an encrypting switch or fibre channel card is not the answer.

    The problem is, that since the administrator can SEE the data, they need to have the classification level of that data to even be an administrator and for some companies, those levels only happen at pay grades MUCH higher than the administrator. Encryption is one solution to this problem. And if they are ‘tenant’ keys, it implies the first 3 as well.

    Best regards,
    Edward L. Haletky

  3. NAKettles
    October 5, 2010 at 3:13 AM

    Although the focus of your article is more on tenant vs. tenant security, the point you make at #5 for a requirement for secure multi-tenancy suggests the need for some kind of privilege management access controls.

    Without them for example, the ESX COS console provides a completely new attack surface to a user with appropriate credentials.

  4. October 5, 2010 at 8:47 AM

    Hello,

    Actually it is more than access to the ESX COS, but access to the underlying functionality of the environment. The Administrative belly of the cloud. Unfortunately, there are many ways this can be attacked. The ESX COS is just one of many.

    Best regards,
    Edward Haletky

Leave a Reply

Your email address will not be published. Required fields are marked *


two + 2 =