Updated: There is a Need for VMsafe Certification from VMware

There are two methods in which VMware VMsafe that can be used: those are fastpath and slowpath. Fastpath entails using just a driver to interact with the VMsafe API and hence the vmkernel. Slowpath is the use of a fastpath driver AND a virtual appliance to do the heavy lifting.

The use of VMware VMsafe enabled third party products introduces third party fastpath drivers into your hypervisor. What these drivers ultimately do is interact with the VMsafe fastpath API, but is that ALL they do? That is why we need some level of certification for VMsafe fast path drivers. We need to KNOW that they do not do anything wrong, bad, or unfortunate.

Many vendors have hybrid systems that do some things fast path and other things slow path. Due to the compute requirements for Antivirus for example, those vendors will most likely use slowpath. Reflex System’s vTrust VMsafe application on the other hand does most of its work in fastpath with slowpath being used to configure the vTrust fastpath driver.

VMware VMsafe fast path drivers have unprecedented access to the hypervisor memory space so that they can inspect or pass on to slowpath VMsafe Appliances, structures that live only within the vmkernel. Due to this several questions arise:

  1. Can more than one VMsafe fastpath driver be used at the same time?
  2. If so, is there some sort of locking to prevent race conditions?
  3. Is there a common driver that allows one to write VMsafe slowpath Appliance with minimal impact to the hypervisor?
  4. Is there a way to certify that the driver(s) in use will not corrupt data or crash my VMware ESX/ESXi 4 hosts?

Some of these questions have unknown answers. At the moment there is no locking mechanism so race conditions could exist between multiple drivers. Consider the case where you have two VMsafe appliance where one reads some data (network data) and the other fixes the same data (inline patching). The network data could be read before the patch was applied, during the middle of the patching process, or after the patch application. If it was anything but the last option, the results could be less than desired. In other words it is possible for corrupted data to be sent to a VM. The result could be catastrophic depending on how the VM interprets the data passed to it.

UPDATE: VMsafe-Net interfaces uses driver chaining mechanisms for Network based data, in other words it passes all data through a chain of fastpath drivers so that race conditions do not occur.  Any VM or vNIC can be configured to use any number of VMsafe-Net modules as well.

There is also no common fastpath driver available for third parties to make their own VMsafe slowpath appliances. Reflex Systems would very much like the core to their vTrust fastpath driver to be the common driver. They have gone so far as to suggest the same to VMware and are now waiting on VMware in order to decide their next steps. They may even place the core of the vTrust driver into an Open Source project which they manage. All I can say is that I personally like this idea as using one driver to handle multiple VMsafe slowpath appliances. Therefore you can build locking into the driver and perhaps have specialized modules for other fastpath applications of VMsafe.

Unfortunately my queries into VMware have not lead me to believe there is a VMsafe Certification program available at this time. It has been rumored to exist but I have no confirmation. One is needed and I fully expect VMware to not only ensure the VMsafe fastpath drivers do nothing harmful to the virtual environment, but also address interaction issues between multiple VMsafe fastpath drivers. In addition, I would like such reports made available to satisfy auditing requirements.

Posted in SecurityTagged , , , , ,