Cisco VM-FEX Limitations

The week before VMworld on 8/25 was the Virtualization Security Podcast featuring Greg Ferro (@etherealmind), CCIE to discuss Cisco VM-FEX and its impact on virtualization and cloud security. VM-FEX is a method by which the fabric of a UCS top of rack switch is extended to the VM, but only if the VM is using VMDirectPath.  So does this impact Virtualization and Cloud Security in any way?

The long and short of it is no, VM-FEX does not really change the virtualization and cloud security footprint from what is already known. Why is this? Because all Cisco VM-FEX does is provide a multi-root IO virtualization device that presents itself as multiple single-root IO virtualization cards so that vSphere can take advantage of VMDirectPath from within the VMs directly through to the top of rack switch within a Cisco UCS cabinet.

But, I could already do that within a Cisco UCS deployment? Yes, you could but the VM-FEX offloads much of the high order processing and binds a VMDirectPath port on the VM direct to a switch port on the top of rack switch. That part is new, with just the Palo adapter, their is no binding of ports to the top of rack switch. Which means that with the help of VMDirectPath and VM-FEX, Cisco’s UCS hardware and switches will know which VMs are connected to it at all times and would bypass the virtual switch entirely.

While this is not stated within the podcast as possible, bypassing the virtual switch entirely, vSphere provides the capability and with Cisco VM-FEX, you now have a way to connect a VM directly to the hardware with better overall performance. However, this support is limited to 4 VMDirectPath devices per VM, with a support of up to 8 VMDirectPath devices per host. In essence, you now have a limit of a maximum of 8 VMs that can make use of VMDirectPath and therefore VM-FEX. At least that is how I interpret the vSphere 5 Configuration Maximums. This is up from the 4 devices available in vSphere 4.x.

Are there more things to consider when using VM-FEX, yes there are, you now have to worry about the addition risks associated with the VM-FEX hardware if there are any, but also the number of VMs that can take advantage of the new functionality. The VM-FEX hardware adds yet another layer to the networking stack within the UCS device, a much needed layer. However, with consolidation rations exceeded 20 per blade, it would have been nice if more VMs could take advantage of the VM-FEX capabilities for performance reasons.

All in all a very intriguing concept with limited usage except for those 8 single vNIC VMs that need wire speed performance without the virtual network layer getting involved. Having non-VMDirectPath VMs and VMDirectPath VMs on the same blade may also be a bit confusing to the administrators, so ensure you keep good network diagrams that include the virtual network.

* The travelogue video was produced by Lars Troen

Posted in SecurityTagged , , , ,