The general answer is yes; they work well enough for most experts to recommend them. However, the more specific answer depends solely on how you have set up your physical and virtual networks and the purpose of the virtual firewall.
Is your purpose to protect all VMs attached to a virtual switch from other VMs on the same virtual switch? You can achieve this with a virtual firewall only if you use portgroups and firewall between different portgroups.
Is your purpose to protect all VMs attached to a virtual switch from other VMs on different virtual switches? You can achieve that by having a virtual firewall between the protected virtual switch and up to three other virtual switches. Why three? There is a limitation on the number of virtual NICs available to a VM.
Is your purpose to firewall a DMZ attached to the outside world from the inside world? This is also achievable with a virtual firewall, however it requires multiple physical NICs attached to different pSwitches or VLANs within your physical network. It also applies the principle of vSwitch to vSwitch protection.
The other big question is which virtual firewall to use? There are several contenders: Smoothwall, m0n0wall, and a host of others. There is also the possibility of using the software from a hardware firewall within a VM, but that depends on the vendor and whether or not the OS they use within the hardware firewall can be virtualized, there is support to do this, and some form of instructions to do this.
The Smoothwall folks for example sell a hardware appliance as well as provide an installable image for a Virtual Machine.
The main concern about using a virtual firewall is to ensure isolation of those items to be protected with proper virtual and physical network layout.
The other concern is that unless you make some low level modifications VMs attached to a vSwitch that is not, itself, attached to a physical NIC cannot participate in VMotion or the ability to move VMs from virtualization server to virtualization server without powering them down.
This last item may dissuade people from using virtual firewalls but it will not stop me. I use them and recommend them as a solution to an often tricky problem that requires them. However, due diligence with your network layout is absolutely required.
Virtualization expert Edward L. Haletky is the author of "VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers," Pearson Education (2008.) He recently left Hewlett-Packard , where he worked in the Virtualization, Linux, and High-Performance Technical Computing teams. Haletky owns AstroArch Consulting, providing virtualization, security, and network consulting and development. Haletky is also a champion and moderator for the VMware discussion forums, providing answers to security and configuration questions.
This story, "Are virtual firewalls a solution for VM security?" was originally published by CIO.