
Your test setup should be just a USB stick with 5.1 or 5.5 plugged in your server, all the remaining storage disconnected, a NFS (a FreeNas NFS share with anonymous access is just fine) to store your pfSense VM. I strongly suggest you go the NFS way, disconnect your DAS vmfs3 (not required if you are careful), install esxi 5.5 on a USB stick so that you don't have to upgrade anything. You can also use NFS storage that's quick to setup and won't require you to play with your current local datastores (hey you never know what might happen). So if you're using direct attached storage check if you have some room for adding another hdd that you will be able to use as vmfs5 local storage for test purposes (so that you don't change anything on your current vmfs3 local datastores).

First problem, vmfs, esxi 4 uses 3.x vmfs version where esxi 5.x uses vmfs5 but can access vmfs3. No need to destroy your current setup yet you will have to make some hardware changes. Yes definitely, that's the first thing you should try, esxi 4 has been released in 2009, if you're using the free license version you should definitely try to upgrade to 5.1 or 5.5.

So i'd say virtualization is not the problem. I'm running pfSense 2.2.3 on esxi 5.1 (no exactly the latest versions) on a 1gigabit up/down link and i'm getting ~118 MBytes/s (950 mbps/s) on ASUS q87T and Intel i-5 4570S.
