Hi all VM expert
I have a question that haunted me for long time in my home lab
I have setup two ESXi 5 box , version 5.0.0.914586, 1 installed with nexenta on a VM with RDM, the other just run typical windows VMs
Disks Performance is great when running VM on this nexenta box as shown in the following benchmark

However, on the second ESXi box tried to run the same test via iscsi, the result is terrible as shown below

If you look at 16K blocks, it is still acceptable as it writes around 40MB/s,
However, it dramatically drops to 1215kb when trying to write 32K block?
Does anyone know why? Or tell me how to troubleshoot this issue
I suspect it is related to networking and attempted to several tricks to enhance performance
1) Setup LACP on my NetGear GS-108T 8 port switch,
2 Reconfigured my psp, using round robin settings
esxcli storage nmp psp roundrobin deviceconfig set --type="bytes" -B 10485760 -d naa.600144f09bfb06000000512447bc0002
Both didn't help
My network settings as follow:

Multipathing settings

Network statistics when running 32k block size test

It would be great if anyone could shed some light on this issue
Currently it takes like 45+ mintues to install windows 2008 R2, which is way too slow
Thanks
Kris