passthrough NICs got poor Throughput performance

[manchesterunite]

Hi

   I am new to ESXi,  so please correct me if I say things not clearly.

BackGround

1. a Dell PowerEdge R730 with a 4-port X710 10G NIC
2. install a Centos with DPDK-16.04；
3. port 0 in 、port 1 out，RFC-2544 Throughput test get almost line rate(64-bytes frame size)

  while on EXSI

1. ESXi 6.0 on the same Dell PowerEdge R730 with a 4-port X710 10G NIC.
2. all 4 ports are set to passthrough
3. CentOS VM with DPDK-16.04.
4. port 0 in 、port 1 out，RFC-2544 Throughput test get only 7Gbps(64-bytes frame size).

   more test on EXSI

1. with 128-bytes frame size packets，2 ports in，2 ports out，  Throughput  is about 14Gbps, twice to 64-bytes
2. run 2 same VMS on EXSI，each VM has its own seperate CPUS, each VM get 2 passthrough ports,VM0(port0,1) VM1(port2,3)

        when test only on VM0, which means 64-bytes packet port0 in port1 out，VM0's Throughput  is about 6Gbps

        when packets stream start goes to port2 too，VM0's Throughput is down to 3.5Gbps，VM1 got the same Throughput. while 3.5*2=7, exactly the same Throughput  with one VM

    3. I also run the same test on ESXi 5.5  Dell PowerEdge R630 with a 2-port 82599 10G NIC, the performance is likewise

   My Guess

1. it looks that the EXSI system could only Rx and Tx some fixed packets per second！
2. the bottleneck is on EXSI，not on VM, because VM's cpu usage is low

Questions

1. Is there any configuration in EXSI concerning with this?
2. what else can i do？

[manchesterunite]

After upgrade EXSI from 6.0 to 6.5，the performance is good now.

Maybe some issue has been fixed in 6.5

[Dave_the_Wave]

Can you remember the ESXi 6.0 build you used exactly?

It'll be interesting to see if a 6.0 Sep2018 build didn't give you what 6.5 did.

I'd hate to abandon a very well supported 6.0 knowing that 6.5 is "moar".

[manchesterunite]

ESXi 6.0 build number 10474991

After upgrade to ESXi 6.5 build number 5969303,   with X710 the performance is roughly better, but still has little frame loss, even at evey low frame rate...  while with 82599, no frame loss at line rate

is something wrong with ESXI or X710 ?

[Dave_the_Wave]

You are using the Dell versions of ESXi, right?

If not, I've discussed their importance here:

Re: custom ESXi ISO?

[manchesterunite]

Yes，I'm using Dell customized ESXi.