VMware NSX

 View Only
Back to discussions
Expand all | Collapse all

NSX-T VDS Uplinks with NUMA, performance less on Numa affinity 1

  • 1.  NSX-T VDS Uplinks with NUMA, performance less on Numa affinity 1

    Posted Jul 24, 2023 08:33 PM

    Have two identical VMs (one cloned from the other).  One VM has numa affinity for numa0 and the other VM for numa1.

    Built a VDS within vCenter and pulled it into a Host Node in NSX-T.

    Send traffic to the numa0 VM and we get a benchline of performance.

    Send the same traffic to the numa1 VM and we get 10-20% less throughput.

    Have checked all of the esxtop statistics and there's no indication of any vCPUs nor RAM being used "cross-numa".

    Suspect that the vmnics are not aligned properly somehow.

     

    [root] vsish -e get /net/pNics/vmnic4/properties | grep 'Device NUMA Node'

       Device NUMA Node:1

    [root] vsish -e get /net/pNics/vmnic8/properties | grep 'Device NUMA Node'

     

     

       Device NUMA Node:0

     

    screenshot from vCenter:

    huchord_1-1690230338602.png

     

    from esxcfg-vswitch_-l.txt:

    huchord_2-1690230402000.png

    This is from NSX-T:

     

    huchord_3-1690230498461.png

     

    Looking at this (this is just a guide, not architecture from our system) :

    huchord_5-1690230687870.png

     

    1) How can I check that the QPI bus (or whatever it's called for the Xeon 8358) was traversed?

     

    2) How can I check that everything is vertically assigned?

     

    TIA