Guest Operating Systems

 View Only
Expand all | Collapse all

Windows VMs experiencing intermediate packet drops

  • 1.  Windows VMs experiencing intermediate packet drops

    Posted Jan 10, 2024 09:49 AM
      |   view attached

    We have a infra with few ESXis, few windows Vms and linux VMs. Infra is connected with 1g Network.

    We have a issue where windows VM is getting intermediate packet lose which result in hang like sutiation and RDP disconnect.

    Packet drops is even between the VMs on same VLAN, say there are two VMs on different ESXi in same VLAN it will observer issue of packet drops.

    VM1 & VM2 : windows server

    VM3 & VM4 : Linux

    VM1 on esxi01

    VM2 on esxi02

    VM3 on esxi03

    VM4 on esxi04

     

    All ESXi have same configuration

    All VMs (both linux and windows) are in same vlan.

     

    scenario

    1. Ping between windows VM : intermediate packet drops on both VMs

    2. Ping between Linux VM : No Packet drop

    3. Ping from windows VM to linux VM : no packet drop

    4. Ping from Linux to windows VM : intermediated packet drops observed

     

    Note: when VMs are on same Esxi than there is no ping drop



  • 2.  RE: Windows VMs experiencing intermediate packet drops

    Posted Jan 10, 2024 09:58 AM

    sorry... what do you mean exactly with "intermediate packet lose"?

    Did you check also with tracert/traceroute?



  • 3.  RE: Windows VMs experiencing intermediate packet drops

    Posted Jan 10, 2024 10:12 AM
      |   view attached

    I mean there are packet drops between continuous ping. pic attached

    Tracert : there is only single hop l2 switch in between, trace complete on 1st hopw.

    all ESXi are connected to this switch directly.

     

     

     

     

     



  • 4.  RE: Windows VMs experiencing intermediate packet drops

    Posted Jan 10, 2024 10:24 AM

    ok, a random loss of packets.

    if you are 100% sure that when VMs are on the same hosts all works fine, we can say "O.S. are ok".
    so, it appears to be something in the path from one host and the others.

    vSwitch portgroup and switch, cables of so on.

    Ping among the management interfaces of esxi hosts work fine? Did you try moving the VM in the same vSwitch of management network? 

     



  • 5.  RE: Windows VMs experiencing intermediate packet drops

    Posted Jan 10, 2024 10:52 AM

    Ping for ESXi mgmt is fine. ESXi mgmt is totally on different Physical NICs which are connected to a different physical switch.

    I would have 100% agree to you if linux\photon VMs were also behaving in same way but for linux\photon VMs there is no drops observed all work fine. only issue is with windows vm ( firewall is off)

    All VMs are on same distributed switch and same port group ( which means same configuration for all VMs).

     

     



  • 6.  RE: Windows VMs experiencing intermediate packet drops

    Posted Jan 10, 2024 12:05 PM

    sure it's a tricky thing, but if the issue is on OS level, it should be also present when VMs are in the same host...



  • 7.  RE: Windows VMs experiencing intermediate packet drops

    Posted Jan 10, 2024 01:02 PM

    that is also true...

    have done lots of testing around but result is -ve



  • 8.  RE: Windows VMs experiencing intermediate packet drops

    Posted Jan 10, 2024 01:52 PM

    did you try adding Vm portgroups on the vSwitch where the management vmkernel is and move some vm there? they would be using different physical nics and switches



  • 9.  RE: Windows VMs experiencing intermediate packet drops

    Posted Jan 11, 2024 07:12 AM

    Esxi mgmt is connected to 10g network (physical switch support 10g) while VM n/w is connected to 1g (physical switch support 1G only).

    I fell like it is something related to 1G n/w only. but linux VMs works fine on this 1G network.



  • 10.  RE: Windows VMs experiencing intermediate packet drops

    Posted Jan 11, 2024 08:28 AM

    I would use the 10g for VM traffic and 1 g for management traffic instead.

    Anyway, now I would suggest you again to move both traffics on the 10g network and see what happens.



  • 11.  RE: Windows VMs experiencing intermediate packet drops

    Posted Jan 12, 2024 01:51 AM

    Based on your reports, if the issue only presents on the Windows VMs and not on the Linux VMs, sound logic would suggest the issue is with the Windows guest OS network settings. A quick Google search returns a large number of results for the search "virtualized microsoft windows vm intermittent packet loss". 

    I would begin by reinstalling VMware Tools on the Windows VMs or installing if it's not there already. If that doesn't solve the problem, start looking into any recent Windows updates that may have been installed on the VMs having the intermitten packet loss issue.



  • 12.  RE: Windows VMs experiencing intermediate packet drops

    Posted Jan 12, 2024 05:38 AM

    Is this a new setup ?

    Was it working before and you recently started seeing this issue ? If the issue started recently check what was the recent changes in the setup after which the issue started.

    Are you using vmxnet3 card or E1000 NIC cards for the VMs (windows and linux). 

    since you have 4 ESXi nodes, were you able to isolate this issue only when the VMs run on particular ESXi nodes or the behavior is same on all 4 ESXi nodes.

    What is the load on 1Gb card ? 

    Is the occurs during particular time period or completely random (may be at the time of high load on physical NIC, thiat could cause the packet drops due to over load)