Holodeck Toolkit via VCF Lab Constructor

 View Only
Back to discussions
Expand all | Collapse all

vMotion Connectivity Issues via Dedicated TCP/IP Stack in Nested VCF Holodeck Environment

  • 1.  vMotion Connectivity Issues via Dedicated TCP/IP Stack in Nested VCF Holodeck Environment

    Broadcom Employee
    Posted Apr 27, 2026 11:48 PM

    I am encountering a persistent networking issue within a nested VCF environment deployed via Holodeck.

    Although the deployment utilizes the dedicated vMotion TCP/IP stack, I have consistently observed connectivity failures between the ESXi hosts' vMotion networks post-deployment. My preliminary troubleshooting results are as follows:

    • Configuration Audit: Both the vMotion VMkernel interface (vmk) settings and the vMotion TCP/IP stack configurations have been verified as correct, Including the default route of the vmotion stack.

    • The Issue: vmkping tests between specific hosts (or sometimes a subset of hosts) consistently fail, despite the configuration appearing valid on the surface，and for some hosts, the vmkping gateway is also not working. (vmkping -I vmk1 10.1.3.1 -S vmotion)

    • Symptom: This intermittent or localized connectivity failure is preventing successful migrations within the nested cluster.

    Has anyone experienced similar routing or encapsulation issues within the Holodeck nested networking layer? I would appreciate any insights on whether this might be related to the underlying virtual switch configuration or a specific timing issue during the automated provisioning.



    -------------------------------------------


  • 2.  RE: vMotion Connectivity Issues via Dedicated TCP/IP Stack in Nested VCF Holodeck Environment

    Posted Jul 14, 2026 11:42 PM
    Hi Shenghai,
    We've seen similar behavior in nested VCF environments where the issue wasn't the vMotion TCP/IP stack configuration itself, but the underlying nested networking.
     
    A few things I'd verify:
    • Ensure the vMotion VMkernel interfaces (vmk) are attached to the correct distributed port group and VLAN on every ESXi host.
    • Confirm that Promiscuous Mode, MAC Address Changes, and Forged Transmits are enabled on the parent vSwitch/port group hosting the nested ESXi VMs. Missing one of these can cause intermittent host-to-host connectivity issues.
    • Verify that the MTU is consistent end-to-end (physical NIC > parent vDS/vSS > nested vDS > VMkernel). Even if standard pings succeed, an MTU mismatch can cause vmkping -S vmotion failures.
     
    Test with:
     
    vmkping -I vmk10 <peer-vmk-ip>
    vmkping -d -s 8972 -I vmk10 <peer-vmk-ip>
     
    If jumbo-frame tests fail while standard pings succeed, it's usually an MTU or encapsulation issue.
     
    • Check whether the affected hosts are all connected to the same parent ESXi host. If the failures occur only across specific parent hosts, it could indicate an uplink or parent vDS configuration issue rather than a VCF deployment problem.
    • Finally, review the routing table (esxcli network ip route ipv4 list) to ensure the vMotion TCP/IP stack is using the expected gateway or remains isolated if L2-only connectivity is intended.
     
    Out of curiosity, are the connectivity failures always between the same host pairs, or do they change after redeploying HoloDeck or rebooting the nested ESXi hosts? That detail could help determine whether this is a provisioning issue or something related to the underlying virtual networking.