VMware vSphere

 View Only

  • 1.  Local RAID5; latency spikes every 5 minutes. Log: vmw_psp_fixed "This may lead to path thrashing"

    Posted May 20, 2021 05:42 PM

    I'm seeing latency spikes every 5 minutes up to almost a second in length. The timestamps match up with the "This may lead to path thrashing" in the vmkernal.log. This is with vSphere 7.0 U2 on a Dell PowerEdge R640 with a PERC H740P and Dell SAS + SATA SSD. Not sure what is causing this.

    Log snippet that repeats every 5 minutes:

    2021-05-20T16:18:50.020Z cpu9:1048764)ScsiVmas: 1057: Inquiry for VPD page 00 to device mpx.vmhba32:C0:T0:L1 failed with error Not supported
2021-05-20T16:18:50.140Z cpu15:1048763)WARNING: VMW_SATP_LOCAL: satp_local_updatePath:807: Marking path vmhba32:C0:T0:L0 as STANDBY
2021-05-20T16:18:50.140Z cpu15:1048763)vmw_psp_fixed: psp_fixedSelectPathToActivateInt:439: Switching to preferred path vmhba32:C0:T0:L0 in STANDBY state for device mpx.vmhba32:C0:T0:L0.
2021-05-20T16:18:50.140Z cpu15:1048763)WARNING: vmw_psp_fixed: psp_fixedSelectPathToActivateInt:464: Selected current STANDBY path vmhba32:C0:T0:L0 for device mpx.vmhba32:C0:T0:L0 to activate. This may lead to path thrashing.
2021-05-20T16:18:50.140Z cpu15:1048763)vmw_psp_fixed: psp_fixedSelectPathToActivateInt:439: Switching to preferred path vmhba32:C0:T0:L0 in STANDBY state for device mpx.vmhba32:C0:T0:L0.
2021-05-20T16:18:50.140Z cpu15:1048763)WARNING: vmw_psp_fixed: psp_fixedSelectPathToActivateInt:464: Selected current STANDBY path vmhba32:C0:T0:L0 for device mpx.vmhba32:C0:T0:L0 to activate. This may lead to path thrashing.
2021-05-20T16:18:50.140Z cpu15:1048763)WARNING: VMW_SATP_LOCAL: satp_local_tryToActivatePath:1415: Checking TUR on STANDBY path vmhba32:C0:T0:L0
2021-05-20T16:18:50.740Z cpu15:1048763)WARNING: VMW_SATP_LOCAL: satp_local_updatePath:807: Marking path vmhba32:C0:T0:L1 as STANDBY
2021-05-20T16:18:50.740Z cpu15:1048763)vmw_psp_fixed: psp_fixedSelectPathToActivateInt:439: Switching to preferred path vmhba32:C0:T0:L1 in STANDBY state for device mpx.vmhba32:C0:T0:L1.
2021-05-20T16:18:50.740Z cpu15:1048763)WARNING: vmw_psp_fixed: psp_fixedSelectPathToActivateInt:464: Selected current STANDBY path vmhba32:C0:T0:L1 for device mpx.vmhba32:C0:T0:L1 to activate. This may lead to path thrashing.
2021-05-20T16:18:50.740Z cpu15:1048763)vmw_psp_fixed: psp_fixedSelectPathToActivateInt:439: Switching to preferred path vmhba32:C0:T0:L1 in STANDBY state for device mpx.vmhba32:C0:T0:L1.
2021-05-20T16:18:50.740Z cpu15:1048763)WARNING: vmw_psp_fixed: psp_fixedSelectPathToActivateInt:464: Selected current STANDBY path vmhba32:C0:T0:L1 for device mpx.vmhba32:C0:T0:L1 to activate. This may lead to path thrashing.
2021-05-20T16:18:50.740Z cpu15:1048763)WARNING: VMW_SATP_LOCAL: satp_local_tryToActivatePath:1415: Checking TUR on STANDBY path vmhba32:C0:T0:L1


  • 2.  RE: Local RAID5; latency spikes every 5 minutes. Log: vmw_psp_fixed "This may lead to path thrashing"

    Posted May 20, 2021 07:52 PM

    Hello
    It is always advisable to use the manufacturer's custom image in this case Dell.


    https://www.dell.com/support/manuals/en-us/vmware-esxi-7.x/vmware_7.0_icg_pub/introduction?guid=guid-3c867498-b577-4929-b7ee-fb4528e80fb2

    With the following commands we can know the adapter and the driver you are using and also know the version of the driver that is loaded.

    # esxcli storage filesystem list

    # vmkload_mod -s Name Driver |grep Version

    e_espinel_0-1621540257668.png

    https://www.vmware.com/resources/compatibility/detail.php?deviceCategory=io&productid=43392

     

     

     

     



  • 3.  RE: Local RAID5; latency spikes every 5 minutes. Log: vmw_psp_fixed "This may lead to path thrashing"
    Best Answer

    Posted May 20, 2021 11:07 PM

    Hi , thanks for the reply. The server is running Dell's manufacturer image. The mkload_mod -s Name Driver | grep Version command does not work (vmkmod: VMKModCheckMod: Access to file /usr/lib/vmware/vmkmod/Name failed: Not found), but I was able to find what I needed. The issue is the basic Performance > Disk view doesn't tell me what device is actually causing the latency. But seeing the cli output from esxcli storage core device list matched up with what's in the vmkernal.log and was just the Dell iDrac USB device.... For the Googlers, to get rid of the latency spikes in the log, I went into the Dell iDrac > Configuration > Virtual Media > Attached Media > Attach Mode > set to AutoAttach (it was previously set to Attached). The devices in VMware disappeared immediately as so did the latency spikes. I'm guessing the latency spikes were just device detection queries.