VMware vSphere

 View Only

  • 1.  ESXi Unable to Detect iSCSI Datastore - "No filesystem on the device" Error

    Posted Sep 11, 2024 05:34 AM

    Hello,

    I am facing an issue with my ESXi server, where an iSCSI datastore that was previously working is no longer visible, and the file system is not recognized. Below are the details of the issue:

    System Information:

    • ESXi Version: VMware ESXi 6.7.0 build-8169922
    • Storage Device: QNAP TS-873AU-RP
    • iSCSI LUN Size: 35TB
    • VMFS Version: VMFS-6

    Problem:

    My ESXi server can see the iSCSI LUN, but it cannot recognize or mount the file system. Here are the results of some commands I've run using esxcli:

    1. LUN is Recognized:I can successfully see the LUN with the above command. The LUN ID is: naa.6e843b6796b5dcfd555cd497cd85bed4 esxcli storage core device list
    2. No Filesystem Detected:These commands do not show any file system or VMFS volumes. When I try to mount the file system, I get the following error: esxcli storage filesystem listesxcli storage vmfs extent list esxcli storage filesystem mount -u naa.6e843b6796b5dcfd555cd497cd85bed4 No volume with uuid 'naa.6e843b6796b5dcfd555cd497cd85bed4' was found
    3. NMP Error: In addition, I see the following error messages in the ESXi logs: NMP: nmp_DeviceStartLoop:728: NMP Device "naa.6e843b6796b5dcfd555cd497cd85bed4" is blocked. Not starting I/O from device. NMP: nmpDeviceAttemptFailover:715: Retry world failover device "naa.6e843b6796b5dcfd555cd497cd85bed4" - failed to issue command due to Not found (APD), try again...

    Steps I've Tried:

    • Rescanned the LUN: I used the command esxcli storage core adapter rescan --all but the result did not change.
    • VMFS Snapshot Check: Ran esxcli storage vmfs snapshot list, but it returned no results.
    • Checked with vmkfstools: Ran vmkfstools -P /vmfs/devices/disks/naa.6e843b6796b5dcfd555cd497cd85bed4, but no VMFS file system was found.
    • Storage Connection is Healthy: The iSCSI target is being presented correctly on the storage device, and there are no issues with other servers accessing it.

    Assistance Needed:

    • How can I reactivate the iSCSI datastore?
    • Could the VMFS file system be corrupted? Any suggestions on how to repair or recover it?
    • Has anyone experienced similar issues with these error messages, and how were they resolved?

      It was working before, there was no connection error here, so I did not look for a problem on the network side, I disabled and re-enabled the iscsi connection on the qnap side, the logs I got on the vmware side;

      cpu25:2104590)iscsi_vmk: iscsivmk_ConnNetRegister:2170: socket 0x430e7df3bda0 network resource pool netsched.pools.persist.iscsi associated
      cpu25:2104590)iscsi_vmk: iscsivmk_ConnNetRegister:2198: socket 0x430e7df3bda0 network tracker id 963589706 tracker.iSCSI.10.1.60.50 associated
      cpu41:2104590)WARNING: iscsi_vmk: iscsivmk_StartConnection:877: vmhba64:CH:0 T:1 CN:0: iSCSI connection is being marked "ONLINE"
      cpu41:2104590)WARNING: iscsi_vmk: iscsivmk_StartConnection:878: Sess [ISID: 00023d000001 TARGET: iqn.2004-04.xxx TPGT: 1 TSIH: 0]cpu41:2104590)WARNING: iscsi_vmk: iscsivmk_StartConnection:879: Conn [CID: 0 L: 10.1.60.131:31356 R: 10.1.60.50:3260]cpu0:2351678 opID=26ab8ccd)World: 11942: VC opID 2169ae1b maps to vmkernel opID 26ab8ccdcpu0:2351678 opID=26ab8ccd)NVDManagement: 1478: No nvdimms found on the systemcpu0:2351770 opID=98e29b39)World: 11942: VC opID 2169ae2b maps to vmkernel opID 98e29b39
      cpu0:2351770 opID=98e29b39)NVDManagement: 1478: No nvdimms found on the system

      when I refresh the devices this I got this logs;
      cpu8:3336047)WARNING: NFS: 1227: Invalid volume UUID mpx.vmhba0:C0:T4:L0
      cpu2:3336047)FSS: 6092: No FS driver claimed device 'mpx.vmhba0:C0:T4:L0': No filesystem on the device
      cpu10:2351673 opID=946ee16f)World: 11942: VC opID 2169b446 maps to vmkernel opID 946ee16f
      cpu10:2351673 opID=946ee16f)VC: 4616: Device rescan time 73 msec (total number of devices 7)
      cpu10:2351673 opID=946ee16f)VC: 4619: Filesystem probe time 176 msec (devices probed 4 of 7)
      cpu10:2351673 opID=946ee16f)VC: 4621: Refresh open volume time 2 msec



  • 2.  RE: ESXi Unable to Detect iSCSI Datastore - "No filesystem on the device" Error

    Posted Oct 03, 2024 05:29 PM

    Hello

    Hope this helps: https://knowledge.broadcom.com/external/article/318524/datastore-will-not-mount-or-lun-resignat.html

    This issue is  fixed in ESXi6.7U1. Upgrade the ESXi hosts to 6.7 U1 or later.

    let me know

    cheers



    ------------------------------
    ------------
    Manlio
    PMP®-PMI, Azure & AWS certified, VCP34562019/21 DCV VCAP5DCD, vExpertx10, RHCVA
    ------------------------------

    Original Message


  • 3.  RE: ESXi Unable to Detect iSCSI Datastore - "No filesystem on the device" Error

    Posted Oct 04, 2024 06:01 AM

    Hello.
    VMware vSphere 6.7 is in the initial build (GA), urgently you should upgrade at least to the latest level Update 1 or if possible to the latest update 3.

    You should consider that version 6.7 is out of support since November 2023.

    https://knowledge.broadcom.com/external/article?legacyId=2143832

    You can get the updates from the broadcom support portal, you need to have a user and password. If you don't have a user yet you can register and get it.

    https://support.broadcom.com/web/ecx/home



    ------------------------------
    Enrique Espinel F.
    Freelance Senior Technical Specialist IBM, Lenovo, Veeam Backup and VMware vSphere VTSP-SV / VTSP-HCI
    ISRAEL – Karmiel (UTC +3)
    I speak Russian, Spanish Hebrew (basic) and a little English.
    ------------------------------

    Original Message