Keep Loosing iSCSI drives on reboot

[ShawnMcC]

I am having a particular issue, have two hosts, lets call them HOSTA (8.0.2), and and HOSTB (8.0.2) connected to a TrueNAS storage

Everything was working fine.  A couple of weeks ago my TrueNAS borked, OS was gone, but all data was good, so rebuilt OS, and rebuilt all of the Datapools, and NFS and iSCSI shares, and lets call this DATA01

Once I rebuilt the DATA01, HOSTB had no issues accessing the 2 NFS and 2 iSCSI shares on the rebuilt server

For some reason HOSTA did not see the 2 iSCSI shares, so I used this document (https://www.rwardrup.com/mounting-missing-esxi-datastores/), use the command esxcfg-volume -l and esxcfg-volume -M to re-attach the drives, and it worked fine, seeing the 2 iSCSI shares, go along my happy way.

Now yesterday, update HOSTA from 8.0.2 to 8.0.3 and when it comes up both iSCSI datastores are gone.  Look at Storage, Devices in the WebGUI, and they are not there.

Do a refresh on adapters, and then a refresh on devices, and the iSCSI drives now show up

re-do the esxcfg-volume -l and esxcfg-volume -M and now I have the datastores, but once I reboot, they are gone again.  I am not sure on where to go from here. 

I have not rebooted HOSTB, so cannot confirm if this is a server specific issue.

[ShawnMcC]

OK, I solved my own problem and just wanted to document it here in case someone else has the same issue.  This Reddit article got me close enough that I could fix it

Missing iSCSI datastores after vSphere 7.0 U3 upgrade

So, essentially with 8.0.3, and I am guessing 8.0.2 the command (esxcfg-volume -M) does not create a persistent mapping, only a in memory mapping. Not sure exactly when it stopped behaving as documented, but just know it is gone

With the reddit article, I executed the following command

configstorecli config current get -c esx -g storage_iscsi -k software_adapter -outfile IscsiConfig.json

and my file looks a little thin, basically, I should be seeing something like this:

[root@ORD-12e7-esxi01:~] vi IscsiConfig5.json
{
   "alias": "iscsi_vmk",
   "delayed_ack": true,
   "discovery_send_targets": [
      {
         "address": "172.15.11.105",
         "port": 3260
      },
      {
         "address": "172.15.11.101",
         "port": 3260
      }
   ],
   "enabled": true,
   "first_burst_length": 262144,
   "iqn": "iqn.1996-06.com.vmware:ord-12e7-esxi01-6qw82d54",
   "login_timeout": 5,
   "max_burst_length": 262144,
   "max_outstanding_r2t": 1,
   "max_recv_data_segment_length": 131072,
   "noop_out_interval": 15,
   "noop_out_timeout": 10,
   "recovery_timeout": 10,
   "targets": [
      {
         "address": "172.15.11.101",
         "iqn": "iqn.1994-12.com.netapp:sn.atux1sql08y4f2fcvxqrx8bswq5f71z3:vs.8",
         "port": 3260,
         "type": "STATIC"
      },
      {
         "address": "172.15.11.105",
         "iqn": "iqn.1994-12.com.netapp:sn.atux1sql08y4f2fcvxqrx8bswq5f71z3:vs.8",
         "port": 3260,
         "type": "STATIC"
      }
   ],
   "vmknics": [
      "vmk3",
      "vmk4",
      "vmk5"
   ]
}

but I am getting something like this:

[root@ORD-12e7-esxi01:~] vi IscsiConfig5.json
{
   "alias": "iscsi_vmk",
   "delayed_ack": true,
   "discovery_send_targets": [
      {
         "address": "172.15.11.105",
         "port": 3260
      },
      {
         "address": "172.15.11.101",
         "port": 3260
      }
   ],
   "enabled": true,
   "first_burst_length": 262144,
   "iqn": "iqn.1996-06.com.vmware:ord-12e7-esxi01-6qw82d54",
   "login_timeout": 5,
   "max_burst_length": 262144,
   "max_outstanding_r2t": 1,
   "max_recv_data_segment_length": 131072,
   "noop_out_interval": 15,
   "noop_out_timeout": 10,
   "recovery_timeout": 10,
   "vmknics": [
      "vmk3",
      "vmk4",
      "vmk5"
   ]
}

so I just added the targets to the exported file, Iscsiconfig.json, and import it with the command:

[root@localhost:~] configstorecli config current set -c esx -g storage_iscsi -k software_adapter -infile IscsiConfig.json  

and the missing Datastores are there.  Do a reboot, and also working fine.  So for me this issue is closed and resolved