Question on HA after power outage

[Kent Wicker]

Hello,

At one site I have a 3 node ESXi cluster connected to a Dell PowerStore storage array.  Yesterday we had a power outage.  When the power was restored, the ESXi hosts come up with no issues except that they were up before the storage array so they had no datastores and therefore did not auto restart the VMs.  Once storage was online, I waited and no VMs started becasue the datastores were not visible.  At this point they were not listed as inaccessible, they just were not in the list anymore.  I did a rescan, got datastores, and HA began starting the VMs.  I had probably waited about 15 minutes after storage was available before doing the manual rescan.

I am unable to sequence the power up at this time to ensure the storage is online first.  My question is, is this normal behavior?  Would ESXi eventually have done an rescan on its own?  Is there any advanced setting that would do this or have ESXi keep looking for datastores?  I looked through the several hundred advanced settings but did not see one that seemed to fit.

I'm running ESXi 7.0.

Thanks!

[wsehringer]

Hi Kent,  You can modify an advanced setting to have ESXi periodically check for new storage.

Go to vSphere Client.

Navigate to Host > Configure > Advanced System Settings.

Search for the setting:

Disk.DeviceReclaimTime

Change its value to 300 (or any lower value, in seconds). This setting controls how frequently ESXi checks for new storage devices.

Best Regards

WS!

[Kent Wicker]

@wsehringer,
Thanks for pointing that setting out!  I checked and it is set at 300 so perhaps I was impatient and didn't wait 300 seconds for it to auto-rescan.

[janatlud]

Hello,

maybe I would perform another solution. I don´t know, if your ESXi hosts HW is also DELL, some DELL poweredge or something, anyway, I think, similar feature will have also another vendors.

If you need to start storage device first, let is in your default setting, it is booting imediatelly, when power is restored. For ESXi hosts, you can set postponed startup in BIOS setting.

"Bios Setting -> Security -> AC Power Recovery -> User defined"   and set in seconds, how long will the server wait until electricity before it starts booting. Minimum are 2 minutes (for G16), I don´t remember max possible limit, but you can set there for example 600/900 seconds and after that should be your storage device already UP, before the ESXi hosts will start booting.

This change needs reboot of the hosts.

BR

J

[Kent Wicker]

@janatlud,

Good I idea, and I thought of that, but these are MX blades which don't support the delayed power on.  The chassis also doesn't have anything to delay the power on.

[Duncan Epping]

Just to be clear, HA will auto retry after the storage returns, and HA itself is not responsible for rescanning storage.

Also note, changing advanced settings like these can have a serious impact on the environment when it is relatively large.

[Kent Wicker]

@Duncan Epping,
Thanks for the clarification.  While it felt like I waited a while for the host to find storage, perhaps I did not wait the default of 300 seconds.  Now that I know it should have scanned every 5 minutes I plan to leave it at the default setting.