vSAN - Fault Domains Error

[MartinKerr]

Hi,

I have created a new 3 host cluster within vCenter 6 and enabled vSAN, added the correct storage (1 x Flash, 3 x HDD). They are all mounted and healthy. vSAN Switch has been set up and is connected via 10GB switches, MTU sizes on the virtual adapters are set at 9000 and set at 9240 on the physical switches. Also IGMP has also been enabled on the switches.

So the problem i am having is that i cannot migrate any VM storage to the vSAN Storage, the error i receive when trying to migrate is:

"Cannot complete file creation operation. This storage policy requires at least 3 fault domains with hosts contributing storage but only 1 were found. Failed to create object."

A bit strange as i have 3 fault domains with one host in each.

I currently don't have any leads on any resolution. Any help would be very grateful.

Many Thanks

[zdickinson]

Good afternoon, I don't think fault domains are what you think they are.  With three hosts you can only have one fault domain.  A fault domain would be if you had three hosts in one rack and then three hosts in another.  You would then create two fault domains so you could lose an entire rack and maintain data availability.  With three hosts you want, and can only have, one fault domain.

Please verify and if someone can confirm, that would be great.  Thank you, Zach.

[MartinKerr]

Hi Zach,

Thanks for your comments and you are correct in what you are saying.

So how am I able to over come this issue?

Do I need to add all 3 hosts into one fault domain? Change the vSAN storage policy to only need one fault domain?

Any help is appreciated.

Thanks

[marcelo_soares]

Hi MartinKerr,

Fault domais, as mentioned before, are only used if you have mutiple racks/chassis and want to separate your ESXi servers on this kind of organization - if you loose one entire rack, VMs are still available on the other. Fault domains must have at least 3 hosts each (or 2 hosts and a witness host in case of VSAN 6.1).

In your case, having only 3 ESX server in the cluster, you CAN'T use fault domains. You must remove all domains you created,leaving your ESX servers not associated with any domain. That way each one of them will be considered a fault domain themselves.

Let me know if this helps you on your configuration.

Marcelo

[MartinKerr]

Hi Marcelo,

I have removed all hosts from their fault domain, leaving all 3 hosts not in a fault domain.

I still receive the same error regarding needing 3 fault domains.

Do i need to change the vSAN storage policy so it doesn't require 3 fault domain, if so where, as i have already removed the line 'Number of failures to tolerate' but i still receive the same error.

Thanks

[Cormac Hogan]

Which version of VSAN is this?

If it is 6.0, can you install the health check?

If it is 6.1 (with vSphere 6.0U1), the health check is pre-installed.

Run the health check and see if there are any errors on the cluster.

[MartinKerr]

I have downloaded, installed and restarted windows services for the Health Plug-In.

But, after install i cannot do anything on the cluster, it does not even show any status? Is this a common thing where it is currently doing a job which would prevent me seeing any information?

Thanks

[Cormac Hogan]

There are a few issues.

For Windows VC, make sure you install in a window opened with "Run as Administrator". This is in the health check guide - http://www.vmware.com/files/pdf/products/vsan/VMware-Virtual-SAN-Health-Check-Guide-6.1.pdf

There is a more recent issue in the VCSA appliance - it seems that certification permissions can affect the health check. See this thread: Problem with VSAN Health Check Plugin installation

[MartinKerr]

There is no option to run the MSI as Administrator, so i logged in as the local administrator and installed.

Currently restarting the services and will update on the outcome, although not holding out much hope!

Thanks

[MartinKerr]

I have also checked the MultiCast network settings and have the below:

Do these settings seem ok?

Thanks

[MartinKerr]

So after overcoming the Plug-In installation by installing via local administrator, i now have the below:

So...... i cannot enable DRS as i only have Essentials Plus Licenses?! Any way around this or any other method to check the health of vSAN?

Not going well at the moment :smileysad:

Thanks

[Cormac Hogan]

‌Download the VIB and install it manually on each of the hosts in the cluster. The link to the VIB is in the health check guide.

Keep persevering. The health check should reveal why you cannot deploy your VMs. I suspect the cluster is partitioned. The health check will tell you why.

[MartinKerr]

And I don't need DRS enabled if I use the VIB?

[MartinKerr]

So... within the documentation it states that DRS in needed for vSAN Health Check but i am not licensed for DRS.

Any other ways i can check the health of the vSAN and resolve the issues?

Thanks

[MartinKerr]

I have a suspicion that it is to do the Network Partition Groups as i believe that they should be the same?

Is someone able to confirm this? If true, how do i overcome this?

Thanks

[Cormac Hogan]

DRS enables a rolling upgrade of the VIB from vCenter.

Without DRS, you will have to manually install the VIB. Health check needs this software on the ESXi hosts so that it can check their state.

And yes - that partition is your problem.  You need to figure out why host 61 is not communicating correctly to host 62 and 63 over the VSAN network.

If you don't install the health check, use the networking section of the VSAN troubleshooting guide to figure it out. Remember, multicast is a requirement, so if you have IGMP, make sure its working properly. This is usually the problem.

http://www.vmware.com/files/pdf/products/vsan/VSAN-Troubleshooting-Reference-Manual.pdf

[MartinKerr]

An update!

I have fixed the issue, it seems the MTU Size of the Virtual Switch did not match the Virtual Adapters (MTU Size 9000).

Now all is working fine and dandy! Thanks for your help.

Thanks