Licensing and Hardware Question vSAN

[mrlizard08]

Hello.  I hope I'm posting this in the correct spot, I wanted to see if someone could check this and make sure that I've got the right hardware and licenses, and that I'm not over provisioning the licenses, as well as answer a couple of questions.  I'm looking into setting up a 2 node vSAN.  There will be a total of three sites.  Two of which will be in the same city, the third will be farther away (250ish miles).  My plan is to have one host in each of the two sites in the same city, and then put the witness vm in the third site.  For the licensing I'm thinking of Vmware HCI Kit Essentials.  If I understand it correctly that should get me everything I need for what I'm wanting to do.  I'll put the hardware below that I'm thinking of using, as well as link speeds between sites.  In addition to this I would like to have a third host setup for disaster recovery in the same site as the witness vm, I'll put the info for that below as well.

Hardware -

Site A and B (Main sites in same city):

Dell PowerEdge R540 (all storage is internal) -
Cache - 2x 1.92 TB SATA SSD's in RAID 1 (configured on PERC H750P).  1.92 TB RAW, 1.78 TB approximate usable capacity.
Data - 10x 2.4 TB 10k SAS HDD's in RAID 6 (configured on PERC H750P).  19.2 TB RAW, 17.4 TB Usable capacity. 

Site C (Witness and DR site in separate city):

Dell PowerEdge R540 (all storage is internal) -

Data - 10x 2.4 TB 10k SAS drives in RAID 6 (configured on PERC H750P).  19.2 TB RAW, 17.4 TB Usable capacity. 

Witness VM will be setup on this host, and I plan to use this host for DR, so I would like to replicate VM's to it.  The RPO I'm shooting for is around 6 hours, but I can go up to 12 if the requirements get to crazy.

Link Speeds -

Link between site A and site B - Layer 2 link provided by ISP at 500 Mbps (would like to lower this though if possible for cost concerns)

Link between site A and site C - Layer 2 link provided by ISP at 100 Mbps

Link between site B and site C - Layer 2 link provided by ISP at 50 Mbps.  This link is here because, as I understand it, at least one host in a 2-node setup has to have connectivity to the witness for it to work.  So if site A goes down, then site B will be able to continue to handle the workloads as long as it has connectivity to the witness.

Licenses -

This was mentioned above, but I figure I'll put it here as well just in case.  Vmware HCI Essentials kit.

Questions -

My questions, summed up, are this.

1. Will the hardware I specified work for a vSAN setup?  The goal is that if site A loses power or internet, site B, and the rest of the remote sites, can continue working from site B.

2. Are the link speeds I mentioned good enough, can they be toned down somewhat?  I know the 100 Mbps is overkill for only witness traffic, but I plan to use that link for DR replication traffic as well. 

3. There is around 12 TB between all the servers at site A that we would be consolidating into this infrastructure as part of this setup.  The initial sync for the vSAN, and the replication for the DR host, I imagine will take quite a while.  Is it recommended to transfer all this data to the empty host, or should it be seeded?  If seeding is recommended, what is the recommended method for this?

4. Is the link between site B and C necessary?

5. Will I be able to do disaster recovery replication using VMWare's built in replication functionality, using the specified hardware and license package?

6. Will the specified license package allow me to use the vSAN like I've mentioned?

7. If a fail over to the DR site is ever necessary, what would the failback procedure be?  If someone has a link to a guide or article I would greatly appreciate it.

 

If I can provide anymore info just let me know. 

[Scott Vessey]

As your post probably sits best in the area for vSAN, I have reported it to the moderators asking them to move it there.

[mrlizard08]

Ok, thank you.

[IRIX201110141]

1. Your setup describes a stretched cluster which means vSAN Enterprise or up licensing. Your HCI Essentials only contains vSAN Standard and is not suitable.
2. The H750p is not a good choice. You must check if its on the HCL and whats needed to re-configure it from RAID to Pass-trough mode. I have seen that latest h755 can convert disks from raid to non-raid. VMware not support having a HBA which offer both modes at same time to the ESXi.  Suitible HBA are HBA330 (14Gen.) and HBA355i(15Gen.) for example. These are simple pass-trough HBAs.
3. All Data sites needs having access to the Witness. The needed bandwidth depends on the number of objects. There is a max. latency value. Just read the docs.
4. Your Cache disk with 1.92TB sounds like a ReadIntensive one. Be carefull if its also a "AG" one. You need to check with your Dell sales Prep if your disks is on the HCL for a CacheDisk.
5. For replication there is vSphere Replication.... or check your VM Backupproduct because most Software not only can create backups. Veeam for example can also replicate your VMs if you like.

Regards,
Joerg

[mrlizard08]

Thank you for the reply.

Had some follow up questions/statements.

1. Does each server need a vSAN Enterprise license?  Since I would be purchasing the vSAN Enterprise license separately, does this mean that I would need to purchase the vSphere and vCenter licenses separately?  Or can I just use the Regular Essentials Plus Kit (without HCI) for the vSphere and vCenter licenses?

     a. As a side note I noticed that in the vSAN documentation it said that stretched clusters are supported in vSAN 8.0 standard for 2-node clusters, did I misunderstand that part, or is my setup different than a 2-node cluster like they are talking about.  Here is a link to the document I'm referring to: https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/products/vsan/vmware-vsan-licensing-guide.pdf

2. So this means that the RAID would be handled by vSAN directly, and not at the hardware level by the controller.   I was checking the vSAN ready nodes in the compatibility guide and noticed that they usually have the HBA355 or HBA330 configured on it.  So I feel pretty safe selecting one of those.

3. For the bandwidth I was reading some documentation which had a formula to use.  Just based on the initial numbers I think I should be good with 500 Mbps, but I've got performance loggers running on all servers so I can get a total iOPS and bandwidth usage figure for a 24 hour period.

4. If by "AG" you mean agnostic I don't know if there is a way to specify a particular brand from Dell, I will contact Dell and look into this, VMWare seems to favor Toshiba based on documentation.

5. I have used Acronis replication in the past and it worked without issue.  If vSphere replication is pretty much the same thing as the replication from a third-party vendor, then I'll stick with that.  I just figured switching this back to the main site might be easier if the replication is being handled by VMWare.  If I'm wrong about that though then, again, I'll just stick with Acronis as I'm already familiar with how it works.

[IRIX201110141]

Answers:

1. It depends. For vSAN7 every Hosts of the Cluster needs a vSAN LIC even if no storage (only Compute) is provided from such Hosts.  This topic might be changed with vSAN 8. Please ask your VMware Sales Prep..... there maybe a different because of ESA and OSA in vSAN8.
   Also in vSAN7 there is a 2 node Cluster... not sure if the stretched config already was supported there.
2. Correct. vSAN would like to see all Disks as a single .... not behind typical HW RAID. Depending on your vSAN Lic, Number of Nodes and your requirements you got a "Network RAID" which can be like a RAID1 or ErasureEncoding 5/6.
3. Bandwith for the dedicated Witness isnt that much. But it sounds that you also want to use it for replication so be sure that you have some kind of Bandwithlimitier in place. With vSAN always there is the vDS available wich offer traffic shapping.
4. Yes.... i mean agnostic. My Dell TSR can me tell the original SKU of all agnistic drives so i can verify if they are supportet as Cache or Capacity disk. But... that only works when using Ready Nodes and is not future proof if you use a plain PowerEdge configuration
5. vSphere Replication is a little bit different. Its not part of a Essentials Plus.

Regards,
Joerg

 

[Duncan Epping]

1. No, it has not changed with 8.0 Every host in a vSAN Cluster needs to be licensed, this applies to both the ESA version and the OSA version.
2. Correct, vSAN Stretched Cluster is supported on a 2-node configuration (1+1+1) with the Standard License. This is ONLY the case for a 2-node configuration with 3 nodes or more you need an Enterprise License. This applies to all versions of vSAN!

[IRIX201110141]

1. Hmm... Dell in the DCS Frankfurt tell something different. But maybe i miss understood something

Regards,
Joerg

[mrlizard08]

Thank you both for the answers.  I think I've got it figured out on how I'm going to do it.

Site A:

R550 with HBA355.  10x 2.4 TB 10k SAS HDD's for data.  2x 1.92 TB SATA SSD's for Cache (Read Intensive).  I will try to get the specific SKU from Dell for the cache drives to verify compatibility.

This server will be licensed with vSAN Standard and the vSphere license that comes with Essentials Plus.  I will use Acronis for replication as it sounds like that will simplify the licensing.  Plus it doesn't care about hardware.

Site B:

R540 (Currently has PERC H750, will be swapped for HBA355) - 10x 2.4 TB 10k SAS HDD's for data.  2x 1.92 TB SATA SSD's for Cache (Read Intensive)

This server will be licensed with vSAN Standard and the vSphere license that comes with Essentials Plus.  As above I will use Acronis for replication.

Witness site:

R550 - 10x 2.4 TB 10k SAS HDD's.  Will be configured in hardware RAID and act as replication target for Acronis.  This host will also have the witness VM on it.  This host will be licensed with the third vSphere license that comes with Essentials Plus.

 

If you see any issues with that setup let me know. 

My final question would be, in your opinion, is it better to setup vCenter at one of the main sites, or at the witness site?  I was thinking the witness site because then even if one of the main sites goes down, I can still manage one of the sites through vCenter.  However I'm unfamiliar with the best practice for a vSAN setup like this.