vSAN1

 View Only

  • 1.  Adding 3rd server to cluster with all Flash in a Hybrid vSAN

    Posted Apr 06, 2022 01:43 AM

    So my current situation is the following

    2 HP Gen10 Servers in a Hybrid vSAN configuration,

    We have purchased

    1 New Server, with all Flash drives

    24 new SSD drives to replace the HDD's in the Original servers. 

    I understand the process to change the Disk group to SSD, but what I am uncertain on is the new server with all Flash participating in the vSAN. I would like to get this new server into the vSAN so that I can move the VM's off 1 of the hosts to the new server, so that I don't have to worry about any screwup and cause data loss. I'm crazy paranoid on this part right now because I just don't have a good understanding on the new server. 

    I keep seeing that you can't have a mixed vSAN environment, thus my many questions about the new server participating. 

    Would appreciate any and all feedback. 

     

    Thanks all. 



  • 2.  RE: Adding 3rd server to cluster with all Flash in a Hybrid vSAN

    Broadcom Employee
    Posted Apr 06, 2022 01:44 PM

    What version of vSAN are you using?

    are you converting from 2-node to full 3-node vSAN  and distributing the 24 SSD drives among all 3 servers?

    or are you going to remain at 2 nodes (with 24 drives split between 2 nodes) ?

    I assume this is 2-node vSAN cluster with a witness appliance?

    if you are going from 2-node to three node  conversion have a read off this 

    https://blogs.vmware.com/virtualblocks/2016/02/26/scaling-2-node-vsan-robo-to-a-3-node-or-more/



  • 3.  RE: Adding 3rd server to cluster with all Flash in a Hybrid vSAN

    Posted Apr 06, 2022 01:56 PM

    Yes, currently in a 2 node environment (6.7) with a witness installed. My concern was more of mixed disk groups in the vSAN. the new server is all Flash, where as the current 2 nodes are Hybrid. 

    The plan in my head is to install the new hypervisor into the vSAN and use this new server to move the current VM's on 1 host to, so that I can then delete the disk groups on the old HV's, install the SSD's and re-add the disk groups. I was just unsure if I was allowed to add the 3rd server to the vSAN as it is all Flash. 



  • 4.  RE: Adding 3rd server to cluster with all Flash in a Hybrid vSAN

    Posted Apr 07, 2022 01:42 PM

    "I was just unsure if I was allowed to add the 3rd server to the vSAN as it is all Flash."

    No you can't. One cannot mix Hybrid and All-Flash Nodes. Replacing HDD's with SSD's in-situ does not convert the vSAN Datastore to an all-flash. It will remain a Hybrid type because the VisorFS of type "Hybrid" is different from "All-Flash". Both have different capabilities and are not interchangeable.

    Your best course of action would be to use a NAS and connect it via NFS to all nodes and migrate all VM's to it. Assuming the NAS has enough horsepower and 10gig networking, the VM's can remain running.
    You do not need to buy or tent a QNAP or Synology etc. as you can re-purpose an older server and build a TrueNAS Core box with it. Throw some RAM in there and use HDD's in the RAID10 equivalent of ZFS (multi-vdev/striped mirrors) and the thing will be fast enough to host all your VM's for a couple of hours.

    Then, remove the HDD disk groups so that is nothing is left. Take out the HDD's and replace them with the SSD's.

    At this point you can do 2 things:
    1.  Just re-create the vSAN Datastore as All-Flash and it will remain a 2-Node + Witness. Then convert to 3-Node later on.
    2.  Install and add the 3rd new Node and build a 3-Node cluster. You will not need the Witness anymore obviously.

    After the vSAN Cluster has been re-created, hot-migrate the VM's from the NAS back to vSAN en detach the NAS NFS.

    I have done this entire operation about two years ago so I can assure you that it works. If you do the right steps in the right sequence, the VM's do not need to go down at any point. The NAS becomes your new shared storage, keeping everything running while you perform disk-surgery on the vSAN Datastore.



  • 5.  RE: Adding 3rd server to cluster with all Flash in a Hybrid vSAN

    Posted Apr 07, 2022 08:17 PM

    , So, is it technically feasible to create a mix of Hybrid and All-Flash Disk-Groups in a cluster? Yes.

    Is this supported? No (https://kb.vmware.com/s/article/2150329).

     

    I will be straight honest and say that I don't think anything really bad would happen from doing this temporarily (from seeing crazy things in support like people marking HDDs as SSDs and using wifi dongles instead of NICs and it not being as bad as it could be ), but still, unsupported and thus you are (mostly) on your own if it goes badly.

     

    Hence, probably best to go with something like  suggestion if possible - there are of course other things you could do if you don't have other storage e.g. you could configure the new node with the disks as a VMFS datastore (or FTT=0 vSAN + backups), migrate all the data to it, reconfigure the original nodes as All-Flash, migrate the data back, remove the VMFS, add the node to the cluster and configure Disk-Group(s) as All-Flash.



  • 6.  RE: Adding 3rd server to cluster with all Flash in a Hybrid vSAN

    Posted Apr 07, 2022 11:52 PM

    So I ended up putting the 3rd HV online and into the vSAN and so far it is just fine. Working on moving the VM's now over to the new HV so that I can put #1 in maintenance mode and start removing the disk groups and installing the new Flash drives, and then re-adding the disks as flash to the vsan. 

    So far so good. lol

    Appreciate all the comments.