SSD with SAS cache tier, SATA capacity tier - Horizon 7 VDI

[RealQuiet]

Doing a build out for a VDI cluster and we are getting quotes for an AF-vSAN. The vSAN will be running in RAID 6 with compression, dedup, and encryption enabled with a workload of Windows 10 full-clones and persistent data.

The vendor is doing a mix with the following:

Cache: 800GB SSD SAS 12Gb x 2 (4k reads @ 240k, 4k writes at 130k)

Capacity: 1.92TB SSD SATA 6Gb x 6 (4KB reads @ 90k, 4KB writes @ 50k)

Each node will have two disk groups comprised of a 800GB cache drive with 3x1.92TB drives for capacity.

I have some room in the budget to jump up to 1.6TB SAS drives for the capacity tier. This increases cost per drive and I lose 1.8TB of space per a node. The gain is a capacity tier where each drive has 4KB reads @ 200k and 4KB writes @ 90k.

Does anyone here have experience with using a mixed SAS cache and SATA capacity tier for an AF-vSAN deployment? For VDIs, would you save money and use a mix of SAS and SATA or SAS across both tiers?

[m49808]

Cant comment on use case with VDI, but i have a similar setup. actually 2 disk groups on each host with 1x400GB SAS Cache 6x 1.92TB SATA capacity per disk group. So far less cache than what you are thinking (our IO requirements are lower likely)

I've heard that the queue depth of SATA could cause issues. I've read that mixing SATA/SAS can have complications with an expander (causing both to negotiated down to 6gbps). But i havent seen any real world issues. It screams.

There is some good info about mixing sas/sata in this thread. The issue in the thread turned out to be firmware/driver, not related to the mixing drive types (i think)

https://www.reddit.com/r/vmware/comments/86pxid/hba330_vsan_issues_controller_disks_resetting/

[RealQuiet]

I actually pushed up to 800GB, they started with 400GB and I just thought I could push performance more if it had the 600GB to leave objects in as long as possible. Personally I feel that losing a disk group because a cache drive wears out is a little much. Especially when I consider that if it wore out, and vSAN is doing this evenly, then the remaining cache tier drives may follow. If anything, I want the capacity drives to wear out first and, if the system is still around, then the caching tier.

I spoke with the vendor about my SATA concerns, and they still recommended SATA over SAS for the capacity tier. They put me at ease over that, of course I still want to hear from other's experiences :smileyhappy:

Anyone have VDI experience with SSD SAS caching tier and SSD SATA capacity tier? Your input would be appreciated.

[bryanvaneeden]

I am wondering what AF-vSAN config you are getting quoted. If I check the sizings, I'd say the AF-8 version.

Do you guys have an IO-Profile you could match to the sizing guidelines? Check out this blogpost for the new sizing guidelines, it used to be 1:10 cache:capacity ratio but it has changed to a workload profile ratio.

https://blogs.vmware.com/virtualblocks/2017/01/18/designing-vsan-disk-groups-cache-ratio-revisited/

You might be able to get away with a lower cache disk (just make sure it is an Write Intensive one) depending on your IO-profile needs.

Just like mark I can't really comment on using mixed SATA/SAS drives in an environment. But I can make some statements that makes sense to myself:

What is supported as far as I know:

- Using all SAS for cache tier

- Using all SATA for capacity tier.

Using a mixed SAS/SATA capacity tier is not something I would do. In a vSAN environment all blocks get written on all nodes. In the case a VM would want to access a certain block and needs to read from multiple hosts, you would insert latency because the SAS drive responds a lot quicker then the SATA drive.

And as always, remember the HCL for the environment, but I would assume that your vendor is doing that for you.

[RealQuiet]

Yes it is an AF-8 sizing. I am actually gathering IO data to see what our needs our. Gathering the IO data this week. Working on a data capture for at least a week. We have a follow the sun model, so workloads vary depending on the shift. Thank you for your link, it accounts for changes to SSD endurance and now I am seeing errors in my calculations for the caching tier change. I may revert it back to 400GB... but I do like the endurance and ability of a larger caching drive to hold objects longer before having to write them to the capacity tier.

I was using older information from: https://cormachogan.com/2015/05/19/vsan-6-0-part-10-10-cache-recommendation-for-af-vsan/

[Therese21]

Cant comment on use case with VDI, but i have a similar setup. actually 2 disk groups on each host with 1x400GB SAS Cache 6x 1.92TB SATA capacity per disk group. So far less cache than what you are thinking (our IO requirements are lower likely)

I've heard that the queue depth of SATA could cause issues. I've read that mixing SATA/SAS can have complications with an expander (causing both to negotiated down to 6gbps). But i havent seen any real world issues. It screams.

---

krogerfeedback