VMFS Datastore and LUN Best Practices (I/O Queue) ?

[SteveAntle]

Hello,

I have a few quick questions regarding VMFS datastores and SAN LUN best practices.

Assumptions:

1. iSCSI SAN (Single Storage Processor)

2. 2 Disk Pools (2 x RAID5 @ 3 Disks/RAID5)

3. 4 LUNS (2 LUNs/Disk Pool)

4. LUN Size: 500GB/LUN for a toal of 2TB storage

Question:

1. On what datastore would VMs have greater disk performance?

A. Datastore1 created on LUN0 on Disk Pool 1 and Datastore2 created on LUN1 on Disk Pool 1 OR

B. Datastore1 created on LUN0 on Disk Pool 1 and Datastore2 created on LUN0 on Disk Pool 2?

I guess what I'm trying to ask here is having LUNs on separate Disk Pools faster than having all the LUNS on the same Disk Pool? If VM A's .vmdk exists on Datastore1 from B above, would placing VM B's .vmdk on Datastore2 from B above improve overall performance for both VMs? On the other hand, placing both VM A and VM B's .vmdk's on Datastore1 and Datastore2, respectively, from A above still an efficient option? This leads me into question 2.

2. Does placing datastores on seperate LUNs divide the SAN LUN I/O Queue over all LUNS? In other words, will the SAN access all LUNS in parallel therefore increasing performance vs. having all VMs on a single shared LUN with a single I/O Queue?

Thanks,

Steve

[a_p_]

If there was no workload on the second LUN in each pool, I would say place one datastore on pool 1 and the other one on pool 2. However, regarding your setup, I would definitely not create 2 RAID 5 pools with 3 disks each. Due to the way RAID 5 works, a RAID 5 with 3 disks is one of the slowest RAID configurations you can have. It may make more sense to create 1 RAID 5 pool with all 6 disks.

André

[SteveAntle]

André,

Thanks for the reply but you didn't answer my question. I'm asking what the most efficient datastore configuration given the two possible configurations (A & B). Also, I'm wondering how LUN I/O queues work when the LUNS are located on the same disk pool vs. different disk pools. Can anybody answer that question? Ultimately I will be installing Microsoft SQL Server as a VM and I want to be able to create efficient datastores.

Thanks,

Steve

[a_p_]

It's not easy to answer the question without knowing the technical specs. What kind of storage (vendor/model) are you using? Does the controller have multiple ports? Does it support the Round-Robin or only MRU? Most storage vendors who support ESX(i) offer best practices for an optimized setup.

André

[SteveAntle]

André,

I'm not looking for a specific answer - I want a general answer to this question; will LUNs on different Disk Pools perform better than similar LUNs located on the same Disk Pool? (eg. LUN0 and LUN1 on Disk Pool 1 contain .vmdks vs. LUN0 on Disk Pool 1 and LUN0 on Disk Pool 2 containing the same .vmdks - what config has the better performance?). And just to answer your question, it's an EMC AX150SCi (single SP, 2 ports, SATA disks, write cache enabled).

Thanks,

Steve

[EXO3AW]

Hi Steve,

why do you bother about hypothetical questions like "what's more performant" "how about IO queues" when using SATA disks in a 3-spindle RAID 5?

The AX150SCi is a NAS, and qualifies IMO only as a testing lab. If you stick to a 2x 3-spindle Raid5 i expect DAVG/wr issues way before anything else

Like a.p already mentioned the most "performant" way to use your hardware could be to use a full-spindle Raid 5, but since you want to deploy a MSSQL it might also be feasible to create a 2-disk Raid1 + 4-disk Raid10 or even 3 Raid1 - it all depends on the IO, Burst, and access characterstics of your specific MSSQL server - not to mention your iSCSI Infrastructure as well.

If i have to P2V one or more "small" machine to a single ESXi i almost always go with a 6-spindle R5 "Fire and forget"-solution. If performance is an issue, i might switch to a 6-spindle R10 - but that rarely happened.

I know, this is actually of no help to you, but there is no "general way" to go - it all depends always on your HW, requirements and workload.

As a very very general answer:

- Spreading vdisks across different LUNs is recommended best practice as long as the storage performance is sufficient.

- Limiting the datastores to 500 MB is recommended for multiple vms.

[SteveAntle]

Thanks for the reply EXO3AW.

I am speaking in general terms because I'm looking for the best way to provision my SAN properly for a small MS SQL server installation without getting into too much detail. I wasn't sure if placing a VMFS datastore on a LUN on its own array vs. on an array with other LUNs would improve performance.

In reality, I will be creating a VMFS datastore on a single LUN on a RAID 1 array for the SQL DB (the DB is ~ 50GB in size). I will be creating another VMFS datastore on a single LUN on an additional RAID 1 array for the SQL LOG files. Finally, I will be placing the SQL binaries on a VMFS datastore on a LUN on a 6 spindle RAID 5 shared with other VMs. The SAN is connected to the iSCSI network via 2 x 1Gbps ports. Each ESX host (there are two in my environment) is connected to the iSCSI network via 2 x 1Gbps software iSCSI initiators. This configuration should be sufficient for my environment (there should be more than enough I/O for the SQL server - I hope).

Any comments?

Steve

[EXO3AW]

Hi Steve,

for a small environment that sounds like a feasible way to place stuff, although it might not be the best solution - it still all depends on your environment :smileywink:

In this setup you're primarily limiting your SQL DB to <100 SATA IOPS (before cache) limits. There's a great wikipedia article with charts about IOPS.

If you can assure that's more than enough for your DB, then you shouldn't worry about anything.

On the other hand, you might as well be better off with a R10 for Data+Logs if the SQL DB has so low requirements.

[SteveAntle]

So if I decide to place the SQL DB and LOGS on two different LUNs (on a dedicated RAID 10 array) will I essentially be increasing the total IOPS from ~ 90 IOPS (single 'active' SATA drive in RAID 1) to ~180 IOPS (two 'active' SATA drives in the RAID 0 portion of the RAID 10 array)? That being said, contention will be created between the DB and LOG reads/writes as they will share the same array... will this array sharing cancel the potential performance again from the increased RAID 10 IOPS (generally speaking)?

Steve

[EXO3AW]

Generally speaking you're trading the 90 guaranteed IOPS per VMDK for 180 maximum IOPS, but shared among 2 VMDKs.

This tends to be a quite theoretical and sometimes nearly philosophic question, whether to trade off this for that.

If you think about it further, it's quite similar to placing vCPUs and VMs on ESX hosts - is it better to share some faster CPUs (=more expensive systems) or give everything a dedicated baseline CPU (=more Hardware) ?

If i stick my neck out, i would say that the difference between the already mentioned constructs is only marginally visible and you might probably be very happy with a flat 6-Spindle R5 datastore as well.

But the specific difference between 2 R1s and a R10 with 2 LUNs or even a R10 with 1 LUN but 2 VMDKs is hardly worth thinking about.

Like i already said:

If speed and performance is an issue (MSSQL IOPS requirements) then you're already wrong with the SATAs and should rather consider investigating in a more performant array than trying to squeeze out the last bits out of your SATAs.

Don't get me wrong, surely SATA can do miracles in small environment, but especially if i notice a 50GB MSSQL DB to P2V i'd first gather some perfmon data and plan my storage accordingly even more than i'd normally do in SMB environments.

You won't get a PRIUS to take the load of a SUV, no matter how long and diligent you plan to distribute the load.

[EXO3AW]

Can you access and test the storage beforehand ?

If possible, than create the different layouts beforehand and test them with one of the various stresstest / benchmarking tools.

Work yourself from bottom to top - i.e. first set up different storage layouts on the array (like R1+R1+R5, R10+R5, R5 only, R10 only ...) and compare the throughput, burst size, R/W latency / IOPS for different blocksizes and so on.

Then you might as well test different LUN/partition scenarios (like flat R5 single LUN with multiple VMDK vs. R5 multiple LUN with single VMDK), although this test would need to have at least two benchmarking workers running or your results are skewed.

These test will provide you with accurate information about your hardware and its performance characteristics and you can decide which trade-off (throughput vs. versatility vs. IO) suits best to your specific requirements.

[SteveAntle]

Thank you for taking the time to answer my questions. I  will be able to access and test the storage beforehand as I am the person who will be setting it up. I will take your advice and run various IO tests on the different storage configurations like you had mentioned. I'll let you know how it goes.

Thanks again!

Steve