Best practice - Maximum VMs per lun ?

[NHCdmont]

I'm having a quick look at our Sphere environment and trying to work out why VM servers are running a bit slow...

Does anyone here know what the recommended Maximum VMs per LUN is ? ( sorry but I'm new to Sphere ) I know that VM View recommend only 10 VM desktops per lun but I have no idea about VM Servers...

Thanks for any advice

Dmont

[AWo]

I would also look at how many virtual disks with guest swap files are on one LUN and distribute them. Most often these are the system drives.

Another issue is to align the VMFS and guest partitions to have as less I/O as necessary: http://www.vmware.com/pdf/Perf_Best_Practices_vSphere4.0.pdf Page 26

AWo

VCP 3 & 4

\[:o]===\[o:]

=Would you like to have this posting as a ringtone on your cell phone?=

=Send "Posting" to 911 for only $999999,99!=

Added the alignment part

[MattG]

10-16 VMs per LUN is the general BP recommendation. Though you should check with your SAN vendor as they may have more specific recommendations, for example: 10-16 VMDKs per LUN vs VMs.

-MattG

If you find this information useful, please award points for "correct" or "helpful".

[AntonVZhbankov]

>10-16 VMs per LUN is the general BP recommendation.

Matt, there is no such recommendation. It all totally depends on VMs I/O load and SAN performance. For example I have about 40 VMs on single LUN (16 FC SAS 10k disks, RAID6) and there are no signs of "running slow".

---

MCSA, MCTS Hyper-V, VCP 3/4, VMware vExpert

http://blog.vadmin.ru

[MattG]

The recommendation is not speed related, it SCSI reservation related. Too many VMs on a single VMFS and you will get SCSI Reservations which will disk access issues.

With each version of ESX VMware is reducing the SCSI Reservations so these numbers may go up with newer versions.

From the vSphere 4 Design course:

VMware has historically recommended 12-15 virtual machines per VMFS datastore. However,

updates to VMFS and storage typically increase this value over time.

-MattG

[DSTAVERT]

The backyard mechanics torque recommendation. Tighten the bolt until it strips the thread then back it off half a turn. :smileygrin:

[RParker]

The recommendation is not speed related, it SCSI reservation related

This so called "SCSI Reservation" has been bounced around, and it's been totally misinformed. It's NOT true. You can get a SCSI reservation with just 1 VM on 1 LUN, it has to do with OPEN FILES. Nothing at all to do with the number of files on a LUN or Volume.

I called QLogic and Emulex, there is no limit this small, it COULD be 256 but that is an HBA connection, and that ONLY refers to number of queued commands, so there STILL isn't a limit to speak of.

Someone started this rumor and it needs to stop. You open a file that locks a file DURING the open/ close process. Once a file is opened for reading it's open, there is no longer a CONTENTION status because there aren't other IO trying to fight to lock that file for opening.

LUNs can have millions of files, Volumes can have millions of FILES, why all of a sudden can you only have 30 files or less on VMFS makes ZERO sense.

This 16-20 Limit is based on OLD theories which no longer apply, Anton and I both have proven them false, and its going off of LUN sizes that are smaller than we have today, because of DISK size. That's ALL it's based on, old standards and traditions not moder technology OR LIMITS.

[DSTAVERT]

Here is one of many KB articles that deal with SCSI reservations. http://kb.vmware.com/kb/1005009

I would say that it does exist and can cause issues, but probably not at the level it of importance that some have attributed to it.

In the end the best suggestion is "It depends". Test to see what works well for you in your situation.

[MattG]

SCSI Reservations aren't voodoo, they happen. I have seen it.

Best practices are there to help avoid the potential issues. You can either create a VMFS design to avoid them or deal with them when they start affecting your environment.

For example you could have a VMFS with 40 VMs that works fine for years and then you need to enable snapshots on x VMs on that LUN. Snapshots cause reservations. You are now in a situation where you may have performance issues caused by your design.

It always depends, but that why when possible you try to follow BPs.

-MattG

[AntonVZhbankov]

Matt, SCSI reservations still happen, however their number is greatly reduced in latest VMFS versions (3.31 in ESX 3.5 and 3.33 in ESX 4.0).

>It always depends, but that why when possible you try to follow BPs.

What I'm trying to say - there is no BPs for VMFS anymore.

---

MCSA, MCTS Hyper-V, VCP 3/4, VMware vExpert

http://blog.vadmin.ru

[AWo]

I had to remove a lock left on a EMC Sym, once. The EMC guy and I watched closely when locks happen etter he watched) and that was almost never the time. Only during special operations like creating a snaphost a lock appeard for a short time.

AWo

VCP 3 & 4

\[:o]===\[o:]

=Would you like to have this posting as a ringtone on your cell phone?=

=Send "Posting" to 911 for only $999999,99!=

[MattG]

Scsi reservations happen on snapshots, power on, thin provisioning,

vmotion to name a few.

Under normal circumstances you shouldn't have enough to effect

performance. However if you follow no BP and decide to put hundreds

of VMs, especially desktops, on the same VMFS then you run the risk.

It is annoying / sporadic issue that you should have in mind when

designing your volumes.

[geddam]

"You can get a SCSI reservation with just 1 VM on 1 LUN"

Parker, I pretty amazed to see this statement, Can you correct me....'

If you have one VM per LUN and One host is accessing it, where is the point of SCSI reservation conflict.....?

I dont agree with this statement,

Yes, I agree the fact that your HBA and Sp@array and cache algorithms makes some difference here...and yes the Snapshots.

Thanks,,

Ramesh. Geddam,

VCP 3&4, MCTS(Hyper-V), SNIA SCP.

Please award points, if helpful

http://communities.vmware.com/blogs/rameshgeddam

[NHCdmont]

Thank you all for your replies, some varied responses ;o) this should help me fault find..

Many thanks

[NTurnbull]

Hi, are you sure that your disk subsystem is causing an impact on your vm's performance? What storage have you got? How loaded are your hosts? etc...

Thanks,

Neil