VM CPU Speed Limit - - Why?

[wentworth33]

In our environment I've been asked to limit the MHz available to VMs with multiple vCPUs. The reason I was given is that it will prevent a VM with bad software or a broken process from consuming multiple cores on the host.

My thought is that it doesn't matter if we limit the speed. If a VM is going to grab 1000MHz or 2000MHz, either way it has the core on the host. If my host has 3000MHz of power and my VM can only take 1000MHz of that power (because that is it's limit), it still has the core. I don't believe the extra 2000MHz on that core is available to other VMs until the VM with the bad process gives the core back.

With this reasoning, I should remove the limits and just let the VMs run. A bad process that will break the VM will impact the host the same whether the MHz is limited or not.

Am I correct, or not? Can anyone explain this any better, or is there any documentation I can read and use as justification?

[RParker]

1000MHz or 2000MHz, either way it has the core on the host.

Wrong VM's do not have access to the hardware, it's a software emulated time slice.

You can set limits by pools, you give the pool a limit, the VM's share the pool if any one of those VM's go over that pool limit, it will not affect ANY vm's out site that pool.

Besides.. even if an app goes wild.. that's where priorities come in, you can make this VM a lower priority such that other VM's can take precendence over the CPU.

[AntonVZhbankov]

VM will get only what is permitted. If VM has limit 1000 MHz then ESX will give VM CPU time equal to 1000 MHz, even if physical CPU runs at 3000 MHz.

If VM has 2 vCPUs then sum of both CPUs will be limited to 1000MHz.

It doesn't matter if physical CPU has more than enough processing power, VM will be strictly limited.

---

MCSA, MCTS Hyper-V, VCP 3/4, VMware vExpert

http://blog.vadmin.ru

[wentworth33]

In our current environment we have resource pools differentiated by share levels. Within the resource pools we individually limit MHz available to VMs with multiple vCPUs.

It sounds like you are saying our current setup makes some sense? Since the VM can't access actual hardware, limiting vCPU speed will help protect us against overwhelming the host with run-away processes.

Is that correct, or am I still mistaken?

[weinstein5]

also one thing to remember when you set a limit on a multiple vCPU vm you the limit is not set equally to all vCPUs - for example if you have quad vcpu vm with a limit 0f 1000 MHz each vcpu will be capped at 250 MHz (4 x 250 - 1000 MHz)

If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful

[AntonVZhbankov]

wentworth33, you're correct.

David, could you please specify official source where is said that 4 vCPU VM will get 4*250MHz? I always thought that limit will be for summary load, not with equal shares, so situation with 50/50/250/650 MHz load would be possible .

---

MCSA, MCTS Hyper-V, VCP 3/4, VMware vExpert

http://blog.vadmin.ru

[weinstein5]

Anoton let me check - the limit being divided equally was the case in VI-3 and I thought it remained true for vSphere but I will confirm

If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful

[RParker]

I always thought that limit will be for summary load, not with equal shares, so situation with 50/50/250/650 MHz load would be possible .

Actually it would make sense that the limit is at the VM level, not the vCPU level. Because if it weren't at the VM level, it would be more difficult to calculate how much share and limit to give a VM when they are in pools, then you would get skewed numbers. So it should be summary by VM.

1 VM with 1 vCPU of 1000Mhz limit would be the same as a VM with 4 vCPU, they share the same resource from the pool.

[RParker]

Is that correct, or am I still mistaken?

What you need to do is make some plans, and determine your important VM's and those that are not as important. Decide how you want the CPU to work.

We never set limits (at least not at the VM level) the pools take care of this. So you don't even really need to set hard limits, just define a pool with a given amount of resource.

So you have 5 VM in a 12Ghz pool. One VM has 4 vCPU, each CPU is 3Ghz. If that VM goes nuts one day, and pegs all 4 vCPU it just means the rest of the VM's in that pool will be starved.. UNLESS you set priorities. Priorities are based on need, and when resources are low. So if you have a VM that is taking CPU, you make it a lower priority, so it doesn't affect the other VM's ability to get CPU.

You also just made a case for why adding MORE vCPU to a VM isn't helpful, because 1 vCPU VM's will suffer, and a poorly written app in a quad vCPU VM will just peg the CPU and make that VM consume all the CPU.

adding more vCPU in a VM is more likely to cause negative impact, since ALL the vCPU need to be free at the same time for the schedule to allot time to those CPU, and thus performance isn't what you expect, so 1 vCPU may actually equate to the same performance as a 4 vCPU VM due the way ESX time slices the vCPU load.

We VERY rarely assign 2 vCPU to VM's, never for windows (because testing has proved the performance doesn't increase) and no more than 2 vCPU for Linux. WE have no 3 vCPU or more VM's, and we are a software development shop.

Physical machines are vastly different from Virtual, and since you DON'T have direct access to the hardware, the SMP works much different, I would take a close look at those 4 vCPU and analyze them CLOSELY for performance to see if you are truly getting 4 vCPU out of them.

Keep in mind that ALL the hardware is virtual, so the performance benefit for vCPU NEEDS the other hardware components to be at higher levels also, otherwise those vCPU are just basically idle most of the time. Your HBA / SCSI RAID cards are SHARED by ALL the VM's on that ESX host, so IO is ALSO distributed you don't have dedicated access either, the drivers are virtual, when CPU is limited the performance from other components will ALSO be limited, so more vCPU is not necessary.

We have Oracle and SQL, and we can get good numbers regardless of additional CPU, adding more CPU doesn't improve those numbers.