VMware vSphere

 View Only
Expand all | Collapse all

vmware hyperthreading total resources

  • 1.  vmware hyperthreading total resources

    Posted Dec 02, 2009 01:42 PM

    Hi,

    I'm after a simple explination on how ESXi 4.0 works with hyperthreaded CPU.

    eg.

    Intel Xeon X5560 Quad @ 2.8GHz

    CPU Cores: 8 CPUs x 2.793GHz

    Processor Sockets: 2

    Cores per Socket: 4

    Logical Processors: 16

    Hyperthreading: Active

    Using ESXi 4.0 Free, with 2 CPU license

    I'm using vSphere client.

    Why is my total capacity only 8 x 2.793 = 22.344GHz (ie why is it not Logical CPU(16) x 2.793GHz)

    If I create 4 VM each with 4 vCPU, ie a total of 16 vCPU. Then run a cpuhog on each VM I only have around 23GHz being used. each of the 16 cpu under the performance tab is running at just over 1GHz, I would have thought this should be closer to the 2.8GHz..

    Thanks,

    Jason



  • 2.  RE: vmware hyperthreading total resources

    Posted Dec 02, 2009 02:04 PM

    Hyper-threading enables HT aware OS to schedule stalled CPU resource to another thread - for example, making use of cache miss or jump prediction miss penalties (whilst waiting for RAM to provide data, the CPU can execute code in-cache for another thread). It is absolutely not another core, which is an entire execution unit (like a processor, in the old days).

    Also assigned 4 vCPU is a really bad idea, since ESX must have four idle cores available to schedule any resource to that VM. Since this server has 8 cores in total, only two VMs can run concurrently (and then only if no CPU resource is needed by the underlying host).

    Much better overall performance will be realised assigning 2 vCPU per guest, or better yet 1 vCPU per guest, depending on their actual application needs.

    Please award points to any useful answer.



  • 3.  RE: vmware hyperthreading total resources

    Posted Dec 02, 2009 02:14 PM

    Thanks for your answer.. But I have read in previous posts that you can have more vCPU then total cores.. I am at the moment running 4vm with 4vCPU each with no problems, even when I put each vm under load, ive had no issues.

    I have set no reservations and no limits, and given each the same share for their cpu resources.

    I do realise their is a difference between the physical and virtual worlds, but if I was to have 1 OS on this physical box (ie no vmware) then it would have 16 x 2.793GHz cpu's and using a cpu hog script I could have all CPU fully utalized...



  • 4.  RE: vmware hyperthreading total resources

    Posted Dec 02, 2009 02:23 PM

    Can say, that you can face with perfomance troubles using more than 1 vCPU per VM.

    StarWind Software R&D



  • 5.  RE: vmware hyperthreading total resources

    Posted Dec 02, 2009 02:28 PM

    You can indeed have many more vCPU than cores, up to twenty times more I think, but there is a fundament issue here, vmware nor any other hypervisor can add real processing capacity. If you have four 4 vCPU VMs, only two (at best) will actually be running concurrently on an 8 core server. In other words, when all vCPU are fully loaded, each VM will receive at best 50% of 4 physical cores of processing capacity. It cannot suddenly do more actual work because it is somehow virtualised than the underlying hardware can complete.

    Re CPU hog, yes it will start 16 threads of execution on a physical server install on this machine, and yes these will all show 100% load, but that is mearly an indication that there is no spare capacity on any thread to run the Windows idle thread. In fact 16 'cores' throughput will not be achieved, at best it is 9 to 10 physical cores worth of actual throughput. In a time-line view,

    - Windows starts the 16 threads on the 16 processing queues

    - Only threads 1, 3, 5, 7, 9, 11, 13 and 15 are actually started, since there are only 8 cores

    - Thread 1 (say) stalls waiting for memory fetch

    - Core 1 starts work on thread 2 whilst it is otherwise stalled

    - Once thread 2 similarly stalls, it picks up again on thread 1

    and so on with the other cores. Obviously in reality it's much more complicated than that, but this is the jist of what is happening. Note there is no time spent running the WIndows idle thread, hence the CPU load shows as 100% in task manager for all available CPUs (which equals threads in this case).

    Please award points to any useful answer.



  • 6.  RE: vmware hyperthreading total resources

    Posted Dec 02, 2009 02:55 PM

    Thanks for your responses it is starting to make sence...

    So if I create 4 VM, 3 with 2vCPU and 1 with a single vCPU. if I keep no reservations and no limits with equal shares.. Then if I ran the 2vCPU VM under load then it would only use 2x2.793GHz, and if I ran the 1vCPU under laod it would get 1x2.793GHz. I thought one of the main reasons to virtualise is so un-used resources can be utalized by other VMs, but what have read here is that this will not work..ie I didn't want the load averages of one VM to go through the roof when the other VM's are under no load.

    @weinstein5 where is the ready time reported in vSpere client?



  • 7.  RE: vmware hyperthreading total resources

    Posted Dec 02, 2009 03:08 PM

    Virtualisation does indeed make better use of resources - already in this example 4 VMs can be run with one physical server.

    But the whole concept is to make use of idle resources. Servers that are running at high multi-core CPU utisation in the physical world simply are not good candidates for virtualisation.

    However typically we see physical servers with (say) 30% average CPU load across the entire day. Then it follows that three such hosts could probably be virtualised. Of course many servers have next to no utilisation, often domain controllers and file servers fall into that category so consolidation ratios could be much higher.

    Please award points to any useful answer.



  • 8.  RE: vmware hyperthreading total resources

    Posted Dec 02, 2009 03:37 PM

    I agree that virtualisation does make better use of resource.. But from what I can see there is still a far way to go to make best use of resources..

    Sorry for the ingorance, But I have been using Solaris for the last few years and with their virtualisation each VM (or container) can see and use all the host CPU, this can obviously be resource controlled with limits and cpu shares.. this allows many more VM's to be running and and all shareing from the pool of resources...

    Which then raises a question why then have resource pools and reservations when a VM should be allocated a vCPU for a Real core.. ie you can not set a reservation/limit above the number of vCPU, and if you set these less then the vCPU this implies you are using more vCPU then real cores.



  • 9.  RE: vmware hyperthreading total resources

    Posted Dec 02, 2009 03:47 PM

    Which then raises a question why then have resource pools and reservations when a VM should be allocated a vCPU for a Real core.. ie you can not set a reservation/limit above the number of vCPU, and if you set these less then the vCPU this implies you are using more vCPU then real cores.

    Having more vCPU than physical cores is fine if it is via many VMs; it is just that taking out half the box with a single VM will cause scheduling delays (and hence performance problems).

    One thing that might be of interest is that WIndows Datacentre supports hot-adding of CPUs but not (yet) hot removal. Presumably in the future the guest will be able to scale depending on box capacity and workload as you describe using this technology. However there are other issues to think about, particularly NUMA archicture.

    Please award points to any useful answer.



  • 10.  RE: vmware hyperthreading total resources

    Posted Dec 02, 2009 08:07 PM

    Sorry for the ingorance, But I have been using Solaris for the last few years and with their virtualisation each VM (or container) can see and use all the host CPU, this can obviously be resource controlled with limits and cpu shares.. this allows many more VM's to be running and and all shareing from the pool of resources...

    Solaris Containers is is more like multiple applications on a single OS than a virtualization platform. Containers run in user space and share a common kernel if I understand correctly.



  • 11.  RE: vmware hyperthreading total resources

    Posted Dec 02, 2009 03:24 PM

    If you want to use all power of your CPU if other VMs are idle, you can use also CPU reservation to provide maximum perfomance.

    StarWind Software R&D



  • 12.  RE: vmware hyperthreading total resources

    Posted Dec 02, 2009 02:40 PM

    Welcome to the Forums - ESX/ESXi treats a hyper thread as a logical CPU - a spot for the VMKERNEL to schedule another vCPU - I am glad to hear you are not having perfromance problems with the 4 vCPUs - I would keep an eue on the ready time for your VMs because that will be the sure indication if you are running into problems scheduleing all 4 vcpus -

    If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful