VMware vSphere

 View Only
  • 1.  Yet another question about CPU Scheduling

    Posted Jan 28, 2013 03:02 PM

    Hi!

    CPU Scheduling is one of those endless subjects and althought I read quite a lot about it, I found different answers to this very same question. By the way, this is the last paper about this subject and a recommended reading...http://www.vmware.com/files/pdf/techpaper/VMware-vSphere-CPU-Sched-Perf.pdf.

    Is it possible to run 2 VMs at the same time in the same logical processor? Very basic question, I'll appreciate any answer and if it has links to the official documentation, even better!!!

    Regards,

    elgreco81



  • 2.  RE: Yet another question about CPU Scheduling

    Posted Jan 29, 2013 08:50 AM

    Anybody? I will refrase the question.

    Can I "split" the processor in Mhz in order to run simultaneously 10 VMs, each of them only require/use 100Mhz if the only resource I have is 1 CPU with 1000Mhz (no hyperthreading)? With simultaneously I mean that they execute at the same time...I guess that reaching a %usage of 100% while the %rdy is 0.

    Again, I know this is a very basic question but I'm really intriged about this...

    Thanks,

    elgreco81



  • 3.  RE: Yet another question about CPU Scheduling

    Posted Jan 29, 2013 05:39 PM

    Maybe is not a basic question after all! :smileyhappy: More than 70 views but still no answer!!!



  • 4.  RE: Yet another question about CPU Scheduling

    Posted Jan 29, 2013 06:34 PM

    No, and this is one of the main reasons a CPU scheduler is necessary. They have to be divvied out as timeslices. Assume in your example we're looking more closely at a 1 second sample with 10 VMs sharing a non-HT core, and as of the beginning of the sample period, every VM has enough work to keep the 1000 MHz core busy for 100 ms. In this case, every VM effectively requires 100 MHz, but you would still see CPU READY times.

    Each world is only given a time quantum of 50 ms by default (see page 6 on the doc you linked). In our sample period, for the first 50 ms, the first world scheduled would not change its own state to WAIT because it has 100 ms worth of work. After 50 ms, the scheduler would be invoked, and that first world would begin accumulating READY time, until the scheduler circles back around to it. While this is happening, since the other 9 VMs in this example also had work to do, they would have also accumulated 50 ms of READY time each.

    This isn't accounting for I/O contexts or interrupts either. This isn't a typical scenario, but more of an academic exercise to answer the question I think you're asking. :smileyhappy:

    Hope that makes sense!

    EDIT: The answer is spelled out a little more clearly on p11. They're speaking more to skew mitigation for SMP inside the guest here, but the "one at a time" statement is true regardless of whether or not they're sibling vCPUs. If multiple busy vCPUs are ever sharing a pCPU, at least one of them will accumulate READY time.

    To ensure that sibling vCPUs of a multiprocessor virtual machine are co-scheduled as much as possible, the CPU
    scheduler avoids placing multiple sibling vCPUs on the same pCPU. If there are two busy sibling vCPUs of a virtual
    machine scheduled on the same pCPU, only one vCPU can run at a time which may lead to co-stop and cause
    extra migration. Better co-scheduling is accomplished by making the goodness greater if the destination pCPU
    does not already have a sibling vCPU placed.



  • 5.  RE: Yet another question about CPU Scheduling

    Posted Jan 29, 2013 07:14 PM

    Hi Andy!

    Thank you very much for your answer! It makes complete sense but some documentation talk about things like "if xxxMhz are available" which makes me think that maybe the kernel IS able to accomodate more than one world at the same time and to "split" the processor's Mhz.

    http://frankdenneman.nl/2010/06/08/reservations-and-cpu-scheduling/

    http://www.yellow-bricks.com/2010/05/18/limiting-your-vcpu/

    http://pubs.vmware.com/vsphere-51/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-51-resource-management-guide.pdf

    Then things like the one you mention or this paragraph from the document i linked, makes me doubt...it could be ambiguous as it leaves the door open for vCPUs of different VMs running on the same pCPU.

    "To ensure that sibling vCPUs of a multiprocessor virtual machine are co-scheduled as much as possible, the CPU

    scheduler avoids placing multiple sibling vCPUs on the same pCPU. If there are two busy sibling vCPUs of a virtual
    machine scheduled on the same pCPU, only one vCPU can run at a time which may lead to co-stop and cause
    extra migration. Better co-scheduling is accomplished by making the goodness greater if the destination pCPU
    does not already have a sibling vCPU placed."

    Or other things like this one found in the resource management guide (page 12)

    "For example, assume you have 2GHz available and specify a reservation of 1GHz for VM1 and 1GHz for VM2.

    Now each virtual machine is guaranteed to get 1GHz if it needs it. However, if VM1 is using only 500MHz,

    VM2 can use 1.5GHz."

    I feel really dumb knowing that the information is out there but I just can't fully understand it :smileysad: I read those papers over and over but I just can't get to any conclusion.

    Thanks again for your help!!!

    elgreco81



  • 6.  RE: Yet another question about CPU Scheduling
    Best Answer

    Posted Jan 29, 2013 07:59 PM

    Ah, I think I see the confusion. There's no such thing as using a fraction of a core within a single cycle. You CAN use all of a core for a fraction of the total cycles over a period of time, but a physical core is either busy or idle. There is no in-between, especially since we're ignoring hyperthreading in this example.

    Using all of a core for a fraction of the total time in a given sample period is what gives the illusion of using "part" of a core. Actually splitting a single core in that manner would require states and architecture that simply don't exist in the x86 world.

    Notice that in Frank Denneman's article he specifically calls out the scheduler "shopping out" free cycles. In your example, there will only ever be 1 free cycle at any given time. There are 1000 cycles every second in your original example, and many of these cycles may be free, but there will never more than one at a time.

    To put it another way, you can't use all 1000 MHz in the first millisecond.

    The paragraph you quoted from the resource management guide assumes a 2 GHz pCPU. Bear in mind any measurement of requency REQUIRES a time interval. There is no literal "point in time" splitting implied here. What they're saying is that if over some period of time, let's say one second, VM1 only consumes 500 cycles, and does not ask for more, then VM2 is welcome to help itself to any of the unused 500 cycles originally entitled to VM1, as well as the 1000 cycles VM2 was already entitled to.

    Let me know if there's still any ambiguity, or anything I didn't really clarify there.



  • 7.  RE: Yet another question about CPU Scheduling

    Posted Jan 29, 2013 07:18 PM

    LOL!!! I just saw your "EDIT" So funny we both mark "bold" the same phrase! :smileyhappy: Do you have any document that supports your statment of "regardless of whether or not they're sibling vCPUs"?

    Thanks again Andy!

    elgreco81

    PS: I'm very slow when writting in english :smileygrin:



  • 8.  RE: Yet another question about CPU Scheduling

    Posted Jan 29, 2013 09:38 PM

    Sebastián Greco wrote:

    LOL!!! I just saw your "EDIT" So funny we both mark "bold" the same phrase! :smileyhappy: Do you have any document that supports your statment of "regardless of whether or not they're sibling vCPUs"?

    Thanks again Andy!

    elgreco81

    PS: I'm very slow when writting in english :smileygrin:

    I just noticed this too. I'm apparently slow when writing in english too, and it's my native language. :smileyhappy:

    There are quite a few documents to support that, but most of them are huge and not easy to quote here. Probably the easiest way to explain this is to note the following line from the VMware CPU scheduler doc you linked in the original post:

    A vCPU and a world are interchangeably used to refer to a schedulable CPU context that corresponds to a process in conventional operating systems.

    Now I hate to send you to Wikipedia for a source on this, but it's the most concise explanation I could find.

    A process moves into the running state when it is chosen for execution. The process's instructions are executed by one of the CPUs (or cores) of the system. There is at most one running process per CPU or core.

    You might also take a look at this: http://www.cs.uic.edu/~jbell/CourseNotes/OperatingSystems/5_CPU_Scheduling.html

    It's more of an overview of CPU scheduling in general, but reading through that will probably clarify things a bit.



  • 9.  RE: Yet another question about CPU Scheduling

    Posted Jan 30, 2013 07:48 AM

    I finally completly get it!!! :smileyhappy: Thank you very much for such an "easy to understand" explanation and your good humor! :smileyhappy:

    I will read carefully the lasts links (yes, even wikipedia :smileywink:).

    Thanks again!

    elgreco81



  • 10.  RE: Yet another question about CPU Scheduling

    Posted Feb 01, 2013 08:14 PM

    You bet!