VMware EVC Mode vMotion Restart?

[Brian Cummings]

I enabled VMware EVC Mode (Sandy Bridge) on a four-host cluster, all with the same CPU architecture, Intel Xeon E5-2620 (Sandy Bridge).  When I add a new host with Intel Xeon E5-2690 (Sandy Bridge) CPUs to this cluster and a VM is vMotioned and then restarted on this new host, Windows prompts for a restart as a newer CPU driver is installed. Is this expected behavior?  To my current understanding, a restart is not required as EVC mode is enabled.

[admin]

EVC does not make the processors in the cluster appear identical to the guest; it just ensures that they all report the same capabilities.  If Windows picks the CPU driver based upon the Brand ID string (e.g "Intel Xeon E5-2690") rather than on the capabilities of the CPU, then it could prompt for a restart.

[Brian Cummings]

Thank you for the repsonse.  That's what may be going on here.  If so, is there a workaround or is this just something users are going to have to deal with when their VM lands on this particular host and is restarted?

[admin]

There is a potential workaround.  If you upload the vmware.log file for a VM that was booted on one of the original machines in the cluster (as an attachment), I will construct the configuration settings for you.

[ttengea]

Hi - I saw this posting and it references the exact issue I am having now (realizing this is a year old - wasn't sure if starting a new discussion was the way to go). I enabled EVC on my two clusters yesterday (v5.1 Express Patch 5) and still receive the restart prompt, once the VM is vMotioned, then restarted on a host with different CPUs. This appears to happen each time the vMotion is done in either direction. The good thing is that now I'm at least able to do the actual vMotion.

I saw that you requested logs and that there was a possible workaround. I have attached the logs, if you have time and are still willing to take a look.

Thanks in advance

[admin]

Try the following configuration options in /etc/vmware/config:

monitor_control.enable_fullcpuid = TRUE

cpuid.80000002.eax = '0110:0101:0111:0100:0110:1110:0100:1001'

cpuid.80000002.ebx = '0010:1001:0101:0010:0010:1000:0110:1100'

cpuid.80000002.ecx = '0110:1111:0110:0101:0101:1000:0010:0000'

cpuid.80000002.edx = '0010:1001:0101:0010:0010:1000:0110:1110'

cpuid.80000003.eax = '0101:0101:0101:0000:0100:0011:0010:0000'

cpuid.80000003.ebx = '0010:0000:0010:0000:0010:0000:0010:0000'

cpuid.80000003.ecx = '0010:0000:0010:0000:0010:0000:0010:0000'

cpuid.80000003.edx = '0101:1000:0010:0000:0010:0000:0010:0000'

cpuid.80000004.eax = '0011:0000:0011:0101:0011:0110:0011:0101'

cpuid.80000004.ebx = '0010:0000:0100:0000:0010:0000:0010:0000'

cpuid.80000004.ecx = '0011:0111:0011:0110:0010:1110:0011:0010'

cpuid.80000004.edx = '0000:0000:0111:1010:0100:1000:0100:0111'

You will have to power off and power on your VMs for these options to take effect.

[ttengea]

To be clear (as I’ve never modified the config file manually before), the new entries would be added to the parameters already listed in the file shown below):

.encoding = "UTF-8"

libdir = "/usr/lib/vmware"

authd.proxy.vim = "vmware-hostd:hostd-vmdb"

authd.proxy.nfc = "vmware-hostd:ha-nfc"

authd.proxy.nfcssl = "vmware-hostd:ha-nfcssl"

authd.proxy.vpxa-nfcssl = "vmware-vpxa:vpxa-nfcssl"

authd.proxy.vpxa-nfc = "vmware-vpxa:vpxa-nfc"

authd.fullpath = "/sbin/authd"

authd.soapServer = "TRUE"

vmauthd.server.alwaysProxy = "TRUE"

vhv.allow = "TRUE"

featMask.evc.cpuid.Intel = "Val:1"

featMask.evc.cpuid.FAMILY = "Val:6"

featMask.evc.cpuid.MODEL = "Val:0x1a"

featMask.evc.cpuid.STEPPING = "Val:4"

featMask.evc.cpuid.NUMLEVELS = "Val:0xb"

featMask.evc.cpuid.NUM_EXT_LEVELS = "Val:0x80000008"

featMask.evc.cpuid.CMPXCHG16B = "Val:1"

featMask.evc.cpuid.DS = "Val:1"

featMask.evc.cpuid.LAHF64 = "Val:1"

featMask.evc.cpuid.LM = "Val:1"

featMask.evc.cpuid.MWAIT = "Val:1"

featMask.evc.cpuid.NX = "Val:1"

featMask.evc.cpuid.SS = "Val:1"

featMask.evc.cpuid.SSE3 = "Val:1"

featMask.evc.cpuid.SSSE3 = "Val:1"

featMask.evc.cpuid.SSE41 = "Val:1"

featMask.evc.cpuid.POPCNT = "Val:1"

featMask.evc.cpuid.RDTSCP = "Val:1"

featMask.evc.cpuid.SSE42 = "Val:1"

featMask.evc.cpuid.VMX = "Val:1"

featMask.evc.hv.capable = "Val:1"

featureCompat.evc.completeMasks = "TRUE"

Also, if I understand correctly, every VM in the cluster will need to be powered off and on? Reboots, say during my next patching cycle, won’t cut it, correct?

Thanks

[admin]

ttengea wrote:

To be clear (as I’ve never modified the config file manually before), the new entries would be added to the parameters already listed in the file shown below):

.encoding = "UTF-8"

libdir = "/usr/lib/vmware"

authd.proxy.vim = "vmware-hostd:hostd-vmdb"

authd.proxy.nfc = "vmware-hostd:ha-nfc"

authd.proxy.nfcssl = "vmware-hostd:ha-nfcssl"

authd.proxy.vpxa-nfcssl = "vmware-vpxa:vpxa-nfcssl"

authd.proxy.vpxa-nfc = "vmware-vpxa:vpxa-nfc"

authd.fullpath = "/sbin/authd"

authd.soapServer = "TRUE"

vmauthd.server.alwaysProxy = "TRUE"

vhv.allow = "TRUE"

featMask.evc.cpuid.Intel = "Val:1"

featMask.evc.cpuid.FAMILY = "Val:6"

featMask.evc.cpuid.MODEL = "Val:0x1a"

featMask.evc.cpuid.STEPPING = "Val:4"

featMask.evc.cpuid.NUMLEVELS = "Val:0xb"

featMask.evc.cpuid.NUM_EXT_LEVELS = "Val:0x80000008"

featMask.evc.cpuid.CMPXCHG16B = "Val:1"

featMask.evc.cpuid.DS = "Val:1"

featMask.evc.cpuid.LAHF64 = "Val:1"

featMask.evc.cpuid.LM = "Val:1"

featMask.evc.cpuid.MWAIT = "Val:1"

featMask.evc.cpuid.NX = "Val:1"

featMask.evc.cpuid.SS = "Val:1"

featMask.evc.cpuid.SSE3 = "Val:1"

featMask.evc.cpuid.SSSE3 = "Val:1"

featMask.evc.cpuid.SSE41 = "Val:1"

featMask.evc.cpuid.POPCNT = "Val:1"

featMask.evc.cpuid.RDTSCP = "Val:1"

featMask.evc.cpuid.SSE42 = "Val:1"

featMask.evc.cpuid.VMX = "Val:1"

featMask.evc.hv.capable = "Val:1"

featureCompat.evc.completeMasks = "TRUE"

Correct.

Also, if I understand correctly, every VM in the cluster will need to be powered off and on? Reboots, say during my next patching cycle, won’t cut it, correct?

Masked CPUID information is normally set in stone at power-on.  However, in this case, we are setting up new masks for fields that are typically passed-through from the host.  In this situation, it might suffice to simply vMotion the VMs (including self-vMotion).  Windows VMs that are currently running on Ivy Bridge systems will probably ask for a reboot -- if the vMotion picks up the new settings.

[Linjo]

Not sure if it helps but this will probably only happen once per vm and per cpu-type.

[SATHISHVIJAY]

HI JM, Need some clarrification:

"After EVC is enabled, all hosts in the cluster are configured to present the CPU features of user-selected Processor type to all VMs running inthe cluster. This ensures CPU Compatibility for vMotion.

* IDENTICAL CPU features are exposed to VMs regardless of which host they resides on, so that VMs can migrate between any hosts in cluster"

[admin]

SATHISHVIJAY wrote:

HI JM, Need some clarrification:

"After EVC is enabled, all hosts in the cluster are configured to present the CPU features of user-selected Processor type to all VMs running inthe cluster. This ensures CPU Compatibility for vMotion.

* IDENTICAL CPU features are exposed to VMs regardless of which host they resides on, so that VMs can migrate between any hosts in cluster"

Any feature that indicates a functional capability of the CPU (e.g. SSE, AVX, AES) is covered by EVC masking.  (There are some limitations regarding which features can be masked from guest application code when running with binary translation/direct execution.)  However, the brand ID string is not covered by EVC masking.

[SATHISHVIJAY]

Fine thanks..

[Brian Cummings]

I need to scrub the vmware.log file before it's posted.  Thanks everyone for their input.  Any other workarounds or solutions in the meantime?

[admin]

If it helps, I just need the lines containing the string "hostCPUID".

[Brian Cummings]

Please see below:

2014-05-19T05:52:19.068Z| vmx| I120: hostCPUID level 00000004, 3: 0x3c07c163 0x04c0003f 0x00002fff 0x00000006

[admin]

Sorry; I will need all of the lines that contain hostCPUID.