Problem with migrating VM from a cluster to another.

[CrisRO]

Hello guys,

I'm relatively new in this area.

I have this situation. In my Vcenter I have  2 clusters, X and Y,  cluster X one host and in the cluster Y two hosts.

Host X have VMware ESXi, 6.5.0, 4564106  hypervisor and Y VMware ESXi, 6.5.0, 7526125.

X and Y have different processors X- Xeon E5620 and Y=Xeon  E5-2667 v4

When I want to migrate a VM from host Y to X, I have some trouble.

First, some VM from Y don't see X cluster when I want to migrate, but some VM can see X, so where is the problem.

Another thing this is the error when I want to move a VM if I see the host:

The target host does not support the virtual machine's current hardware requirements.

To resolve CPU incompatibilities, use a cluster with Enhanced vMotion Compatibility (EVC) enabled. See KB article 1003212.

Supervisor Mode Access Prevention (SMAP) is unsupported.

Restricted Transactional Memory (RTM) is unsupported.

RDSEED is unsupported.

Hardware Lock Elision (HLE) is unsupported.

Multi-Precision Add-Carry Instruction Extensions (ADX) is unsupported.

3DNow! PREFETCH and PREFETCHW are unsupported.

Invalidate Process-Context Identifier (INVPCID) is unsupported.

XSAVEOPT is unsupported.

1 GB pages are not supported (PDPE1GB).

MOVBE is unsupported.

FMA3 is unsupported.

Bit Manipulation Instruction (BMI) Set 2 is unsupported.

Bit Manipulation Instruction (BMI) Set 1 is unsupported.

Advanced Vector Extensions 2 (AVX2) are unsupported.

Advanced Bit Manipulation (ABM) is unsupported.

CPUID faulting is not supported.

Supervisor Mode Execution Protection (SMEP) is unsupported.

RDRAND is unsupported.

Instructions to read and write FS and GS base registers at any privilege level are unsupported.

Half-precision conversion instructions (F16C) are unsupported.

Fast string operations (Enhanced REP MOVSB/STOSB) are unsupported.

XSAVE is unsupported.

XSAVE YMM State is unsupported.

XSAVE SSE State is unsupported.

PCID is unsupported.

Advanced Vector Extensions (AVX) are unsupported.

com.vmware.vim.vmfeature.cpuid.stibp

com.vmware.vim.vmfeature.cpuid.ibrs

com.vmware.vim.vmfeature.cpuid.ibpb

[sk84]

If you have different clusters of ESXi hosts with different processor families, you need EVC mode for live migration of virtual machines between these clusters.

See:

CPU Compatibility and EVC

VMware Knowledge Base

Each processor type has different features and VMs sometimes use these features. If you try to migrate these VMs to another host that does not have these CPU features, this is only possible if the VMs are turned off.

You can therefore enable EVC mode at the cluster level, which is the lowest common denominator between clusters. For example, you have Skylake CPUs and Broadwell CPUs in another cluster. If you now configure "Broadwell" as EVC mode on the Skylake CPU cluster, these ESXi hosts will be downgraded to Broadwell's CPU feature set and don't use the new Skylake features anymore.

And if you change the EVC mode in a cluster and go to a "lower" CPU version, the VMs will need to be rebooted to reflect this change.

[IRIX201110141]

The root cause is

com.vmware.vim.vmfeature.cpuid.stibp
com.vmware.vim.vmfeature.cpuid.ibrs
com.vmware.vim.vmfeature.cpuid.ibpb

Some of your Hosts are already mitigate against Meltdown/Spectre by newer BIOS or that your esx d 7526125 comes with the microcode patches.  If you have startet a VM on that host it never can migrate with vMotion to one of the others. If you start a VM on one of the others you can.  This behaviour is a little bit confusing :smileyhappy:

Iam pretty sure that some KBs exists and also how this effect EVC.

Outside from this you need the same EVC Mode for all Clusters if you would like to vMotion a VM between them.

Regards,

Joerg