Nested virtualization crash on AMD Zen4 CPUs due to VMLoad function.

[We Wen]

Hi all,

I have assembled a PC in 2024 and I noticed that when I used Vmwareworkstation, it will randomly crash the whole PC (power loss / reboot) after a random time interval, could be a few hours under heavy load, or a few days if i just have one VM open and nothing else.

Recently I have found out that this is due to the nested virtualization, and I suspect highly that it relates to the existing bug that has been discovered in Proxmox community regarding the vmload/vmsave functionality of Zen-4 AMD CPUs (see: Nested Virtualization Crashing Ryzen 7000 Series and Making sure you're not a bot! bugzilla and reddit posts.)

I have been trying to disable the vmload and vmsave functionality using the vmx file with cpu-id blocking with no avail, these function are still enabled despite the cpuid zero-ing. 

Process to reproduce:

• Have a zen4-cpu platform (or ryzen-7000?)
• Create a vm with the option Virtualize AMD-V/RVI
• Run nested virtualization in vm (in my case, bluestacks)
• Optionally run multiple instance of vm to accelerate the triggering
• Wait between a few hours - a few days, PC should crash without BSoD, just a sudden power loss.

Some additional info:

vmware version: 16.1.2 build-17966106

host: win 11 25H2, but it occurs when it wasnt upgraded, with win10 already

vm-OS: win 10 22H2

attached vmx and vmware.log (without nested virt option enabled, you can see the vmload function in the vmware.log file)

Let me know if I can provide any additional file/information to help you fix this.

Kr

Kernel		remove preview
Making sure you're not a bot! Why am I seeing this? You are seeing this because the administrator of this website has set up Anubis to protect the server against the scourge of AI companies aggressively scraping websites. This can and does cause downtime for the websites, which makes their resources inaccessible for everyone. Anubis is a compromise. View this on Kernel >

 

reddit		remove preview
Nested Virtualization Crashing Ryzen 7000 Series View this on reddit >

)

-------------------------------------------

Attachment(s)

BlueStackVM.vmx   4 KB 1 version

vmware.log   284 KB 1 version

[richard612]

I have a Ryzen 5 7600 and the same problem mentioned in the threads above and also in the kernel bug.  Sudden reboot with no crashdump. no kernel panic, and nothing logged when using nested virtualization and a Windows 11 guest.

I can't figure out if this is technically "fixed" or not.  I'm running kernel 7.0 on the host -- in theory any kernel with this commit (therefore >=6.12) should be fixed.

Indeed cat /proc/cpuinfo on the host does not indicate VMLOAD/VMSAVE.

vmware.log suggests it's being virtualized: INFO vmware-vmx 77906 [ws@4413 threadName="vcpu-0"]    Virtualized VMSAVE/VMLOAD 

I do not see VMSAVE in any of the Capability Found: cpuid. lines.  Also checked cpuinfo in a Linux guest and confirmed no VMSAVE/VMLOAD flag.

How does this appear in your vmware.log?  As a cpuid capability or virtualized as mine does?

[We Wen]

Mine appears as virtualized VMsave/VMload on the log as well (attached file).

I tried to mask the cpuid corresponding to these features in vmx file for the vm but it still results in crashes. Hopefully someone from broadcom can look into this issue.

[Michael Ambrose]

This issue affects all AMD Zen4 Raphael CPUs, both Ryzen and Epyc. It's been an issue for years with virtualization, not just VMWare, and I am guessing will never a solution.

[Prajakta Malla]

Engineering team is looking into this issue and will be addressed soon.