Graphics artifacts suddenly fill the VM screen after long use of a Kali Linux VM guest with 3D acceleration enabled

[xxx]

Description:

 

When "Accelerate 3D graphics" is enabled for a Kali Linux VM guest, its desktop environment (GNOME Wayland) becomes filled with graphical artifacts after the VM has been running continuously for around 20 hours on a Windows 11 host (physical machine). At this moment, the Kali Linux VM guest continues running, and the graphical interface can be restored by rebooting the VM.

 

This issue does not usually occur during a short-term running Kali Linux VM guest, like 4 hours.

 

This issue has persisted for at least six months and can be consistently reproduced on two different Windows 11 PCs with different hardware configurations.

 

Below are more details.

Windows PC host 1:

 

```

OS: Windows 11 Pro for Workstation, 24H2, build 26100.3775

    Features:

        Virtual Machine Platform [Disabled]

        Windows Hypervisor Platform [Disabled]

        Windows Sandbox [Disabled]

        Windows Subsystem for Linux [Disabled]

    Vitalization-based security [Disabled]

CPU: AMD Ryzen 9 9950X

GPU: AMD Radeon Graphics (integrated in the CPU)

GPU driver: AMD Software Adrenalin Edition 25.6.1

VMware Workstation Pro 17.6.3 build-24583834

```

 

Windows PC host 2:

 

```

OS: Exactly the same as PC 1

CPU: AMD Ryzen Threadripper Pro 5975WX

GPU: Nvidia GeForce RTX 4060 Ti 16GB

GPU driver: Nvidia Studio Driver 576.80

VMware Workstation Pro: Exactly the same as PC 1

```

 

Kali Linux VM guest：

 

```

OS: Kali Linux 2025.2, kernel 6.12.33+kali-amd64

mesa: 25.0.5-2

Desktop environment: GNOME 48.1, Wayland

vmx file:

    mks.enable3d = "TRUE"

    mks.enableDX12Presentation = "FALSE"

    mks.enableVulkanRenderer = "FALSE"

    mainMem.useNamedFile = "FALSE"

```

[xxx]

Today I added the following two new lines to the vmx file of my Kali Linux VM guest:

 

``` 

svga.noScreenBuffering = "TRUE" 

mks.noScreenBuffering = "TRUE" 

```

 

After running this VM continuously for 13 hours, the VM simply crashed with the following error:

This is something I have never encountered before.

[xxx]

This is the mksSandbox log file associated with the ISBRenderComm error.

Attachment(s)

mksSandbox-0.log   112 KB 1 version

[Shibdas Bandyopadhyay]

For this system, instead of the running on DX12 renderer using the config options 

  mks.enableDX12Presentation = "FALSE"

    mks.enableVulkanRenderer = "FALSE"

Can you instead use these config options and run on vulkan renderer and check if you still hit the crash. If you see rendering corruption (right at the beginning) add mks.enableDX12Presentation = "FALSE" too.

 mks.vk.forceDevice=NVIDIA

 mks.vk.allowMultiGPU=TRUE

[xxx]

Thanks for the advice. Now I am running the Kali Linux VM guest with the following vmx configuration:

```

# mks.enableDX12Presentation = "FALSE"

mks.enableVulkanRenderer = "TRUE"

mks.vk.forceDevice = "NVIDIA"
mks.vk.allowMultiGPU = "TRUE"

```

Let's wait for the VM to run for about 12 hours and see how it goes.

[xxx]

Well, after the Kali Linux VM guest had been running continuously for about 9 hours, once again, the full screen was suddenly covered in graphics artifacts.

This time, the VM was running in "Windows PC host 2" as previously described. And the vmx file is:

```

# mks.enableDX12Presentation = "FALSE"

mks.enableVulkanRenderer = "TRUE"

mks.vk.forceDevice = "NVIDIA"
mks.vk.allowMultiGPU = "TRUE"

```

[xxx]

I tested a new vmx configuration today, and the same problem still occurs after the Kali Linux VM guest has been running continuously for about 14 hours (using "Windows PC host 2" as previously described):

```

mks.enableDX12Presentation = "FALSE"

mks.enableVulkanRenderer = "TRUE"

# mks.vk.forceDevice = "NVIDIA"

mks.vk.allowMultiGPU = "TRUE"

```

[Shibdas Bandyopadhyay]

Can you please attach vmware.log and mksSandbox.log from the VM? Thanks!

[xxx]

Attached are all the log files from the folder where the Kali Linux VM is stored. Some sensitive information has been excluded for security reasons.

Attachment(s)

kali-vm-log.zip   180 KB 1 version

[Shibdas Bandyopadhyay]

I see some errors in the sandbox.log but it's hard to know from where it's coming. Can you please run the VM with full debugging info, reproduce the issue and attach the logs? See https://techdocs.broadcom.com/us/en/vmware-cis/desktop-hypervisors/workstation-pro/17-0/using-vmware-workstation-pro/configuring-virtual-machine-option-settings/configuring-advanced-options-for-a-virtual-machine/gathering-debugging-information.html for how to set this up.

[Shibdas Bandyopadhyay]

Also please attach dmesg.log and /var/log/kern.log from the Kali linux guest. Thanks!

[xxx]

The same issue was reproduced today. And the corresponding attachment below contains all the logs you requested:

* vmware.log and mksSandbox.log are generated under full debugging information mode.

* kern.log is copied directly from the Kali Linux VM.

* dmesg.log is the output of the dmesg command from the Kali Linux VM.

Again, I removed some sensitive information for security reasons.

And I noticed the following error in the log:

* vmware.log contains huge SVGA Driver errors.

* kern.log contains vmwgfx errors

 

I hope these logs can assist in identifying and resolving the issue efficiently.

Attachment(s)

full-debug-log.zip   6.89 MB 1 version

[Shibdas Bandyopadhyay]

> 

* vmware.log contains huge SVGA Driver errors.

* kern.log contains vmwgfx errors

That's what I expected. Please try with the kernel I mentioned with vmwgfx fix is possible

[Shibdas Bandyopadhyay]

@Ian Forbes mentioned a memory leak with vmwgfx is fixed in kernel 6.12.34 so please try to upgrade to that kernel if possible.

[xxx]

Just updated the kernel to 6.12.37. Will let you know if anything comes up!

[xxx]

Unfortunately, the issue still persists with kernel 6.12.37 - I'm seeing the same behavior in the Kali Linux VM after about 10 hours of continuous use.

Would you mind taking a look at the full debug log I uploaded earlier to see if there's anything else I could try or investigate?

[xxx]

Using VMware Workstation Pro 17.6.4 and Kali Linux VM guest kernel 6.12.37, the same issue still occurs. The rest of the environment remains unchanged.

[Shibdas Bandyopadhyay]

We have been reproduce this in house and team is working on finding the bug. Does this happen with other distros like Ubuntu or other display managers?

[xxx]

Glad to hear that you've been able to reproduce the issue. As for other distributions and environments, I haven't used them extensively or over a long period, so I'm not sure if the issue has a broader impact.

[Meet patel]

can you tell me how did you enable or install the Wayland in Kali with the VMware Workstation Pro i have AMD Ryzen 7 7840HS w/ Radeon 780M Graphics, 3801 Mhz, 8 Core(s), and the i have RTX 4060 can you teach how can i enable it in my machine currently i am using the Gnome X11 and it sucks i also have the similar problem as you i guess you can check this is my thread:-https://community.broadcom.com/communities/community-home/digestviewer/viewthread?GroupId=7171&MessageKey=afd7b8b3-1564-466d-a45f-466379788268&CommunityKey=fb707ac3-9412-4fad-b7af-018f5da56d9f 

[xxx]

The official Kali Linux documentation mentions how to turn on Wayland instead of using the default X11.