WS 17.6.3 Linux host, Kernel 6.13.4 - Still fails with Kernel BUG on HP Envy 17 (2023) Laptop..

[rgadsdon]

Attempted to upgrade to 17.6.3 on an HP Envy 17 2023 Laptop, and this still fails with a kernel BUG error, but is OK on other Linux systems:
[   92.603890] vmmon: loading out-of-tree module taints kernel.
[   92.607109] /dev/vmmon[2877]: Module vmmon: registered as misc device
[   92.607112] /dev/vmmon[2877]: Using tsc_khz as TSC frequency: 2611200
[   92.607113] /dev/vmmon[2877]: Module vmmon: initialized
[   92.626464] Missing ENDBR: init_module+0x0/0xd0 [vmnet]
[   92.627804] ------------[ cut here ]------------
[   92.628863] Kernel BUG at exc_control_protection+0x29d/0x2a0 [verbose debug info unavailable]
[   92.629887] Oops: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
[   92.630922] CPU: 0 UID: 0 PID: 2901 Comm: modprobe Tainted: G           O       6.13.4 #1
[   92.631974] Tainted: [O]=OOT_MODULE
[   92.633018] Hardware name: HP HP Envy Laptop 17-cw0xxx/8BDE, BIOS F.08 08/07/2024
[   92.634068] RIP: 0010:exc_control_protection+0x29d/0x2a0
[   92.635124] Code: d8 b9 09 00 00 00 48 8b 93 80 00 00 00 be 80 00 00 00 48 c7 c7 8b 01 55 b6 e8 7f 6c 33 ff 80 a3 8a 00 00 00 fb e9 31 fe ff ff <0f> 0b 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 66 0f 1f
[   92.636274] RSP: 0018:ffffa3c205a379d0 EFLAGS: 00010002
[   92.637405] RAX: 000000000000002b RBX: ffffa3c205a37a08 RCX: 0000000000000000
[   92.638555] RDX: 0000000000000000 RSI: ffff8e658f81c880 RDI: ffff8e658f81c880
[   92.639709] RBP: 0000000000000000 R08: 0000000000000000 R09: ffffa3c205a37868
[   92.640877] R10: ffffffffb6921e28 R11: 0000000000000003 R12: 0000000000000003
[   92.642045] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[   92.643228] FS:  00007f67566e5740(0000) GS:ffff8e658f800000(0000) knlGS:0000000000000000
[   92.644423] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   92.645614] CR2: 00007ffd873e9ef0 CR3: 000000010407e005 CR4: 0000000000f70ef0
[   92.646829] PKRU: 55555554
[   92.648019] Call Trace:
[   92.649204]  <TASK>
[   92.650372]  ? __die_body.cold+0x19/0x26
[   92.651552]  ? die+0x29/0x50
[   92.652732]  ? do_trap+0xc5/0x110
[   92.653905]  ? do_error_trap+0x60/0x80
[   92.655073]  ? exc_control_protection+0x29d/0x2a0
[   92.656239]  ? exc_invalid_op+0x4d/0x70
[   92.657401]  ? exc_control_protection+0x29d/0x2a0
[   92.658558]  ? asm_exc_invalid_op+0x1a/0x20
[   92.659722]  ? exc_control_protection+0x29d/0x2a0
[   92.660870]  asm_exc_control_protection+0x26/0x40
[   92.662016] RIP: 0010:init_module+0x0/0xd0 [vmnet]
[   92.663148] Code: 90 b8 f7 ff ff ff c3 cc cc cc cc b8 ff ff ff ff c3 cc cc cc cc 0f 1f 44 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 <66> 0f 1f 00 53 e8 a6 4e 00 00 85 c0 0f 85 65 7d 00 00 48 c7 c2 e0
[   92.664357] RSP: 0018:ffffa3c205a37ab8 EFLAGS: 00010246
[   92.665550] RAX: 0000000000000000 RBX: ffffffffc1312120 RCX: 0000000000000000
[   92.666740] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffa3c205a37aa0
[   92.667929] RBP: 0000000000000000 R08: 0000000000000000 R09: ffffa3c205a37806
[   92.669106] R10: 00000000000007e0 R11: 00000000801e001e R12: 000055f672463715
[   92.670329] R13: ffffa3c205a37ac0 R14: ffff8e5e10da4480 R15: ffff8e5e5c0f3ce0
[   92.671508]  ? __pfx_init_module+0x10/0x10 [vmnet]
[   92.672688]  do_one_initcall+0x5b/0x290
[   92.673873]  do_init_module+0x5b/0x1f0
[   92.675062]  init_module_from_file+0x81/0xc0
[   92.676250]  idempotent_init_module+0x110/0x310
[   92.677447]  __x64_sys_finit_module+0x5e/0xa0
[   92.678646]  do_syscall_64+0x7d/0x160
[   92.679853]  ? syscall_exit_to_user_mode+0x10/0x1c0
[   92.681073]  ? do_syscall_64+0x89/0x160
[   92.682293]  ? syscall_exit_to_user_mode+0x10/0x1c0
[   92.683512]  ? do_syscall_64+0x89/0x160
[   92.684719]  ? do_sys_openat2+0x97/0xe0
[   92.685912]  ? syscall_exit_to_user_mode+0x10/0x1c0
[   92.687105]  ? do_syscall_64+0x89/0x160
[   92.688283]  ? __count_memcg_events+0x4e/0xe0
[   92.689471]  ? handle_mm_fault+0x1be/0x290
[   92.690660]  ? do_user_addr_fault+0x31b/0x570
[   92.691856]  ? exc_page_fault+0x5f/0x80
[   92.693042]  entry_SYSCALL_64_after_hwframe+0x71/0x79
[   92.694228] RIP: 0033:0x7f67560fff1d
[   92.695406] Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d c3 5e 0f 00 f7 d8 64 89 01 48
[   92.696667] RSP: 002b:00007ffd873ec088 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[   92.697917] RAX: ffffffffffffffda RBX: 000055f6b1b39f30 RCX: 00007f67560fff1d
[   92.699178] RDX: 0000000000000000 RSI: 000055f672463715 RDI: 0000000000000003
[   92.700432] RBP: 00007ffd873ec140 R08: 00007f67561f6b20 R09: 0000000000000000
[   92.701678] R10: 000055f6b1b3a080 R11: 0000000000000246 R12: 000055f672463715
[   92.702924] R13: 0000000000040000 R14: 000055f6b1b39fd0 R15: 000055f6b1b3ab80
[   92.704161]  </TASK>
[   92.705377] Modules linked in: vmnet(O+) vmmon(O) xt_comment nft_compat nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat ip_set nf_tables qrtr vmw_vsock_vmci_transport vsock vmw_vmci bnep sunrpc cdc_mbim cdc_wdm cdc_ncm cdc_ether usbnet binfmt_misc vfat fat snd_ctl_led snd_soc_skl_hda_dsp snd_soc_intel_sof_board_helpers snd_soc_intel_hda_dsp_common snd_sof_probes iwlmvm snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic snd_hda_scodec_component snd_soc_dmic snd_sof_pci_intel_tgl snd_sof_pci_intel_cnl snd_sof_intel_hda_generic soundwire_intel soundwire_cadence snd_sof_intel_hda_common snd_soc_hdac_hda snd_hda_scodec_cs35l41_spi snd_sof_intel_hda_mlink regmap_spi snd_sof_intel_hda snd_sof_pci snd_sof_xtensa_dsp snd_sof snd_sof_utils snd_soc_acpi_intel_match uvcvideo snd_soc_acpi_intel_sdca_quirks soundwire_generic_allocation videobuf2_vmalloc snd_soc_acpi uvc btusb soundwire_bus
[   92.705403]  videobuf2_memops snd_soc_sdca videobuf2_v4l2 btmtk btrtl snd_soc_avs mac80211 btbcm videodev snd_soc_hda_codec btintel libarc4 snd_hda_ext_core videobuf2_common intel_uncore_frequency snd_soc_core r8152 bluetooth intel_uncore_frequency_common mc mii snd_compress ac97_bus snd_pcm_dmaengine snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi snd_hda_codec x86_pkg_temp_thermal snd_hda_core intel_powerclamp spi_nor snd_hwdep coretemp snd_seq iTCO_wdt mei_hdcp mei_pxp processor_thermal_device_pci snd_hda_scodec_cs35l41_i2c rapl mtd ee1004 snd_seq_device intel_pmc_bxt processor_thermal_device snd_hda_scodec_cs35l41 iwlwifi intel_cstate iTCO_vendor_support snd_hda_cs_dsp_ctls intel_rapl_msr processor_thermal_wt_hint snd_pcm processor_thermal_rfim snd_soc_cs_amp_lib hp_wmi processor_thermal_rapl cs_dsp mei_me wmi_bmof intel_uncore platform_profile spi_intel_pci cfg80211 pcspkr snd_timer snd_soc_cs35l41_lib i2c_i801 intel_rapl_common thunderbolt spi_intel mei snd rfkill i2c_smbus idma64 processor_thermal_wt_req
[   92.712427]  soundcore processor_thermal_power_floor processor_thermal_mbox igen6_edac intel_pmc_core int3403_thermal serial_multi_instantiate int340x_thermal_zone intel_vsec intel_hid pmt_telemetry int3400_thermal pmt_class acpi_thermal_rel sparse_keymap acpi_pad joydev soc_button_array loop dm_multipath dm_mod nfnetlink zram intel_ishtp_hid hid_sensor_hub crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic ghash_clmulni_intel rtsx_pci_sdmmc sha512_ssse3 mmc_core sha256_ssse3 ucsi_acpi intel_ish_ipc hid_multitouch sha1_ssse3 typec_ucsi rtsx_pci intel_ishtp typec i2c_hid_acpi i2c_hid pinctrl_tigerlake serio_raw nf_conntrack_pptp nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 fuse i2c_dev
[   92.725647] ---[ end trace 0000000000000000 ]---
[   92.727522] RIP: 0010:exc_control_protection+0x29d/0x2a0
[   92.729387] Code: d8 b9 09 00 00 00 48 8b 93 80 00 00 00 be 80 00 00 00 48 c7 c7 8b 01 55 b6 e8 7f 6c 33 ff 80 a3 8a 00 00 00 fb e9 31 fe ff ff <0f> 0b 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 66 0f 1f
[   92.731336] RSP: 0018:ffffa3c205a379d0 EFLAGS: 00010002
[   92.733265] RAX: 000000000000002b RBX: ffffa3c205a37a08 RCX: 0000000000000000
[   92.735200] RDX: 0000000000000000 RSI: ffff8e658f81c880 RDI: ffff8e658f81c880
[   92.737145] RBP: 0000000000000000 R08: 0000000000000000 R09: ffffa3c205a37868
[   92.739099] R10: ffffffffb6921e28 R11: 0000000000000003 R12: 0000000000000003
[   92.741054] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[   92.743008] FS:  00007f67566e5740(0000) GS:ffff8e658f800000(0000) knlGS:0000000000000000
[   92.744975] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   92.746950] CR2: 00007ffd873e9ef0 CR3: 000000010407e005 CR4: 0000000000f70ef0
[   92.748937] PKRU: 55555554
[   92.750911] note: modprobe[2901] exited with irqs disabled

The last version that compiles/loads/runs OK on this system is 17.5.2, and fortunately this older version (with patches..) is still OK with the latest 6.14-rc kernels..
As a point of reference, 17.6.3 installs OK on other Linux systems with the same Kernel - 6.13.4 and Distro (Fedora 41)..

[Dhairya Tomar]

rgadsdon What is the OS of the HOST where you are seeing this issue?

Please execute these commands "cat /etc/os-release", "uname -a" and share the respective output.

Also share the vmware-installer log & System log from location "/var/log" AND latest failure "vmware-vmispy***" & "vmware-***" logs from this location "/tmp/vmware-root"

[rgadsdon]

Uploaded logs, uname, etc. from crash...

[Dhairya Tomar]

@rgadsdon Unable to see the uploaded logs, it will be great if you can put them in a drive and share the direct download link.

[rgadsdon]

The uploaded files were in plain text format, but I have put them in a ZIP folder, and uploaded again..

[rgadsdon]

Additional info:

I looked at the ..cannot build modules... post, and saw that it recommended disabling Indirect branch tracking (ibt=off on the kernel command line) 

I tried this, and the installer now works correctly:

root@rgenvy17:/home/rgadsdon/kernel# ./VMware-Workstation-17.6.3-24583834.x86_64.bundle
Extracting VMware Installer...done.
Installing VMware Workstation 17.6.3
   Configuring...
[######################################################################] 100%
Installation was successful.
root@rgenvy17:/home/rgadsdon/kernel# vmware --version
VMware Workstation 17.6.3 build-24583834

But the guest (Win10) fails to start, with 'VMware Workstation irrecoverable error: (mks)
Unexpected signal 11... etc...

Log file uploaded..

[Dhairya Tomar]

@rgadsdon For this issue, please upload Support bundle of the affected VM.

[Dhairya Tomar]

rgadsdon Ticket raised internally, relevant team will look into the same.

[rgadsdon]

More data, from # dmesg at time of guest startup fail:

[ 1119.736944] vmware-tray[8538]: segfault at 11b276 ip 000000000011b276 sp 00007ffece1d4ab8 error 14 likely on CPU 2 (core 4, socket 0)
[ 1119.736954] Code: Unable to access opcode bytes at 0x11b24c.
[ 1138.576911] /dev/vmmon[9254]: PTSC: initialized at 2611200000 Hz using TSC, TSCs are synchronized.
[ 1138.635862] /dev/vmmon[9254]: Monitor IPI vector: 0
[ 1138.635866] /dev/vmmon[9254]: HV      IPI vector: 0

[Antonin Skala]

Hi, I observe same error on my ROG Strix G834JY with kernel 6.13.5-xanmod (Ubuntu 24.10) so it probably won't be specific to one piece of HW. Logs posted here: https://community.broadcom.com/vmware-cloud-foundation/discussion/1762-cannot-build-modules-on-kernel-613  (post check pending).