Hi! I'm running latest release VMware ESXi 8.0.3 build-24449057 on RPI4, 8 GB mem. Running 3 Ubuntu 24 VMs on this ESXi.
My issue is actually related to open-vm-tools that crash/coredumps every time I take a snapshot of any of the running VMs (I'm running the vghetto backup script for daily backups). Error that vmtoolsd throws is below. What I'm trying to figure out is whether this is related to ESXi version (this would not happen with esxi arm fling 7.x) or related to vmware tools. The only clue I could find was vmtoolsd exit status 4/ILL, which points to some illegal instruction being called. I even rebuilt the latest version of open-vm-tools from source, same error. Once crash occurs vmtoolsd cannot be successfully restart from systemd, VM needs restart.
2025-03-05T15:57:51.800836+00:00 u24-vm-01 systemd[1]: Created slice system-systemd\x2dcoredump.slice - Slice /system/systemd-coredump.
2025-03-05T15:57:51.819003+00:00 u24-vm-01 systemd[1]: Started systemd-coredump@0-1587-0.service - Process Core Dump (PID 1587/UID 0).
2025-03-05T15:57:53.461145+00:00 u24-vm-01 systemd[1]: Started systemd-coredump@1-1589-0.service - Process Core Dump (PID 1589/UID 0).
2025-03-05T15:57:53.661030+00:00 u24-vm-01 systemd-coredump[1588]: Process 696 (vmtoolsd) of user 0 dumped core.#012#012Module libudev.so.1 from deb systemd-255.4-1ubuntu8.5.arm64#012Stack trace of thread 696:#012#0 0x0000e55e132940e4 Backdoor_InOut (libvmtools.so.0 + 0x240e4)#012#1 0x0000e55e132b5858 Message_Send (libvmtools.so.0 + 0x45858)#012#2 0x0000e55e132c32d0 n/a (libvmtools.so.0 + 0x532d0)#012#3 0x0000e55e132c39f0 n/a (libvmtools.so.0 + 0x539f0)#012#4 0x0000e55e1308f780 n/a (libglib-2.0.so.0 + 0x5f780)#012#5 0x0000e55e130f2998 n/a (libglib-2.0.so.0 + 0xc2998)#012#6 0x0000e55e13090240 g_main_loop_run (libglib-2.0.so.0 + 0x60240)#012#7 0x0000b5a6b688b668 n/a (vmtoolsd + 0xb668)#012#8 0x0000b5a6b6885240 main (vmtoolsd + 0x5240)#012#9 0x0000e55e12e984c4 n/a (libc.so.6 + 0x284c4)#012#10 0x0000e55e12e98598 __libc_start_main (libc.so.6 + 0x28598)#012#11 0x0000b5a6b6885470 _start (vmtoolsd + 0x5470)#012#012Stack trace of thread 781:#012#0 0x0000e55e12f596a4 syscall (libc.so.6 + 0xe96a4)#012#1 0x0000e55e130ecb64 g_cond_wait_until (libglib-2.0.so.0 + 0xbcb64)#012#2 0x0000b5a6b6887d3c n/a (vmtoolsd + 0x7d3c)#012#3 0x0000b5a6b6887140 n/a (vmtoolsd + 0x7140)#012#4 0x0000e55e130bf248 n/a (libglib-2.0.so.0 + 0x8f248)#012#5 0x0000e55e12ef595c n/a (libc.so.6 + 0x8595c)#012#6 0x0000e55e12f5ba4c n/a (libc.so.6 + 0xeba4c)#012#012Stack trace of thread 759:#012#0 0x0000e55e12f596a4 syscall (libc.so.6 + 0xe96a4)#012#1 0x0000e55e130ec15c g_cond_wait (libglib-2.0.so.0 + 0xbc15c)#012#2 0x0000e55e13057848 n/a (libglib-2.0.so.0 + 0x27848)#012#3 0x0000e55e130c2ffc n/a (libglib-2.0.so.0 + 0x92ffc)#012#4 0x0000e55e130bf248 n/a (libglib-2.0.so.0 + 0x8f248)#012#5 0x0000e55e12ef595c n/a (libc.so.6 + 0x8595c)#012#6 0x0000e55e12f5ba4c n/a (libc.so.6 + 0xeba4c)#012#012Stack trace of thread 782:#012#0 0x0000e55e12f596a4 syscall (libc.so.6 + 0xe96a4)#012#1 0x0000e55e130ec15c g_cond_wait (libglib-2.0.so.0 + 0xbc15c)#012#2 0x0000b5a6b68884fc n/a (vmtoolsd + 0x84fc)#012#3 0x0000b5a6b6887140 n/a (vmtoolsd + 0x7140)#012#4 0x0000e55e130bf248 n/a (libglib-2.0.so.0 + 0x8f248)#012#5 0x0000e55e12ef595c n/a (libc.so.6 + 0x8595c)#012#6 0x0000e55e12f5ba4c n/a (libc.so.6 + 0xeba4c)#012ELF object binary architecture: AARCH64
2025-03-05T15:57:53.799790+00:00 u24-vm-01 systemd[1]: open-vm-tools.service: Main process exited, code=dumped, status=4/ILL
2025-03-05T15:57:54.291526+00:00 u24-vm-01 systemd[1]: open-vm-tools.service: Failed with result 'core-dump'.
2025-03-05T15:57:54.292974+00:00 u24-vm-01 systemd[1]: open-vm-tools.service: Consumed 7.933s CPU time, 8.3M memory peak, 0B memory swap peak.
2025-03-05T15:57:54.294820+00:00 u24-vm-01 systemd[1]: systemd-coredump@0-1587-0.service: Deactivated successfully.
2025-03-05T15:57:54.295834+00:00 u24-vm-01 systemd[1]: systemd-coredump@0-1587-0.service: Consumed 1.812s CPU time.
2025-03-05T15:57:54.296432+00:00 u24-vm-01 systemd[1]: systemd-coredump@0-1587-0.service: Triggering OnSuccess= dependencies.
2025-03-05T15:57:54.296837+00:00 u24-vm-01 systemd[1]: Created slice system-apport\x2dcoredump\x2dhook.slice - Slice /system/apport-coredump-hook.
2025-03-05T15:57:54.298567+00:00 u24-vm-01 systemd[1]: Starting apport-coredump-hook@0-1587-0.service...
2025-03-05T15:57:55.325980+00:00 u24-vm-01 systemd-coredump[1590]: Process 1060 (vmtoolsd) of user 0 dumped core.#012#012Module libudev.so.1 from deb systemd-255.4-1ubuntu8.5.arm64#012Stack trace of thread 1060:#012#0 0x0000e852075a40e4 Backdoor_InOut (libvmtools.so.0 + 0x240e4)#012#1 0x0000e852075c55d0 Message_OpenAllocated (libvmtools.so.0 + 0x455d0)#012#2 0x0000e852075c56a0 Message_Open (libvmtools.so.0 + 0x456a0)#012#3 0x0000e852075d3fa8 n/a (libvmtools.so.0 + 0x53fa8)#012#4 0x0000e852075abf18 n/a (libvmtools.so.0 + 0x2bf18)#012#5 0x0000e852075e70ec n/a (libvmtools.so.0 + 0x670ec)#012#6 0x0000e852075b0068 n/a (libvmtools.so.0 + 0x30068)#012#7 0x0000e8520739f780 n/a (libglib-2.0.so.0 + 0x5f780)#012#8 0x0000e85207402998 n/a (libglib-2.0.so.0 + 0xc2998)#012#9 0x0000e852073a0240 g_main_loop_run (libglib-2.0.so.0 + 0x60240)#012#10 0x0000b328576f654c ToolsCoreRunLoop (vmtoolsd + 0x654c)#012#11 0x0000b328576f5230 main (vmtoolsd + 0x5230)#012#12 0x0000e852071a84c4 n/a (libc.so.6 + 0x284c4)#012#13 0x0000e852071a8598 __libc_start_main (libc.so.6 + 0x28598)#012#14 0x0000b328576f5430 _start (vmtoolsd + 0x5430)#012#012Stack trace of thread 1145:#012#0 0x0000e852072696a4 syscall (libc.so.6 + 0xe96a4)#012#1 0x0000e852073fc15c g_cond_wait (libglib-2.0.so.0 + 0xbc15c)#012#2 0x0000e85207367848 n/a (libglib-2.0.so.0 + 0x27848)#012#3 0x0000e852073d2ffc n/a (libglib-2.0.so.0 + 0x92ffc)#012#4 0x0000e852073cf248 n/a (libglib-2.0.so.0 + 0x8f248)#012#5 0x0000e8520720595c n/a (libc.so.6 + 0x8595c)#012#6 0x0000e8520726ba4c n/a (libc.so.6 + 0xeba4c)#012#012Stack trace of thread 1169:#012#0 0x0000e852072696a4 syscall (libc.so.6 + 0xe96a4)#012#1 0x0000e852073fcb64 g_cond_wait_until (libglib-2.0.so.0 + 0xbcb64)#012#2 0x0000b328576fa37c SleepToExit (vmtoolsd + 0xa37c)#012#3 0x0000b328576f8aec ToolsCorePoolRunThread (vmtoolsd + 0x8aec)#012#4 0x0000e852073cf248 n/a (libglib-2.0.so.0 + 0x8f248)#012#5 0x0000e8520720595c n/a (libc.so.6 + 0x8595c)#012#6 0x0000e8520726ba4c n/a (libc.so.6 + 0xeba4c)#012#012Stack trace of thread 1170:#012#0 0x0000e852072696a4 syscall (libc.so.6 + 0xe96a4)#012#1 0x0000e852073fc15c g_cond_wait (libglib-2.0.so.0 + 0xbc15c)#012#2 0x0000b328576fb488 GlobalConfigThreadStart (vmtoolsd + 0xb488)#012#3 0x0000b328576f8aec ToolsCorePoolRunThread (vmtoolsd + 0x8aec)#012#4 0x0000e852073cf248 n/a (libglib-2.0.so.0 + 0x8f248)#012#5 0x0000e8520720595c n/a (libc.so.6 + 0x8595c)#012#6 0x0000e8520726ba4c n/a (libc.so.6 + 0xeba4c)#012ELF object binary architecture: AARCH64
2025-03-05T15:57:55.366389+00:00 u24-vm-01 systemd[1]: rc-local.service: Main process exited, code=dumped, status=4/ILL
2025-03-05T15:57:56.031823+00:00 u24-vm-01 systemd[1]: rc-local.service: Failed with result 'core-dump'.
2025-03-05T15:57:56.032975+00:00 u24-vm-01 systemd[1]: rc-local.service: Consumed 3.672s CPU time.
2025-03-05T15:57:56.037232+00:00 u24-vm-01 systemd[1]: systemd-coredump@1-1589-0.service: Deactivated successfully.
2025-03-05T15:57:56.039818+00:00 u24-vm-01 systemd[1]: systemd-coredump@1-1589-0.service: Triggering OnSuccess= dependencies.
2025-03-05T15:57:56.040803+00:00 u24-vm-01 systemd[1]: Starting apport-coredump-hook@1-1589-0.service...