From a high level perspective it seems that ESXi has issues with the USB/SSD adapter (the RPI seems to be able to boot from it, so it's unlikely to be a hardware problem).
Can you boot ESXi, plug the disk, and then copy the /var/log/vmkernel.log file? I will likely contains errors related to USB.
Have you tried the same SSD with a different adapter?