Ah yes, I understand your point about stealing the server now. You are saying that if they steal the physical box on which the KMS is deployed, they should be able to start any encrypted VMs that are also hosted on that box.
That is not true. With VM encryption, vCenter is required to push the keys to the ESXi hosts in order to unlock and encrypted VMs. As long as your vCenter and KMS are not hosted on the same ESXi, you do not have a complete supply chain to get keys to the hosts. Without the keys, the VMs will remain encrypted. You will not be able to forcibly retrieve the keys from the KMS server either.
So, you could just simply enforce an Anti-affinity rule that keeps your vCenter and KMS on physically separate at all time.