Hi Hristo - this is usually set by setting up host Affinity Rules.
these prevent VMs migrating using DRS etc and should also cover you in the instance of HA being invoked. Of course, you need to be aware that in the instance of ESX host loss, your VM will not restart elsewhere (but I guess if you have a hardware dependency, that is not a problem?)
How are you adding this hardware to your VM (if it is virtual hardware, the virtual hardware could be available across all hosts in a cluster - if it is USB, you could consider one of the 3rd party USB network style replicators which should allow access to it from anywhere)