well, I`m a player too,and has some EXP with remote gaming.
please use ESXi 6.7 or 7.0u3(laster ver) as ur game server
there are 2 way to use ur graphics card in VM
the new way is 7.0ver has 2 different way pass through ur PCI device(i can`t test it now)
if u use 6.5 or 6.7 the old pass through, u can assign ur graphics card direct to VM,that means the VM has all graphics card,and can`t migrate this VM.
if u use 7 , there are a new pass though way, u can give a graphics card model to VM , if u have another same model card in another ESXi server in same Cluster, u can migrate VM on them.(I`not test yet)
till this moment ESXi6.7 ,vsphere not support pool the graphics card and easy assign,i hope this feature too.
for u game only,it eazy to assign graphics card to VM
for graphics card pool and cloud gaming for players , maybe has other choose