Placing the virtual machine’s swap file on a high speed/high bandwidth storage system will result in the smallest performance impact. The swap file location can be set with the sched.swap.dir option in the vSphere Client (select Edit virtual machine settings, choose the Options tab, select Advanced, and click Configuration Parameters). If this option is not set, the swap file will be created in the virtual machine’s working directory: either the directory specified by workingDir in the virtual machine’s .vmx file, or, if this variable is not set, in the directory where the .vmx file is located. The latter is the default behavior.
You are correct that would not be much of a performance hit. I have had a couple of different deployments where we had OS disk on cheaper SATA drives and the data on a different storage type. So yes, it is possible, will you see much of a performance increase, probably not.