Mark,
I know this is a bit late, but I'm replying for those of us that end up here as a result of your post being one of the first google results..
Hopefully you didn't give up and figured this out already.
I understand exactly what you're trying to do, and don't really understand why RParker was so confused..
My intention was to configure a machine similarly but with one vm on the tv, and another on a display in my office, thus having just an lcd, keyboard, and mouse on my desk.. no noise, always on, etc..
Anyways, what I ended up finding was that ESXi didn't support it at the time.. I think that's on its way to changing, but Xen appeared to be the best option when I first researched this, and looks to have come quite a ways since then. There's a wiki page on their site titled Xen vga passthrough, here's a link: http://wiki.xen.org/xenwiki/XenVGAPassthrough.html
There are also a list of adapters that have been tested and are known to work: http://wiki.xen.org/xenwiki/XenVGAPassthroughTestedAdapters.html
It's entirely awesome and obviously ironic that a technology that started as a mainframe with terminal access has evolved to powerful desktop systems, and is now moving right back to a mainframe (the internet) and terminals (desktops, netbooks, chromebooks, smartphones).
I think it's only fitting that those of us with an affinity for integrating and exploiting technology at home, should set up a 'mainframe'
As I recall, the hard part of vga passthrough is that the videocard needs to be 'rebooted' each time the vm boots, and the videocard's bios needs to be transferred into the vm's mobo bios. It took a while before anyone took that on. It would be great if VMware would tackle that, but as they cater primarily to server farms, I don't think it'll ever happen..