A majority of your memory is also probably going to SQL, SQL by nature tries to cache as much memory as it can get its hands on in the event it needs to use it. Exchagne does the same thing. However like the pervious user stated I would just bumped the memory to 10GB and let vCenter sort its self out, or break out the roles a little by having a differetn server for SQL then your vCenter, ect, ect
Hope this has helped