First check your Vm memory usage
From VC server go to summary tab of selected Vm and check the CPU usage. Also is this Vm is under any resource pool or any cluster?
You may also check with free sofwtars to check exact memory requirement and free them eg: Memory optimizer 2002a , isat memory 1.1.
But if you wish to just decrease RAM of required Linux box , then first switch off your Vm and then go to edit settings and memory section and assign the relevan memory thats illustrated by above mentioned free softwares or your observation.
There is no fix memory you can say and assign to your VM..it depends which applications you are running over it..
To get rid out of it ..i thnk better option is to put that Vm in a cluster and creates some resource pool.
Note: If you find this answer helpfull or correct kinldy revert with appropriate check box for same.
Regards,
vDeepak