You can restart the service - the impact will be if any user happens to be logged into a session through that server will likely get a session interrupt/disconnect. Do you know if that service has historically run high on that VM?
Another option would be to schedule a maintenance reboot on the VM if that is possible - you could also look at if there are resources available to add virtual memory to the VM that would take effect upon a restart.