General advice - upgrade your vcenter to 5.5 as minimum. 5.1 is pretty and it has numerous issues.
To troubleshoot current issue:
1. How many CPU and RAM is configured for vcenter? What is the usage of CPU\RAM on the server?
2. Is it a virtual machine or physical server?
3. Are all components (SSO, inventory service, vcenter and web client) installed on the same server?
4. Is database also installed on the same server?