I have been experiencing this same problem since March. Ever since RU5. We are currently at RU6a and still have this problem.
Constant and continous errors is the logs - over and over again
Address already in use: connect [Site:sepm] [Server:sepm]
Failed to connect to server.
SEPM "works" but is slow. I have had a ticket open with Symantec for at least 5 + months. We are not running replication.
If this was fixed in MR1 - it reappeared in RU5.