Hi Alan,
First off all, please go through the following KB to understand how the web agent to policy server connectivity works :
http://www.ca.com/us/support/ca-support-online/product-content/knowledgebase-articles/tec450832.aspx
Then, coming back to your question , of when does the web agent determines that the policy server connection is bad is :
It determines the connection is bad when it cannot open the socket connection to the policy server on the policy server ports.
and I believe, in your case, when policy server is having issue with policy store connectivity and logs "reactor down" , it is my belief that , it is still accepting the tcp socket connection to those ports.
It is just that reactor threads are not available to properly initialize the agent to PS connections, which is why the agent is not failing over to the next available policy server.
I am not in front of my computer at the moment, but I could do some testing and validate this sometime next week and get back to you.
In the interim if you have any question, please let us know.
Regards,
Ujwol Shrestha