Poor hub performance with multiple tunnels initiated

[Anon Anon]

Hello, everyone.

I've got a setup where a single hub has almost 80 tunnels established to my firewalled remote locations. Remote machines mostly report the standard metrics - cdm, interface_traffic and storage qoses, so the qos flow isn't that huge. Once the tunneling is enabled on the hub, responsiveness of the hub dramatically deteriorates. I'm receiving frequent communication errors when opening probe's configurations and the distribution tool is absolutely unusable as it complains that there is "no contact with distribution server". Infrastructure Manager is sluggish and unviable as well. In the beginning I though there might be some excessive DB load and increased hub_bulk_size to 2000, but to no avail. The disk IO for the SQL server varies from 1 to 30 mb\s per second, which is acceptable and shouldn't be a bottleneck. Average CPU load on the server is as low as 5%. Therefore, it's neither the DB fault, nor the lack of resources. At least this is how it looks. Once the tunneling is disabled, the GUI becomes lightning fast.

How do I find the real bottleneck that makes the hub that slow?

 

UIM 8.31

Win8kR2

32 GB RAM

24 cores CPU

[Anon Anon]

Hi,

 

You should probably check your subscriber numbers: http://www.ca.com/us/support/ca-support-online/product-content/knowledgebase-articles/tec000004446.aspx?intcmp=searchres…

 

I would also check what netstat says, if there's a lot of connection is close_wait, etc. I would suspect network issues first..

 

Also I suspect the distribution thing is mostly due to the distribution progress window: If you close it you'll likely find that IM works better altogether. I experience this on all my primary hubs.

 

-jon

[Anon Anon]

The hub probe lists only 32 active subscribers/queues. According to the link, the issues arise only when this number exceeds 64 queues.

Also there's just a dozen of close_wait and a dozen of time_wait connections in netstat, all others are established (~ 230 connections). Is that too much?

[RowanCollis]

Yes, look at that article. I would say you have too many tunnels.

Try putting half (40) on another hub and I bet it will work fine.

cheers

[Anon Anon]

I've tried disabling half of my tunnels, but the hub is still sluggish. However, once I've cut them down to just 10 connections, the hub revived. Maybe it's worth mentioning that my hub acts as a tunnel server initiating connections to my tunnel clients, not the other way round.