DX Infrastructure Manager

Expand all | Collapse all

HUB_QOS Queued

Jump to Best Answer
  • 1.  HUB_QOS Queued

    Posted 05-20-2020 03:06 PM

    This is a secondary Hub, Need help on how to clear the queue.Our database admin restarted the sql services and mentioned everything is fine from the sql side.



    ------------------------------
    Thanks
    ------------------------------


  • 2.  RE: HUB_QOS Queued

    Posted 05-20-2020 03:10 PM
    right click on the queue in this gui and select clear or the equivelant, if you have not tried already.


  • 3.  RE: HUB_QOS Queued

    Posted 05-20-2020 03:35 PM
    Are there any typical side effects of clearing  the queue​?

    ------------------------------
    Thanks
    ------------------------------



  • 4.  RE: HUB_QOS Queued

    Posted 05-20-2020 03:13 PM
    HI,

    So when you have an attach queue, there has to be a corresponding get queue on another hub.
    If this is not set up messages from this secondary hub will not make it up the chain to the primary,
    Currently, the GET queue is not connecting for some reason.
    You need to check the upstream GET queue to see why it is not connecting and maybe restart that hub.
    this has nothing to do with the back end SQL server.


    ------------------------------
    Gene Howard
    Principal Support Engineer
    Broadcom
    ------------------------------



  • 5.  RE: HUB_QOS Queued

    Posted 05-20-2020 03:34 PM
    How do we check if  GET has issues with connecting​. Our database admin said there are lot of table created in database .Does it has anything to do with the queue?


    ------------------------------
    Thanks
    ------------------------------



  • 6.  RE: HUB_QOS Queued

    Posted 05-20-2020 03:39 PM
    again this has nothing to do with the back end database.
    this has to do with hub to hub setup and communication.
    I do not know who set up your environment but your documents should have a layout of how your Queue messages are to go from secondary hubs to the primary.
    This needs to be reviewed to understand how this is meant to be set up in your environment.

    Do you have a two-tiered environment or three-tiered?
    two-tiered:
    Primary hub -> secondary hub
    three-tiered:
    Primary hub ->tunnel/concentrator hub -> client/monitoring hub.

    you might want to review the below to better understand how UIM works.
    http://techdocs.broadcom.com/content/dam/broadcom/techdocs/us/en/assets/docops/uim902/ca_uim_reference_architecture.pdf

    http://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-enterprise-software/it-operations-management/unified-infrastructure-management/20-1/getting-started/ca-uim-overview.html

    ------------------------------
    Gene Howard
    Principal Support Engineer
    Broadcom
    ------------------------------



  • 7.  RE: HUB_QOS Queued

    Posted 05-20-2020 03:51 PM
    It is two- tiered setup.I think the messages are setup to go from hub2 to hub1.​

    ------------------------------
    Thanks
    ------------------------------



  • 8.  RE: HUB_QOS Queued

    Posted 05-20-2020 03:59 PM
    Is hub1 your primary hub?
    if  not than you do not have a two tiered or that is not where the get queue is setup.


    ------------------------------
    Gene Howard
    Principal Support Engineer
    Broadcom
    ------------------------------



  • 9.  RE: HUB_QOS Queued

    Posted 05-20-2020 04:20 PM
    Yes hub1 is our primary hub and it has the get queue setup as below




    HUB2 as below


    ------------------------------
    Thanks
    ------------------------------



  • 10.  RE: HUB_QOS Queued

    Posted 05-20-2020 04:24 PM
    try deleting the queue from the primary hub, restarting the hub, and then adding it back.
    check your hub loglevel 3 logs for hub1 for errors.
    If you are using a tunnel make sure the tunnel is working.
    if not using tunnels make sure the named IP address is set on hub1 properly.

    ------------------------------
    Gene Howard
    Principal Support Engineer
    Broadcom
    ------------------------------



  • 11.  RE: HUB_QOS Queued

    Posted 05-20-2020 05:53 PM
    on HUB2 can we try right click and emplty before deleting the queue in the primary hub?​
    Should we double check anything before emptying the queue?

    ------------------------------
    Thanks
    ------------------------------



  • 12.  RE: HUB_QOS Queued

    Posted 05-20-2020 06:04 PM
    Edited by Gabriel Orlando 05-20-2020 06:07 PM
    I had the same issue several times. The delete / create (attach/post) queues procedure always helps. And a nimbus service restart in primary and then secondary hub is always helpfull.
    ------------------------------
    Cordinador Ingenieria y Soporte
    Banco Patagonia
    ------------------------------



  • 13.  RE: HUB_QOS Queued

    Posted 05-21-2020 09:01 AM
    no need to empty the queue before recreating the get side.

    ------------------------------
    Gene Howard
    Principal Support Engineer
    Broadcom
    ------------------------------



  • 14.  RE: HUB_QOS Queued
    Best Answer

    Posted 05-21-2020 09:58 AM
    The queue on hub2 in the original screenshot is green, so it looks like hub1 is connected to hub2. It even shows hundreds of thousands of messages processed, so that probably means the connection has been active for quite a while rather than flapping. This kind of thing could still be caused by a network connectivity issue that slows the transfer of data between the hubs, but it doesn't look like communication is fully blocked.

    How many queues do you have enabled on hub1? And is hub1 running Windows? If you're at more than 64 queues on a Windows hub, you are likely to run into slowdowns processing messages through the queues. Hopefully that's not the issue, as it ends up being something you have to fix at an architectural level. In the meantime you could get temporary relief by combining some of the alarm and QoS queues on the secondary hubs to reduce the load. It would be best not to keep it that way permanently.


  • 15.  RE: HUB_QOS Queued

    Posted 05-21-2020 10:03 AM
    At @Keith Kruepke thanks for pointing that out. I think when I looked at the first time I did not see the hub name in the ID column.
    I just looked at it again and realized it had been masked, and not that is was missing.
    I can see you have the default bulk size set to 100.
    This may just be too low for this connection.
    Go to the get queue side and set the bulk size to 1000 and restart and see if it does not clear up the problem.​

    ------------------------------
    Gene Howard
    Principal Support Engineer
    Broadcom
    ------------------------------



  • 16.  RE: HUB_QOS Queued

    Posted 05-21-2020 10:13 AM
    Good call! If there were just a lot of messages in the queue due to a downtime or something, it could just be working through them very slowly with a low bulk size.

    One thing I've noticed is that sometimes the hubs will automatically use a larger bulk size than what is configured when connecting to other hubs. (When that happens, I think it is usually or always 1000.) But that doesn't happen every time. I haven't been able to pinpoint a reason it happens only some of the time though, which always makes me a little uncomfortable.