Mar 30 15:21:34:666 [140605547460672] 0 hub: WARNING: license not set in config file Mar 30 15:21:34:666 [140605547460672] 0 hub: -------------------------------------------------------------------------------------------------------- Mar 30 15:21:34:666 [140605547460672] 0 hub: ----- Hub 9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019] initializing ----- Mar 30 15:21:34:666 [140605547460672] 4 hub: hubinfo create: /DOMAIN/TunnelBHA 10.x.x.x/48002 0x20f5780 Mar 30 15:21:34:666 [140605547460672] 3 hub: set hglob.myhub = 0x20f5780 Mar 30 15:21:34:666 [140605547460672] 2 hub: ch_hublist /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:21:34:666 [140605547460672] 0 hub: Tunnel has started in thread [ 140605367465728 ] Mar 30 15:21:34:666 [140605547460672] 2 hub: nns(+):/DOMAIN/TunnelBHA 127.0.0.1/48002 Mar 30 15:21:34:666 [140605547460672] 2 hub: nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:21:34:666 [140605547460672] 2 hub: nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/controller 127.0.0.1/48000 Mar 30 15:21:34:666 [140605367465728] 1 hub: CORE - thread starting Mar 30 15:21:34:666 [140605547460672] 2 hub: nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME 127.0.0.1/48000 Mar 30 15:21:34:666 [140605547460672] 2 hub: HUB SEC: using plain-text mode: security.cfg Mar 30 15:21:34:666 [140605547460672] 1 hub: HUB SEC: session=1717591255 Mar 30 15:21:34:666 [140605547460672] 4 hub: cfgReader file open: security.cfg Mar 30 15:21:34:667 [140605547460672] 4 hub: cfgReader file close: security.cfg Mar 30 15:21:34:667 [140605547460672] 1 hub: HUB SEC: expire=21600 Mar 30 15:21:34:667 [140605547460672] 1 hub: HUB SEC: version=0217 Mar 30 15:21:34:667 [140605547460672] 1 hub: IP validation check on messages is enabled (ignore_ip [local hub.cfg] = no, ignore_ip [global security.cfg] = no) Mar 30 15:21:34:667 [140605547460672] 1 hub: HUB SEC: secure_callbacks_from_primary_hub_only=no Mar 30 15:21:34:667 [140605547460672] 1 hub: HUB SEC: password_security_version=2 Mar 30 15:21:34:667 [140605547460672] 1 hub: HUB SEC: incompatible_security_propagation=no Mar 30 15:21:34:667 [140605547460672] 1 hub: HUB SEC: auth_mode=0000 Mar 30 15:21:34:667 [140605547460672] 1 hub: HUB SEC: trusted IPs= Mar 30 15:21:34:667 [140605547460672] 3 hub: '10.x.x.x' added as a trusted IP address Mar 30 15:21:34:667 [140605547460672] 3 hub: '10.x.x.x' added as a trusted IP address Mar 30 15:21:34:667 [140605547460672] 3 hub: '10.x.x.x' added as a trusted IP address Mar 30 15:21:34:667 [140605547460672] 1 hub: There are now 3 registered trusted IP addresses Mar 30 15:21:34:667 [140605547460672] 4 hub: cfgReader file open: security.cfg Mar 30 15:21:34:668 [140605547460672] 4 hub: cfgReader file close: security.cfg Mar 30 15:21:34:668 [140605547460672] 4 hub: cfgReader file open: security.cfg Mar 30 15:21:34:668 [140605367465728] 2 hub: RAND wrote 1024 new random bytes to rand.rnd Mar 30 15:21:34:668 [140605547460672] 4 hub: cfgReader file close: security.cfg Mar 30 15:21:34:668 [140605367465728] 2 hub: RAND snagged 1024 random bytes from rand.rnd Mar 30 15:21:34:668 [140605367465728] 2 hub: RAND claims sufficient entropy for the PRNG Mar 30 15:21:34:668 [140605367465728] 1 hub: Using OpenSSL 1.0.2p 14 Aug 2018 Mar 30 15:21:34:668 [140605367465728] 4 hub: cfgReader file open: hub.cfg Mar 30 15:21:34:668 [140605547460672] 4 hub: cfgReader file open: security.cfg Mar 30 15:21:34:669 [140605367465728] 4 hub: cfgReader file close: hub.cfg Mar 30 15:21:34:669 [140605367465728] 4 hub: cfgReader file open: hub.cfg Mar 30 15:21:34:669 [140605367465728] 4 hub: cfgReader file close: hub.cfg Mar 30 15:21:34:669 [140605367465728] 3 hub: CORE - server common name: 10.x.x.x Mar 30 15:21:34:669 [140605367465728] 0 hub: CORE server weak cipher configuration upgraded Mar 30 15:21:34:669 [140605367465728] 1 hub: CORE server cipher=EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH:!RC4:!DSS Mar 30 15:21:34:669 [140605367465728] 1 hub: (ssl_ctx_setup) ca location -> certs/server.ca.ec.cert.pem Mar 30 15:21:34:669 [140605367465728] 1 hub: (ssl_ctx_setup) certificate -> certs/server.ec.cert.pem Mar 30 15:21:34:669 [140605367465728] 1 hub: (ssl_ctx_setup) private key file -> certs/server.ec.key.pem Mar 30 15:21:34:669 [140605547460672] 4 hub: cfgReader file close: security.cfg Mar 30 15:21:34:669 [140605547460672] 4 hub: cfgReader file open: security.cfg Mar 30 15:21:34:670 [140605367465728] 1 hub: Could not open DH file 'certs/dh0.pem' Mar 30 15:21:34:670 [140605367465728] 2 hub: DH Diffie-Hellman initialized with 1024 bit key Mar 30 15:21:34:670 [140605547460672] 4 hub: cfgReader file close: security.cfg Mar 30 15:21:34:670 [140605547460672] 4 hub: cfgReader file open: security.cfg Mar 30 15:21:34:671 [140605547460672] 4 hub: cfgReader file close: security.cfg Mar 30 15:21:34:671 [140605547460672] 4 hub: cfgReader file open: security.cfg Mar 30 15:21:34:672 [140605547460672] 4 hub: cfgReader file close: security.cfg Mar 30 15:21:34:672 [140605547460672] 1 hub: HUB SEC: ON Mar 30 15:21:34:672 [140605547460672] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:21:34:672 [140605547460672] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:21:34:672 [140605547460672] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:21:34:672 [140605547460672] 3 hub: add robots from robot.sds file to this hub's robot lists Mar 30 15:21:34:672 [140605547460672] 0 hub: Initially read robot file: r_ip 10.x.x.x:48000 r_reachable_via_ip 127.0.0.1:48000 Mar 30 15:21:34:673 [140605547460672] 0 hub: At the end, read robot file: r_ip 10.x.x.x:48000 r_reachable_via_ip 127.0.0.1:48000 Mar 30 15:21:34:673 [140605547460672] 5 hub: add robot - robot: name=ROBOT_NAME ip=10.x.x.x port=48000 is_passive=0 Mar 30 15:21:34:673 [140605547460672] 5 hub: add robot success, robot added (ROBOT_NAME) Mar 30 15:21:34:673 [140605547460672] 1 hub: Added 1 robots from cache Mar 30 15:21:34:673 [140605547460672] 2 hub: nimSessionServerStrict - host 127.0.0.1, port = 48002 Mar 30 15:21:34:673 [140605547460672] 2 hub: sockServer - 0x20f9050:127.0.0.1/48002:fd=4 Mar 30 15:21:34:673 [140605547460672] 0 hub: ----- HUB started ----- Mar 30 15:21:34:673 [140605547460672] 0 hub: Name = TunnelBHA External IP = '10.x.x.x' Local IP = '127.0.0.1' Port = 48002 Mar 30 15:21:34:673 [140605547460672] 0 hub: Domain = DOMAIN addr = /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:21:34:673 [140605547460672] 0 hub: log_level= 5 , log_file=hub.log Mar 30 15:21:34:673 [140605547460672] 1 hub: LDAP LIB = 2.00 Mar 30 15:21:34:673 [140605547460672] 0 hub: license on robots=0 expire=Unlimited Mar 30 15:21:34:673 [140605547460672] 0 hub: ----- Mar 30 15:21:34:673 [140605547460672] 2 hub: do_assign_to_list entered... Mar 30 15:21:34:673 [140605547460672] 1 hub: AssignToList - used 0 ms to lookup 1 users Mar 30 15:21:34:673 [140605547460672] 2 hub: do_assign_to_list finished... status: OK (0), time used: 0 ms Mar 30 15:21:34:673 [140605547460672] 2 hub: hubi is starting in thread 939521792 Mar 30 15:21:34:673 [140605283890944] 0 hub: hubi main thread started Mar 30 15:21:34:673 [140605547460672] 4 hub: CONNECT: 0x2101790(5) 127.0.0.1/43924->127.0.0.1/48000 Mar 30 15:21:34:673 [140605547460672] 2 hub: nimSessionServerStrict - host 127.0.0.1, port = 48001 Mar 30 15:21:34:673 [140605547460672] 2 hub: sockServer - 0x2101790:127.0.0.1/48001:fd=5 Mar 30 15:21:34:673 [140605547460672] 4 hub: CONNECT: 0x20fdb40(6) 127.0.0.1/43926->127.0.0.1/48000 Mar 30 15:21:34:674 [140605547460672] 1 hub: Spooler - initializing Mar 30 15:21:34:674 [140605547460672] 3 hub: validating character encoding of config file: hub.cfg Mar 30 15:21:34:674 [140605547460672] 4 hub: nimCharsetValidateFile: hub.cfg: no target charset Mar 30 15:21:34:674 [140605547460672] 4 hub: cfgReader file open: hub.cfg Mar 30 15:21:34:674 [140605547460672] 4 hub: cfgReader file close: hub.cfg Mar 30 15:21:34:674 [140605547460672] 4 hub: cfgReader file open: ../robot/robot.cfg Mar 30 15:21:34:674 [140605547460672] 4 hub: cfgReader file close: ../robot/robot.cfg Mar 30 15:21:34:674 [140605547460672] 2 hub: spooler - user tags: os_user1='', os_user2='', os_user_retrieve_from_robot=0 (OFF) Mar 30 15:21:34:674 [140605547460672] 4 hub: cfgReader file open: hub.cfg Mar 30 15:21:34:674 [140605547460672] 4 hub: cfgReader file close: hub.cfg Mar 30 15:21:34:674 [140605547460672] 4 hub: (load_plugins_from_cfg) Plugin loading started... Mar 30 15:21:34:674 [140605547460672] 4 hub: (load_plugins_from_cfg) Found plugin section (/hub/plugins/plugin_metric) at index (0) Mar 30 15:21:34:674 [140605547460672] 4 hub: (load_plugins_from_cfg) Read plugin (pluginName=plugin_metric, pluginPath=plugins/plugin_metric/libplugin_metric.so, active=1) from cfg. Mar 30 15:21:34:674 [140605547460672] 0 hub: (load_bus_plugin) Loaded plugin (/opt/nimsoft/plugins/plugin_metric/libplugin_metric.so) successfully. Mar 30 15:21:34:674 [140605547460672] 1 hub: (load_plugins_from_cfg) Loaded plugin lib (plugin_metric). Calling 'configure'. Mar 30 15:21:34:674 [140605547460672] 4 hub: cfgReader file open: /opt/nimsoft/plugins/plugin_metric/plugin_metric.cfg Mar 30 15:21:34:675 [140605547460672] 4 hub: cfgReader file close: /opt/nimsoft/plugins/plugin_metric/plugin_metric.cfg Mar 30 15:21:34:675 [140605547460672] 4 hub: cfgReader file open: /opt/nimsoft/plugins/plugin_metric/plugin_metric.cfg Mar 30 15:21:34:675 [140605547460672] 4 hub: cfgReader file close: /opt/nimsoft/plugins/plugin_metric/plugin_metric.cfg Mar 30 15:21:34:675 [140605547460672] 4 hub: cfgReader file open: /opt/nimsoft/plugins/plugin_metric/plugin_metric.cfg Mar 30 15:21:34:675 [140605547460672] 4 hub: cfgReader file close: /opt/nimsoft/plugins/plugin_metric/plugin_metric.cfg Mar 30 15:21:34:675 [140605547460672] 5 hub: (Metric Plugin): Read probes from metric cfg - no in line metrics configuration found Mar 30 15:21:34:675 [140605547460672] 5 hub: (Metric Plugin): Read probes from metric cfg - successfully read profile and policy configuration for probes Mar 30 15:21:34:675 [140605547460672] 5 hub: (Metric Plugin): reading cached information from ./../metrics/qos_definitions.pds Mar 30 15:21:34:675 [140605547460672] 0 hub: Attempt to create malformed metric object Mar 30 15:21:34:675 [140605547460672] 0 hub: Metrics post-processing plugin [9.20SHF9.289 Feb 5 2020] successfully loaded and configured Mar 30 15:21:34:675 [140605547460672] 3 hub: (configure_bus_plugin) loading (4) callbacks from plugin into array Mar 30 15:21:34:675 [140605547460672] 3 hub: (load_plugins_from_cfg) Successfully configured module (plugin_metric) and loaded callbacks. Mar 30 15:21:34:675 [140605547460672] 4 hub: cfgReader file open: ../robot/spooler.cfg Mar 30 15:21:34:675 [140605547460672] 4 hub: cfgReader file close: ../robot/spooler.cfg Mar 30 15:21:34:676 [140605547460672] 3 hub: Removed 'q/temp' Mar 30 15:21:34:676 [140605547460672] 1 hub: System has 8 available cores, default spooler inbound thread pool size is 8 Mar 30 15:21:34:676 [140605547460672] 1 hub: Using default inbound thread pool size Mar 30 15:21:34:676 [140605547460672] 3 hub: newRoute name=TunBHA_Alarms pr=0x20ff400 Mar 30 15:21:34:676 [140605547460672] 1 hub: Configured ATTACH queue route 'TunBHA_Alarms' for 'alarm' (bulk:100 min:0 wait:0 timeout:180) Mar 30 15:21:34:676 [140605547460672] 3 hub: newRoute name=TunBHA_Others pr=0x20ffb10 Mar 30 15:21:34:676 [140605275498240] 2 hub: Queue management thread for 'TunBHA_Alarms' started Mar 30 15:21:34:676 [140605547460672] 1 hub: Configured ATTACH queue route 'TunBHA_Others' for 'QOS_MESSAGE,QOS_DEFINITION,QOS_BASELINE,audit,probe_discovery' (bulk:100 min:0 wait:0 timeout:180) Mar 30 15:21:34:677 [140605547460672] 3 hub: newRoute name=baseline_engine.BASELINE_CONFIG pr=0x213c6e0 Mar 30 15:21:34:677 [140605267105536] 2 hub: Queue management thread for 'TunBHA_Others' started Mar 30 15:21:34:677 [140605275498240] 1 hub: Queue 'TunBHA_Alarms' has queued file '15.sds' for future processing Mar 30 15:21:34:677 [140605547460672] 1 hub: Configured ATTACH queue route 'baseline_engine.BASELINE_CONFIG' for 'BASELINE_CONFIG' (bulk:100 min:0 wait:0 timeout:180) Mar 30 15:21:34:677 [140605258712832] 2 hub: Queue management thread for 'baseline_engine.BASELINE_CONFIG' started Mar 30 15:21:34:677 [140605267105536] 1 hub: Queue 'TunBHA_Others' has queued file '13.sds' for future processing Mar 30 15:21:34:677 [140605267105536] 1 hub: Queue 'TunBHA_Others' has queued file '14.sds' for future processing Mar 30 15:21:34:677 [140605267105536] 1 hub: Queue 'TunBHA_Others' has queued file '15.sds' for future processing Mar 30 15:21:34:677 [140605547460672] 3 hub: newRoute name=prediction_engine.PREDICTION_CONFIG pr=0x213d090 Mar 30 15:21:34:677 [140605250320128] 1 hub: Initalizing read file '15.sds' for 'TunBHA_Alarms' Mar 30 15:21:34:677 [140605275498240] 1 hub: Queue 'TunBHA_Alarms' has 1 existing data files Mar 30 15:21:34:677 [140605275498240] 1 hub: Queue 'TunBHA_Alarms' will start reading at file sequence no. 15 Mar 30 15:21:34:677 [140605258712832] 1 hub: Queue 'baseline_engine.BASELINE_CONFIG' has queued file '14.sds' for future processing Mar 30 15:21:34:677 [140605267105536] 1 hub: Queue 'TunBHA_Others' has 3 existing data files Mar 30 15:21:34:677 [140605267105536] 1 hub: Queue 'TunBHA_Others' will start reading at file sequence no. 13 Mar 30 15:21:34:677 [140605233534720] 2 hub: Queue management thread for 'prediction_engine.PREDICTION_CONFIG' started Mar 30 15:21:34:677 [140605275498240] 1 hub: Queue 'TunBHA_Alarms' will start writing at file sequence no. 16 Mar 30 15:21:34:677 [140605547460672] 1 hub: Configured ATTACH queue route 'prediction_engine.PREDICTION_CONFIG' for 'PREDICTION_CONFIG' (bulk:100 min:0 wait:0 timeout:180) Mar 30 15:21:34:677 [140605250320128] 1 hub: Read file '15.sds' initialized for 'TunBHA_Alarms', 13 msgs found (@ pos 0), 0 ms used Mar 30 15:21:34:677 [140605233534720] 1 hub: Queue 'prediction_engine.PREDICTION_CONFIG' has queued file '14.sds' for future processing Mar 30 15:21:34:677 [140605015455488] 1 hub: Initalizing read file '14.sds' for 'baseline_engine.BASELINE_CONFIG' Mar 30 15:21:34:677 [140605267105536] 1 hub: Queue 'TunBHA_Others' will start writing at file sequence no. 16 Mar 30 15:21:34:677 [140605015455488] 1 hub: Read file '14.sds' initialized for 'baseline_engine.BASELINE_CONFIG', 0 msgs found (@ pos 0), 0 ms used Mar 30 15:21:34:677 [140605241927424] 1 hub: Initalizing read file '13.sds' for 'TunBHA_Others' Mar 30 15:21:34:677 [140605267105536] 1 hub: Queue 'TunBHA_Others' opened file '16.sds' for writing Mar 30 15:21:34:677 [140605275498240] 1 hub: Queue 'TunBHA_Alarms' opened file '16.sds' for writing Mar 30 15:21:34:677 [140605258712832] 1 hub: Queue 'baseline_engine.BASELINE_CONFIG' has 1 existing data files Mar 30 15:21:34:677 [140605258712832] 1 hub: Queue 'baseline_engine.BASELINE_CONFIG' will start reading at file sequence no. 14 Mar 30 15:21:34:677 [140605258712832] 1 hub: Queue 'baseline_engine.BASELINE_CONFIG' will start writing at file sequence no. 15 Mar 30 15:21:34:677 [140605258712832] 1 hub: Queue 'baseline_engine.BASELINE_CONFIG' opened file '15.sds' for writing Mar 30 15:21:34:677 [140605250320128] 2 hub: Queue sender thread for 'TunBHA_Others' started Mar 30 15:21:34:678 [140605015455488] 1 hub: Queue 'baseline_engine.BASELINE_CONFIG' file '14.sds' does not contain sendable msgs, deleted Mar 30 15:21:34:678 [140604998670080] 1 hub: Initalizing read file '14.sds' for 'prediction_engine.PREDICTION_CONFIG' Mar 30 15:21:34:678 [140604998670080] 1 hub: Read file '14.sds' initialized for 'prediction_engine.PREDICTION_CONFIG', 0 msgs found (@ pos 0), 0 ms used Mar 30 15:21:34:678 [140604998670080] 1 hub: Queue 'prediction_engine.PREDICTION_CONFIG' file '14.sds' does not contain sendable msgs, deleted Mar 30 15:21:34:678 [140605241927424] 1 hub: Read file '13.sds' initialized for 'TunBHA_Others', 20 msgs found (@ pos 678699), 1 ms used Mar 30 15:21:34:678 [140605241927424] 1 hub: Initalizing read file '14.sds' for 'TunBHA_Others' Mar 30 15:21:34:678 [140605241927424] 1 hub: Read file '14.sds' initialized for 'TunBHA_Others', 18 msgs found (@ pos 0), 0 ms used Mar 30 15:21:34:678 [140605241927424] 1 hub: Initalizing read file '15.sds' for 'TunBHA_Others' Mar 30 15:21:34:678 [140605241927424] 1 hub: Read file '15.sds' initialized for 'TunBHA_Others', 13 msgs found (@ pos 0), 0 ms used Mar 30 15:21:34:679 [140605547460672] 2 hub: spooler_main_thread - has started in thread 140605007062784 Mar 30 15:21:34:679 [140605233534720] 1 hub: Queue 'prediction_engine.PREDICTION_CONFIG' has 1 existing data files Mar 30 15:21:34:679 [140605233534720] 1 hub: Queue 'prediction_engine.PREDICTION_CONFIG' will start reading at file sequence no. 14 Mar 30 15:21:34:679 [140605233534720] 1 hub: Queue 'prediction_engine.PREDICTION_CONFIG' will start writing at file sequence no. 15 Mar 30 15:21:34:679 [140605233534720] 1 hub: Queue 'prediction_engine.PREDICTION_CONFIG' opened file '15.sds' for writing Mar 30 15:21:34:679 [140604990277376] 2 hub: Queue sender thread for 'TunBHA_Alarms' started Mar 30 15:21:34:679 [140605007062784] 2 hub: spooler main thread started Mar 30 15:21:34:680 [140605547460672] 3 hub: passive robot thread - has started in thread 140605241927424 Mar 30 15:21:34:680 [140605241927424] 1 hub: passive robot thread - starting ... Mar 30 15:21:34:680 [140605547460672] 4 hub: cfgReader file open: hub.cfg Mar 30 15:21:34:680 [140604981884672] 2 hub: Queue sender thread for 'baseline_engine.BASELINE_CONFIG' started Mar 30 15:21:34:680 [140605015455488] 2 hub: Post input handling thread - started Mar 30 15:21:34:680 [140604998670080] 2 hub: Post input handling thread - started Mar 30 15:21:34:680 [140604965099264] 2 hub: Post input handling thread - started Mar 30 15:21:34:680 [140604973491968] 2 hub: Queue sender thread for 'prediction_engine.PREDICTION_CONFIG' started Mar 30 15:21:34:680 [140605241927424] 4 hub: cfgReader file open: hub.cfg Mar 30 15:21:34:680 [140605547460672] 4 hub: cfgReader file close: hub.cfg Mar 30 15:21:34:680 [140604612802304] 2 hub: Post input handling thread - started Mar 30 15:21:34:680 [140605547460672] 5 hub: topology - main thread - cfg - on = 1, max_threads = 10, restart_wait = 60 seconds Mar 30 15:21:34:681 [140604604409600] 2 hub: Post input handling thread - started Mar 30 15:21:34:681 [140604587624192] 2 hub: Post input handling thread - started Mar 30 15:21:34:681 [140604596016896] 3 hub: topology - main thread - starting worker thread pool ... Mar 30 15:21:34:681 [140604579231488] 2 hub: Post input handling thread - started Mar 30 15:21:34:681 [140604570838784] 2 hub: Post input handling thread - started Mar 30 15:21:34:681 [140605547460672] 1 hub: topology - main thread - started Mar 30 15:21:34:681 [140604562446080] 2 hub: spooler subscriber management thread started Mar 30 15:21:34:681 [140605241927424] 4 hub: cfgReader file close: hub.cfg Mar 30 15:21:34:681 [140604596016896] 5 hub: [-268437760] started worker thread (0xeffff700) Mar 30 15:21:34:681 [140603933320960] 2 hub: do_assign_to_list entered... Mar 30 15:21:34:681 [140604075931392] 5 hub: [-268437760] thread started Mar 30 15:21:34:681 [140604596016896] 5 hub: [-285223168] started worker thread (0xeeffd700) Mar 30 15:21:34:681 [140603933320960] 1 hub: AssignToList - used 0 ms to lookup 1 users Mar 30 15:21:34:681 [140603933320960] 2 hub: do_assign_to_list finished... status: OK (0), time used: 0 ms Mar 30 15:21:34:681 [140605241927424] 5 hub: [-293615872] started worker thread (0xee7fc700) Mar 30 15:21:34:681 [140604059145984] 5 hub: [-285223168] thread started Mar 30 15:21:34:681 [140604042360576] 5 hub: There are 0 objects waiting on 8 threads Mar 30 15:21:34:681 [140604050753280] 5 hub: [-293615872] thread started Mar 30 15:21:34:681 [140605241927424] 5 hub: [-318793984] started worker thread (0xecff9700) Mar 30 15:21:34:681 [140604025575168] 5 hub: [-318793984] thread started Mar 30 15:21:34:681 [140604596016896] 5 hub: [-310401280] started worker thread (0xed7fa700) Mar 30 15:21:34:681 [140605241927424] 5 hub: [-411048192] started worker thread (0xe77fe700) Mar 30 15:21:34:681 [140603933320960] 5 hub: [-411048192] thread started Mar 30 15:21:34:682 [140604596016896] 5 hub: [-402655488] started worker thread (0xe7fff700) Mar 30 15:21:34:682 [140603941713664] 5 hub: [-402655488] thread started Mar 30 15:21:34:681 [140604033967872] 5 hub: [-310401280] thread started Mar 30 15:21:34:682 [140603916535552] 5 hub: [-427833600] thread started Mar 30 15:21:34:682 [140605241927424] 5 hub: [-419440896] started worker thread (0xe6ffd700) Mar 30 15:21:34:682 [140603924928256] 5 hub: [-419440896] thread started Mar 30 15:21:34:682 [140604596016896] 5 hub: [-427833600] started worker thread (0xe67fc700) Mar 30 15:21:34:682 [140605241927424] 5 hub: [-436226304] started worker thread (0xe5ffb700) Mar 30 15:21:34:682 [140603908142848] 5 hub: [-436226304] thread started Mar 30 15:21:34:682 [140604596016896] 5 hub: [-444619008] started worker thread (0xe57fa700) Mar 30 15:21:34:682 [140605241927424] 5 hub: [-453011712] started worker thread (0xe4ff9700) Mar 30 15:21:34:682 [140603899750144] 5 hub: [-444619008] thread started Mar 30 15:21:34:682 [140604596016896] 5 hub: [-536873216] started worker thread (0xdffff700) Mar 30 15:21:34:682 [140603807495936] 5 hub: [-536873216] thread started Mar 30 15:21:34:682 [140603799103232] 5 hub: [-545265920] thread started Mar 30 15:21:34:682 [140603891357440] 5 hub: [-453011712] thread started Mar 30 15:21:34:682 [140605241927424] 5 hub: [-545265920] started worker thread (0xdf7fe700) Mar 30 15:21:34:682 [140604596016896] 5 hub: [-553658624] started worker thread (0xdeffd700) Mar 30 15:21:34:682 [140603790710528] 5 hub: [-553658624] thread started Mar 30 15:21:34:682 [140605241927424] 5 hub: [-562051328] started worker thread (0xde7fc700) Mar 30 15:21:34:682 [140604596016896] 5 hub: [-570444032] started worker thread (0xddffb700) Mar 30 15:21:34:682 [140603773925120] 5 hub: [-570444032] thread started Mar 30 15:21:34:682 [140603782317824] 5 hub: [-562051328] thread started Mar 30 15:21:34:682 [140605241927424] 5 hub: [-578836736] started worker thread (0xdd7fa700) Mar 30 15:21:34:682 [140603757139712] 5 hub: [-587229440] thread started Mar 30 15:21:34:682 [140604596016896] 5 hub: [-587229440] started worker thread (0xdcff9700) Mar 30 15:21:34:682 [140605241927424] 5 hub: [-595622144] started worker thread (0xdc7f8700) Mar 30 15:21:34:682 [140603748747008] 5 hub: [-595622144] thread started Mar 30 15:21:34:682 [140603765532416] 5 hub: [-578836736] thread started Mar 30 15:21:34:768 [140605367465728] 0 hub: CORE server started on port 10.x.x.x:48003 Mar 30 15:21:34:768 [140605367465728] 4 hub: cfgReader file open: hub.cfg Mar 30 15:21:34:768 [140605367465728] 4 hub: cfgReader file close: hub.cfg Mar 30 15:21:34:768 [140605367465728] 5 hub: CORE sessctrl - client<0> has no defined nimport Mar 30 15:21:34:768 [140605367465728] 4 hub: hubinfo create: /DOMAIN/Secondary 10.x.x.x/48002 0x7fe138027630 Mar 30 15:21:34:768 [140605367465728] 2 hub: ch_hublist /DOMAIN/Secondary/ROBOT_NAME/hub Mar 30 15:21:34:768 [140605367465728] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:21:34:768 [140605367465728] 0 hub: CORE load -- (CLIENT) -- private key path load () failed. Reason=(not found) Mar 30 15:21:34:768 [140605367465728] 1 hub: (ssl_ctx_setup) ca location -> certs/client3.pem Mar 30 15:21:34:768 [140605367465728] 1 hub: (ssl_ctx_setup) certificate -> certs/client3.pem Mar 30 15:21:34:768 [140605367465728] 1 hub: (ssl_ctx_setup) private key file -> certs/client3.pem Mar 30 15:21:34:769 [140605367465728] 1 hub: CORE client<0> cipher=EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH:!RC4:!DSS Mar 30 15:21:34:769 [140605367465728] 1 hub: Could not open DH file 'certs/dh0.pem' Mar 30 15:21:34:769 [140605367465728] 1 hub: Could not open DH file 'certs/dh0.pem' Mar 30 15:21:34:769 [140605367465728] 3 hub: Decode Certificate - certificate certs/client3.pem expires 1898905177 Mar 30 15:21:34:769 [140605367465728] 4 hub: CONNECT: 0x7fe138036090(14) 127.0.0.1/43928->127.0.0.1/48000 Mar 30 15:21:34:770 [140605367465728] 4 hub: CONNECT: 0x7fe1380337a0(14) 127.0.0.1/43930->127.0.0.1/48000 Mar 30 15:21:34:770 [140605367465728] 1 hub: tunnel_port_next - Tunnel-000-name[10.x.x.x] received port 48007 from controller Mar 30 15:21:34:770 [140605367465728] 1 hub: CORE added SESSCTRL for 10.x.x.x/48003 Mar 30 15:21:34:770 [140605367465728] 5 hub: CORE sessctrl - client<1> has no defined nimport Mar 30 15:21:34:770 [140605367465728] 4 hub: hubinfo create: /DOMAIN/SecondaryHA 10.x.x.x/48002 0x7fe1380368c0 Mar 30 15:21:34:770 [140605367465728] 2 hub: ch_hublist /DOMAIN/SecondaryHA/ROBOT_NAME/hub Mar 30 15:21:34:770 [140605367465728] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:21:34:770 [140605367465728] 0 hub: CORE load -- (CLIENT) -- private key path load () failed. Reason=(not found) Mar 30 15:21:34:770 [140605367465728] 1 hub: (ssl_ctx_setup) ca location -> certs/client4.pem Mar 30 15:21:34:770 [140605367465728] 1 hub: (ssl_ctx_setup) certificate -> certs/client4.pem Mar 30 15:21:34:770 [140605367465728] 1 hub: (ssl_ctx_setup) private key file -> certs/client4.pem Mar 30 15:21:34:770 [140605367465728] 1 hub: CORE client<1> cipher=EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH:!RC4:!DSS Mar 30 15:21:34:770 [140605367465728] 1 hub: Could not open DH file 'certs/dh0.pem' Mar 30 15:21:34:770 [140605367465728] 1 hub: Could not open DH file 'certs/dh0.pem' Mar 30 15:21:34:770 [140605367465728] 3 hub: Decode Certificate - certificate certs/client4.pem expires 1898906533 Mar 30 15:21:34:771 [140605367465728] 1 hub: tunnel_port_next - Tunnel-001-name[10.x.x.x] received port 48008 from controller Mar 30 15:21:34:771 [140605367465728] 1 hub: CORE added SESSCTRL for 10.x.x.x/48003 Mar 30 15:21:34:771 [140605367465728] 5 hub: CORE sessctrl - client<2> has no defined nimport Mar 30 15:21:34:771 [140605367465728] 4 hub: hubinfo create: /DOMAIN/TunnelB 10.x.x.x/48002 0x7fe138036f20 Mar 30 15:21:34:771 [140605367465728] 2 hub: ch_hublist /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:21:34:771 [140605367465728] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:21:34:771 [140605367465728] 0 hub: CORE load -- (CLIENT) -- private key path load () failed. Reason=(not found) Mar 30 15:21:34:771 [140605367465728] 1 hub: (ssl_ctx_setup) ca location -> certs/client5.pem Mar 30 15:21:34:771 [140605367465728] 1 hub: (ssl_ctx_setup) certificate -> certs/client5.pem Mar 30 15:21:34:771 [140605367465728] 1 hub: (ssl_ctx_setup) private key file -> certs/client5.pem Mar 30 15:21:34:771 [140605367465728] 1 hub: CORE client<2> cipher=EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH:!RC4:!DSS Mar 30 15:21:34:771 [140605367465728] 1 hub: Could not open DH file 'certs/dh0.pem' Mar 30 15:21:34:771 [140605367465728] 1 hub: Could not open DH file 'certs/dh0.pem' Mar 30 15:21:34:771 [140605367465728] 3 hub: Decode Certificate - certificate certs/client5.pem expires 1898810071 Mar 30 15:21:34:772 [140605367465728] 1 hub: tunnel_port_next - Tunnel-002-name[10.x.x.x] received port 48009 from controller Mar 30 15:21:34:772 [140605367465728] 1 hub: CORE added SESSCTRL for 10.x.x.x/48003 Mar 30 15:21:34:772 [140605367465728] 1 hub: CORE - setup complete, starting to work (tunnel_version: 7) Mar 30 15:21:34:872 [140605367465728] 1 hub: CORE started managed thread 0 [140603740354304] Mar 30 15:21:34:872 [140605367465728] 1 hub: CORE started managed thread 1 [140603731961600] Mar 30 15:21:34:872 [140605367465728] 1 hub: CORE started managed thread 2 [140603723568896] Mar 30 15:21:34:872 [140603723568896] 1 hub: CTRL TunnelB address resolved (10.x.x.x:48003) Mar 30 15:21:34:872 [140603723568896] 5 hub: ciOpen - cache path: /opt/nimsoft/niscache Mar 30 15:21:34:872 [140603723568896] 2 hub: ciOpen - initializing global CI cache Mar 30 15:21:34:872 [140603723568896] 2 hub: ciSave - saving CI [CAE44DABA3450C4938A185989F7E82767] Mar 30 15:21:34:873 [140603723568896] 2 hub: ciSaveMetric - saving MET [M1631C24C459DB8371A453D92171065EF] 10.1.2:1 Mar 30 15:21:34:873 [140603731961600] 1 hub: CTRL SecondaryHA address resolved (10.x.x.x:48003) Mar 30 15:21:34:873 [140603731961600] 5 hub: ciOpen - cache path: /opt/nimsoft/niscache Mar 30 15:21:34:873 [140603740354304] 1 hub: CTRL Secondary address resolved (10.x.x.x:48003) Mar 30 15:21:34:873 [140603731961600] 2 hub: ciSave - saving CI [C419D34EB5E354FCDFF9AB06BB7FE7F81] Mar 30 15:21:34:873 [140603723568896] 2 hub: internal alarm - The tunnel certificate for TunnelB (10.x.x.x) will expire in 3625 days., 0, certificate/certs/client5.pem Mar 30 15:21:34:873 [140603723568896] 4 hub: ciClose - [CAE44DABA3450C4938A185989F7E82767] Mar 30 15:21:34:873 [140603740354304] 5 hub: ciOpen - cache path: /opt/nimsoft/niscache Mar 30 15:21:34:873 [140603740354304] 2 hub: ciSave - saving CI [CDDE01B3B12E19C9B6AA88D26A4EC6977] Mar 30 15:21:34:873 [140603731961600] 2 hub: ciSaveMetric - saving MET [MA42859F0620876ECA34130ABF26569F9] 10.1.2:1 Mar 30 15:21:34:873 [140603740354304] 2 hub: ciSaveMetric - saving MET [MAF0D676769C9AD7890B786C74BB7A12A] 10.1.2:1 Mar 30 15:21:34:873 [140603723568896] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=17: before/connect initialization Mar 30 15:21:34:873 [140603731961600] 2 hub: internal alarm - The tunnel certificate for SecondaryHA (10.x.x.x) will expire in 3626 days., 0, certificate/certs/client4.pem Mar 30 15:21:34:873 [140603731961600] 4 hub: ciClose - [C419D34EB5E354FCDFF9AB06BB7FE7F81] Mar 30 15:21:34:873 [140603723568896] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:34:873 [140603723568896] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:34:874 [140603740354304] 2 hub: internal alarm - The tunnel certificate for Secondary (10.x.x.x) will expire in 3626 days., 0, certificate/certs/client3.pem Mar 30 15:21:34:874 [140603740354304] 4 hub: ciClose - [CDDE01B3B12E19C9B6AA88D26A4EC6977] Mar 30 15:21:34:874 [140603731961600] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=15: before/connect initialization Mar 30 15:21:34:874 [140603731961600] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:34:874 [140603740354304] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=16: before/connect initialization Mar 30 15:21:34:874 [140603731961600] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:34:874 [140603740354304] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:34:874 [140603740354304] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:34:876 [140603731961600] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:34:876 [140603731961600] 3 hub: SSL state (connect): SSLv3 read server certificate A Mar 30 15:21:34:876 [140603731961600] 3 hub: SSL state (connect): SSLv3 read server key exchange A Mar 30 15:21:34:876 [140603731961600] 3 hub: SSL state (connect): SSLv3 read server certificate request A Mar 30 15:21:34:876 [140603731961600] 3 hub: SSL state (connect): SSLv3 read server done A Mar 30 15:21:34:877 [140603731961600] 3 hub: SSL state (connect): SSLv3 write client certificate A Mar 30 15:21:34:877 [140603731961600] 3 hub: SSL state (connect): SSLv3 write client key exchange A Mar 30 15:21:34:877 [140603731961600] 3 hub: SSL state (connect): SSLv3 write certificate verify A Mar 30 15:21:34:877 [140603731961600] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:34:877 [140603731961600] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:34:877 [140603731961600] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:34:878 [140603731961600] 3 hub: SSL state (connect): SSLv3 read server session ticket A Mar 30 15:21:34:878 [140603731961600] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:34:878 [140603731961600] 3 hub: SSL handshake done fd=15: SSL negotiation finished successfully Mar 30 15:21:34:878 [140603731961600] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:34:878 [140603731961600] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:34:878 [140603731961600] 3 hub: TSESS-A-1 connected to 10.x.x.x on socket 15 Mar 30 15:21:34:878 [140603731961600] 3 hub: CTRL SecondaryHA SSL_SESSION() created for TSESS-A-1 Mar 30 15:21:34:878 [140603731961600] 1 hub: SSL cert CN=10.x.x.x Mar 30 15:21:34:878 [140603731961600] 1 hub: SSL using cipher: ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(256) Mac=AEAD Mar 30 15:21:34:879 [140603731961600] 1 hub: CTRL other side version: 7 Mar 30 15:21:34:879 [140603731961600] 1 hub: CTRL SecondaryHA connected to 10.x.x.x/48003 Mar 30 15:21:34:879 [140603731961600] 4 hub: TSESS-P-1 releasing resources Mar 30 15:21:34:879 [140603731961600] 4 hub: SSL_shutdown on TSESS-P-1: CLOSING SSL connection Mar 30 15:21:34:879 [140603731961600] 3 hub: SSL alert (write): warning: close notify Mar 30 15:21:34:880 [140603731961600] 3 hub: SSL alert (read): warning: close notify Mar 30 15:21:34:880 [140603731961600] 4 hub: SSL_shutdown on TSESS-P-1: CLOSED Mar 30 15:21:34:880 [140603731961600] 3 hub: TSESS-P-1 [] SSL_shutdown succeeded (acknowledged) Mar 30 15:21:34:880 [140603731961600] 4 hub: TSESS-P-1 [] resources released Mar 30 15:21:34:880 [140603731961600] 1 hub: CTRL SecondaryHA is starting with tunnel id HE1543014603 Mar 30 15:21:34:880 [140603731961600] 2 hub: nimSessionServerStrict - host 127.0.0.1, port = 48008 Mar 30 15:21:34:880 [140603731961600] 2 hub: sockServer - 0x7fe0d40015d0:127.0.0.1/48008:fd=15 Mar 30 15:21:34:880 [140603731961600] 1 hub: CTRL SecondaryHA NIMSESS port=48008 Mar 30 15:21:34:880 [140603731961600] 1 hub: hub_send_event_up - /DOMAIN/SecondaryHA Mar 30 15:21:34:880 [140603731961600] 2 hub: ch_hublist /DOMAIN/SecondaryHA/ROBOT_NAME/hub Mar 30 15:21:34:880 [140603731961600] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:21:34:880 [140603731961600] 0 hub: CTRL SecondaryHA added /DOMAIN/SecondaryHA 127.0.0.1/48008 Mar 30 15:21:34:880 [140603731961600] 1 hub: CTRL SecondaryHA send hubup now Mar 30 15:21:34:880 [140603731961600] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=18: before/connect initialization Mar 30 15:21:34:880 [140603731961600] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:34:880 [140603731961600] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:34:890 [140603731961600] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:34:890 [140603731961600] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:34:890 [140603731961600] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:34:890 [140603731961600] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:34:890 [140603731961600] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:34:890 [140603731961600] 3 hub: SSL handshake done fd=18: SSL negotiation finished successfully Mar 30 15:21:34:890 [140603731961600] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:34:890 [140603731961600] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:34:890 [140603731961600] 3 hub: TSESS-A-3 connected to 10.x.x.x on socket 18 Mar 30 15:21:34:890 [140603731961600] 3 hub: CTRL SecondaryHA SSL_SESSION() reused for TSESS-P-3 Mar 30 15:21:34:931 [140603731961600] 3 hub: CTRL SecondaryHA new active TSESS-A-0 added Mar 30 15:21:34:931 [140603731961600] 3 hub: CTRL SecondaryHA added new active session (v7) (0 running, 1 available) Mar 30 15:21:34:931 [140603731961600] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=19: before/connect initialization Mar 30 15:21:34:931 [140603731961600] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:34:932 [140603731961600] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:34:942 [140603731961600] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:34:942 [140603731961600] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:34:942 [140603731961600] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:34:942 [140603731961600] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:34:942 [140603731961600] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:34:942 [140603731961600] 3 hub: SSL handshake done fd=19: SSL negotiation finished successfully Mar 30 15:21:34:942 [140603731961600] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:34:942 [140603731961600] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:34:942 [140603731961600] 3 hub: TSESS-P-4 connected to 10.x.x.x on socket 19 Mar 30 15:21:34:942 [140603731961600] 3 hub: CTRL SecondaryHA SSL_SESSION() reused for TSESS-P-4 Mar 30 15:21:34:947 [140603723568896] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:34:948 [140603723568896] 3 hub: SSL state (connect): SSLv3 read server certificate A Mar 30 15:21:34:948 [140603723568896] 3 hub: SSL state (connect): SSLv3 read server key exchange A Mar 30 15:21:34:948 [140603723568896] 3 hub: SSL state (connect): SSLv3 read server certificate request A Mar 30 15:21:34:948 [140603723568896] 3 hub: SSL state (connect): SSLv3 read server done A Mar 30 15:21:34:948 [140603723568896] 3 hub: SSL state (connect): SSLv3 write client certificate A Mar 30 15:21:34:948 [140603723568896] 3 hub: SSL state (connect): SSLv3 write client key exchange A Mar 30 15:21:34:948 [140603723568896] 3 hub: SSL state (connect): SSLv3 write certificate verify A Mar 30 15:21:34:948 [140603723568896] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:34:948 [140603723568896] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:34:948 [140603723568896] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:34:950 [140603723568896] 3 hub: SSL state (connect): SSLv3 read server session ticket A Mar 30 15:21:34:950 [140603723568896] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:34:950 [140603723568896] 3 hub: SSL handshake done fd=17: SSL negotiation finished successfully Mar 30 15:21:34:950 [140603723568896] 3 hub: (x509_verify_cert_subjectAltName) - extracted subjectAltName (localhost) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:34:950 [140603723568896] 3 hub: (x509_verify_cert_subjectAltName) - using regex matching... Mar 30 15:21:34:950 [140603723568896] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:34:950 [140603723568896] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:34:950 [140603723568896] 3 hub: TSESS-A-0 connected to 10.x.x.x on socket 17 Mar 30 15:21:34:950 [140603723568896] 3 hub: CTRL TunnelB SSL_SESSION() created for TSESS-A-0 Mar 30 15:21:34:950 [140603723568896] 1 hub: SSL cert CN=10.x.x.x Mar 30 15:21:34:950 [140603723568896] 1 hub: SSL using cipher: ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESGCM(256) Mac=AEAD Mar 30 15:21:34:950 [140603723568896] 1 hub: CTRL other side version: 7 Mar 30 15:21:34:950 [140603723568896] 1 hub: CTRL TunnelB connected to 10.x.x.x/48003 Mar 30 15:21:34:950 [140603723568896] 4 hub: TSESS-P-0 releasing resources Mar 30 15:21:34:950 [140603723568896] 4 hub: SSL_shutdown on TSESS-P-0: CLOSING SSL connection Mar 30 15:21:34:950 [140603723568896] 3 hub: SSL alert (write): warning: close notify Mar 30 15:21:34:950 [140603723568896] 3 hub: SSL alert (read): warning: close notify Mar 30 15:21:34:951 [140603723568896] 4 hub: SSL_shutdown on TSESS-P-0: CLOSED Mar 30 15:21:34:951 [140603723568896] 3 hub: TSESS-P-0 [] SSL_shutdown succeeded (acknowledged) Mar 30 15:21:34:951 [140603723568896] 4 hub: TSESS-P-0 [] resources released Mar 30 15:21:34:951 [140603723568896] 1 hub: CTRL TunnelB is starting with tunnel id GZ658429467 Mar 30 15:21:34:951 [140603723568896] 2 hub: nimSessionServerStrict - host 127.0.0.1, port = 48009 Mar 30 15:21:34:951 [140603723568896] 2 hub: sockServer - 0x7fe0e00026b0:127.0.0.1/48009:fd=17 Mar 30 15:21:34:951 [140603723568896] 1 hub: CTRL TunnelB NIMSESS port=48009 Mar 30 15:21:34:951 [140603723568896] 1 hub: hub_send_event_up - /DOMAIN/TunnelB Mar 30 15:21:34:951 [140603723568896] 2 hub: ch_hublist /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:21:34:951 [140603723568896] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:21:34:951 [140603723568896] 0 hub: CTRL TunnelB added /DOMAIN/TunnelB 127.0.0.1/48009 Mar 30 15:21:34:951 [140603723568896] 1 hub: CTRL TunnelB send hubup now Mar 30 15:21:34:951 [140603723568896] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=20: before/connect initialization Mar 30 15:21:34:951 [140603723568896] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:34:951 [140603723568896] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:34:952 [140603740354304] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:34:953 [140603740354304] 3 hub: SSL state (connect): SSLv3 read server certificate A Mar 30 15:21:34:953 [140603740354304] 3 hub: SSL state (connect): SSLv3 read server key exchange A Mar 30 15:21:34:953 [140603740354304] 3 hub: SSL state (connect): SSLv3 read server certificate request A Mar 30 15:21:34:953 [140603740354304] 3 hub: SSL state (connect): SSLv3 read server done A Mar 30 15:21:34:953 [140603740354304] 3 hub: SSL state (connect): SSLv3 write client certificate A Mar 30 15:21:34:953 [140603740354304] 3 hub: SSL state (connect): SSLv3 write client key exchange A Mar 30 15:21:34:953 [140603740354304] 3 hub: SSL state (connect): SSLv3 write certificate verify A Mar 30 15:21:34:953 [140603740354304] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:34:953 [140603740354304] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:34:953 [140603740354304] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:34:955 [140603740354304] 3 hub: SSL state (connect): SSLv3 read server session ticket A Mar 30 15:21:34:955 [140603740354304] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:34:955 [140603740354304] 3 hub: SSL handshake done fd=16: SSL negotiation finished successfully Mar 30 15:21:34:955 [140603740354304] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:34:955 [140603740354304] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:34:955 [140603740354304] 3 hub: TSESS-A-2 connected to 10.x.x.x on socket 16 Mar 30 15:21:34:955 [140603740354304] 3 hub: CTRL Secondary SSL_SESSION() created for TSESS-A-2 Mar 30 15:21:34:955 [140603740354304] 1 hub: SSL cert CN=10.x.x.x Mar 30 15:21:34:955 [140603740354304] 1 hub: SSL using cipher: ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(256) Mac=AEAD Mar 30 15:21:34:956 [140603740354304] 1 hub: CTRL other side version: 7 Mar 30 15:21:34:956 [140603740354304] 1 hub: CTRL Secondary connected to 10.x.x.x/48003 Mar 30 15:21:34:956 [140603740354304] 4 hub: TSESS-P-2 releasing resources Mar 30 15:21:34:956 [140603740354304] 4 hub: SSL_shutdown on TSESS-P-2: CLOSING SSL connection Mar 30 15:21:34:956 [140603740354304] 3 hub: SSL alert (write): warning: close notify Mar 30 15:21:34:956 [140603740354304] 3 hub: SSL alert (read): warning: close notify Mar 30 15:21:34:956 [140603740354304] 4 hub: SSL_shutdown on TSESS-P-2: CLOSED Mar 30 15:21:34:956 [140603740354304] 3 hub: TSESS-P-2 [] SSL_shutdown succeeded (acknowledged) Mar 30 15:21:34:956 [140603740354304] 4 hub: TSESS-P-2 [] resources released Mar 30 15:21:34:956 [140603740354304] 1 hub: CTRL Secondary is starting with tunnel id UO3688742191 Mar 30 15:21:34:956 [140603740354304] 2 hub: nimSessionServerStrict - host 127.0.0.1, port = 48007 Mar 30 15:21:34:957 [140603740354304] 2 hub: sockServer - 0x7fe0cc0015d0:127.0.0.1/48007:fd=16 Mar 30 15:21:34:957 [140603740354304] 1 hub: CTRL Secondary NIMSESS port=48007 Mar 30 15:21:34:957 [140603740354304] 1 hub: hub_send_event_up - /DOMAIN/Secondary Mar 30 15:21:34:957 [140603740354304] 2 hub: ch_hublist /DOMAIN/Secondary/ROBOT_NAME/hub Mar 30 15:21:34:957 [140603740354304] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:21:34:957 [140603740354304] 0 hub: CTRL Secondary added /DOMAIN/Secondary 127.0.0.1/48007 Mar 30 15:21:34:957 [140603740354304] 1 hub: CTRL Secondary send hubup now Mar 30 15:21:34:957 [140603740354304] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=21: before/connect initialization Mar 30 15:21:34:957 [140603740354304] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:34:957 [140603740354304] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:34:962 [140603723568896] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:34:962 [140603723568896] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:34:962 [140603723568896] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:34:962 [140603723568896] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:34:962 [140603723568896] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:34:962 [140603723568896] 3 hub: SSL handshake done fd=20: SSL negotiation finished successfully Mar 30 15:21:34:962 [140603723568896] 3 hub: (x509_verify_cert_subjectAltName) - extracted subjectAltName (localhost) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:34:962 [140603723568896] 3 hub: (x509_verify_cert_subjectAltName) - using regex matching... Mar 30 15:21:34:962 [140603723568896] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:34:962 [140603723568896] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:34:962 [140603723568896] 3 hub: TSESS-A-5 connected to 10.x.x.x on socket 20 Mar 30 15:21:34:962 [140603723568896] 3 hub: CTRL TunnelB SSL_SESSION() reused for TSESS-P-5 Mar 30 15:21:34:967 [140603740354304] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:34:967 [140603740354304] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:34:967 [140603740354304] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:34:967 [140603740354304] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:34:967 [140603740354304] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:34:967 [140603740354304] 3 hub: SSL handshake done fd=21: SSL negotiation finished successfully Mar 30 15:21:34:967 [140603740354304] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:34:967 [140603740354304] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:34:967 [140603740354304] 3 hub: TSESS-A-6 connected to 10.x.x.x on socket 21 Mar 30 15:21:34:967 [140603740354304] 3 hub: CTRL Secondary SSL_SESSION() reused for TSESS-P-6 Mar 30 15:21:34:983 [140603731961600] 3 hub: CTRL SecondaryHA new passive TSESS-P-1 added Mar 30 15:21:34:983 [140603731961600] 3 hub: CTRL SecondaryHA added new passive session (0 running, 1 available) Mar 30 15:21:35:004 [140603723568896] 3 hub: CTRL TunnelB new active TSESS-A-0 added Mar 30 15:21:35:004 [140603723568896] 3 hub: CTRL TunnelB added new active session (v7) (0 running, 1 available) Mar 30 15:21:35:006 [140603723568896] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=22: before/connect initialization Mar 30 15:21:35:006 [140603723568896] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:35:006 [140603723568896] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:35:008 [140603740354304] 3 hub: CTRL Secondary new active TSESS-A-0 added Mar 30 15:21:35:008 [140603740354304] 3 hub: CTRL Secondary added new active session (v7) (0 running, 1 available) Mar 30 15:21:35:008 [140603740354304] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=23: before/connect initialization Mar 30 15:21:35:008 [140603740354304] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:35:008 [140603740354304] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:35:014 [140603723568896] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:35:014 [140603723568896] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:35:014 [140603723568896] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:35:014 [140603723568896] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:35:014 [140603723568896] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:35:014 [140603723568896] 3 hub: SSL handshake done fd=22: SSL negotiation finished successfully Mar 30 15:21:35:014 [140603723568896] 3 hub: (x509_verify_cert_subjectAltName) - extracted subjectAltName (localhost) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:35:014 [140603723568896] 3 hub: (x509_verify_cert_subjectAltName) - using regex matching... Mar 30 15:21:35:014 [140603723568896] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:35:014 [140603723568896] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:35:014 [140603723568896] 3 hub: TSESS-P-7 connected to 10.x.x.x on socket 22 Mar 30 15:21:35:014 [140603723568896] 3 hub: CTRL TunnelB SSL_SESSION() reused for TSESS-P-7 Mar 30 15:21:35:018 [140603740354304] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:35:018 [140603740354304] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:35:018 [140603740354304] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:35:018 [140603740354304] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:35:018 [140603740354304] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:35:018 [140603740354304] 3 hub: SSL handshake done fd=23: SSL negotiation finished successfully Mar 30 15:21:35:018 [140603740354304] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:35:018 [140603740354304] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:35:018 [140603740354304] 3 hub: TSESS-P-8 connected to 10.x.x.x on socket 23 Mar 30 15:21:35:018 [140603740354304] 3 hub: CTRL Secondary SSL_SESSION() reused for TSESS-P-8 Mar 30 15:21:35:054 [140603723568896] 3 hub: CTRL TunnelB new passive TSESS-P-1 added Mar 30 15:21:35:054 [140603723568896] 3 hub: CTRL TunnelB added new passive session (0 running, 1 available) Mar 30 15:21:35:059 [140603740354304] 3 hub: CTRL Secondary new passive TSESS-P-1 added Mar 30 15:21:35:059 [140603740354304] 3 hub: CTRL Secondary added new passive session (0 running, 1 available) Mar 30 15:21:35:234 [140603731961600] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=24: before/connect initialization Mar 30 15:21:35:235 [140603731961600] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:35:235 [140603731961600] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:35:305 [140603723568896] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=25: before/connect initialization Mar 30 15:21:35:305 [140603723568896] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:35:306 [140603723568896] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:35:324 [140603731961600] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:35:324 [140603731961600] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:35:324 [140603731961600] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:35:324 [140603731961600] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:35:324 [140603731961600] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:35:324 [140603731961600] 3 hub: SSL handshake done fd=24: SSL negotiation finished successfully Mar 30 15:21:35:324 [140603731961600] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:35:324 [140603731961600] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:35:324 [140603731961600] 3 hub: TSESS-A-9 connected to 10.x.x.x on socket 24 Mar 30 15:21:35:324 [140603731961600] 3 hub: CTRL SecondaryHA SSL_SESSION() reused for TSESS-P-9 Mar 30 15:21:35:325 [140603731961600] 3 hub: CTRL SecondaryHA new active TSESS-A-2 added Mar 30 15:21:35:325 [140603731961600] 3 hub: CTRL SecondaryHA added new active session (v7) (0 running, 2 available) Mar 30 15:21:35:325 [140603731961600] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=26: before/connect initialization Mar 30 15:21:35:325 [140603731961600] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:35:325 [140603731961600] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:35:335 [140603731961600] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:35:335 [140603731961600] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:35:335 [140603731961600] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:35:335 [140603731961600] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:35:335 [140603731961600] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:35:335 [140603731961600] 3 hub: SSL handshake done fd=26: SSL negotiation finished successfully Mar 30 15:21:35:335 [140603731961600] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:35:335 [140603731961600] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:35:335 [140603731961600] 3 hub: TSESS-P-11 connected to 10.x.x.x on socket 26 Mar 30 15:21:35:335 [140603731961600] 3 hub: CTRL SecondaryHA SSL_SESSION() reused for TSESS-P-11 Mar 30 15:21:35:360 [140603740354304] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=27: before/connect initialization Mar 30 15:21:35:360 [140603740354304] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:35:360 [140603740354304] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:35:376 [140603731961600] 3 hub: CTRL SecondaryHA new passive TSESS-P-3 added Mar 30 15:21:35:376 [140603731961600] 3 hub: CTRL SecondaryHA added new passive session (0 running, 2 available) Mar 30 15:21:35:395 [140603723568896] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:35:395 [140603723568896] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:35:395 [140603723568896] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:35:395 [140603723568896] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:35:395 [140603723568896] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:35:395 [140603723568896] 3 hub: SSL handshake done fd=25: SSL negotiation finished successfully Mar 30 15:21:35:395 [140603723568896] 3 hub: (x509_verify_cert_subjectAltName) - extracted subjectAltName (localhost) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:35:395 [140603723568896] 3 hub: (x509_verify_cert_subjectAltName) - using regex matching... Mar 30 15:21:35:395 [140603723568896] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:35:395 [140603723568896] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:35:395 [140603723568896] 3 hub: TSESS-A-10 connected to 10.x.x.x on socket 25 Mar 30 15:21:35:395 [140603723568896] 3 hub: CTRL TunnelB SSL_SESSION() reused for TSESS-P-10 Mar 30 15:21:35:396 [140603723568896] 3 hub: CTRL TunnelB new active TSESS-A-2 added Mar 30 15:21:35:396 [140603723568896] 3 hub: CTRL TunnelB added new active session (v7) (0 running, 2 available) Mar 30 15:21:35:396 [140603723568896] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=28: before/connect initialization Mar 30 15:21:35:396 [140603723568896] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:35:396 [140603723568896] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:35:400 [140603740354304] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:35:400 [140603740354304] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:35:400 [140603740354304] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:35:400 [140603740354304] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:35:400 [140603740354304] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:35:400 [140603740354304] 3 hub: SSL handshake done fd=27: SSL negotiation finished successfully Mar 30 15:21:35:400 [140603740354304] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:35:400 [140603740354304] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:35:400 [140603740354304] 3 hub: TSESS-A-12 connected to 10.x.x.x on socket 27 Mar 30 15:21:35:400 [140603740354304] 3 hub: CTRL Secondary SSL_SESSION() reused for TSESS-P-12 Mar 30 15:21:35:406 [140603723568896] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:35:406 [140603723568896] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:35:406 [140603723568896] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:35:406 [140603723568896] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:35:406 [140603723568896] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:35:406 [140603723568896] 3 hub: SSL handshake done fd=28: SSL negotiation finished successfully Mar 30 15:21:35:406 [140603723568896] 3 hub: (x509_verify_cert_subjectAltName) - extracted subjectAltName (localhost) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:35:406 [140603723568896] 3 hub: (x509_verify_cert_subjectAltName) - using regex matching... Mar 30 15:21:35:406 [140603723568896] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:35:406 [140603723568896] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:35:406 [140603723568896] 3 hub: TSESS-P-13 connected to 10.x.x.x on socket 28 Mar 30 15:21:35:406 [140603723568896] 3 hub: CTRL TunnelB SSL_SESSION() reused for TSESS-P-13 Mar 30 15:21:35:442 [140603740354304] 3 hub: CTRL Secondary new active TSESS-A-2 added Mar 30 15:21:35:442 [140603740354304] 3 hub: CTRL Secondary added new active session (v7) (0 running, 2 available) Mar 30 15:21:35:442 [140603740354304] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=29: before/connect initialization Mar 30 15:21:35:442 [140603740354304] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:35:442 [140603740354304] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:35:445 [140603723568896] 3 hub: CTRL TunnelB new passive TSESS-P-3 added Mar 30 15:21:35:445 [140603723568896] 3 hub: CTRL TunnelB added new passive session (0 running, 2 available) Mar 30 15:21:35:452 [140603740354304] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:35:452 [140603740354304] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:35:452 [140603740354304] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:35:452 [140603740354304] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:35:452 [140603740354304] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:35:452 [140603740354304] 3 hub: SSL handshake done fd=29: SSL negotiation finished successfully Mar 30 15:21:35:452 [140603740354304] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:35:452 [140603740354304] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:35:452 [140603740354304] 3 hub: TSESS-P-14 connected to 10.x.x.x on socket 29 Mar 30 15:21:35:452 [140603740354304] 3 hub: CTRL Secondary SSL_SESSION() reused for TSESS-P-14 Mar 30 15:21:35:493 [140603740354304] 3 hub: CTRL Secondary new passive TSESS-P-3 added Mar 30 15:21:35:493 [140603740354304] 3 hub: CTRL Secondary added new passive session (0 running, 2 available) Mar 30 15:21:35:677 [140603731961600] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=30: before/connect initialization Mar 30 15:21:35:677 [140603731961600] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:35:677 [140603731961600] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:35:682 [140604596016896] 3 hub: topology - main thread - worker thread pool started Mar 30 15:21:35:682 [140605241927424] 3 hub: passive robot thread - reinitializing passive robots from hub.cfg Mar 30 15:21:35:682 [140605241927424] 3 hub: passive robot thread - prepare to reinitialize passive robots by removing all passive robots from robot lists Mar 30 15:21:35:682 [140605241927424] 4 hub: cfgReader file open: hub.cfg Mar 30 15:21:35:682 [140605241927424] 4 hub: cfgReader file close: hub.cfg Mar 30 15:21:35:682 [140605241927424] 4 hub: cfgReader file open: hub.cfg Mar 30 15:21:35:683 [140605241927424] 4 hub: cfgReader file close: hub.cfg Mar 30 15:21:35:683 [140605241927424] 3 hub: robot thread configuration - add passive robots from hub.cfg to this hub's robot lists Mar 30 15:21:35:683 [140605241927424] 4 hub: cfgReader file open: hub.cfg Mar 30 15:21:35:683 [140605241927424] 4 hub: cfgReader file close: hub.cfg Mar 30 15:21:35:683 [140605241927424] 3 hub: robot thread configuration - completed adding passive robots from hub.cfg Mar 30 15:21:35:717 [140603731961600] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:35:717 [140603731961600] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:35:717 [140603731961600] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:35:717 [140603731961600] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:35:717 [140603731961600] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:35:717 [140603731961600] 3 hub: SSL handshake done fd=30: SSL negotiation finished successfully Mar 30 15:21:35:717 [140603731961600] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:35:717 [140603731961600] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:35:717 [140603731961600] 3 hub: TSESS-A-15 connected to 10.x.x.x on socket 30 Mar 30 15:21:35:717 [140603731961600] 3 hub: CTRL SecondaryHA SSL_SESSION() reused for TSESS-P-15 Mar 30 15:21:35:718 [140603731961600] 3 hub: CTRL SecondaryHA new active TSESS-A-4 added Mar 30 15:21:35:718 [140603731961600] 3 hub: CTRL SecondaryHA added new active session (v7) (0 running, 3 available) Mar 30 15:21:35:718 [140603731961600] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=31: before/connect initialization Mar 30 15:21:35:718 [140603731961600] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:35:718 [140603731961600] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:35:728 [140603731961600] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:35:728 [140603731961600] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:35:728 [140603731961600] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:35:728 [140603731961600] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:35:728 [140603731961600] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:35:728 [140603731961600] 3 hub: SSL handshake done fd=31: SSL negotiation finished successfully Mar 30 15:21:35:728 [140603731961600] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:35:728 [140603731961600] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:35:728 [140603731961600] 3 hub: TSESS-P-16 connected to 10.x.x.x on socket 31 Mar 30 15:21:35:728 [140603731961600] 3 hub: CTRL SecondaryHA SSL_SESSION() reused for TSESS-P-16 Mar 30 15:21:35:746 [140603723568896] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=32: before/connect initialization Mar 30 15:21:35:746 [140603723568896] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:35:747 [140603723568896] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:35:770 [140603731961600] 3 hub: CTRL SecondaryHA new passive TSESS-P-5 added Mar 30 15:21:35:770 [140603731961600] 3 hub: CTRL SecondaryHA added new passive session (0 running, 3 available) Mar 30 15:21:35:787 [140603723568896] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:35:787 [140603723568896] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:35:787 [140603723568896] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:35:787 [140603723568896] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:35:787 [140603723568896] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:35:787 [140603723568896] 3 hub: SSL handshake done fd=32: SSL negotiation finished successfully Mar 30 15:21:35:787 [140603723568896] 3 hub: (x509_verify_cert_subjectAltName) - extracted subjectAltName (localhost) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:35:787 [140603723568896] 3 hub: (x509_verify_cert_subjectAltName) - using regex matching... Mar 30 15:21:35:787 [140603723568896] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:35:787 [140603723568896] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:35:787 [140603723568896] 3 hub: TSESS-A-17 connected to 10.x.x.x on socket 32 Mar 30 15:21:35:787 [140603723568896] 3 hub: CTRL TunnelB SSL_SESSION() reused for TSESS-P-17 Mar 30 15:21:35:787 [140603723568896] 3 hub: CTRL TunnelB new active TSESS-A-4 added Mar 30 15:21:35:787 [140603723568896] 3 hub: CTRL TunnelB added new active session (v7) (0 running, 3 available) Mar 30 15:21:35:788 [140603723568896] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=33: before/connect initialization Mar 30 15:21:35:788 [140603723568896] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:35:788 [140603723568896] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:35:794 [140603740354304] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=34: before/connect initialization Mar 30 15:21:35:794 [140603740354304] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:35:794 [140603740354304] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:35:798 [140603723568896] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:35:798 [140603723568896] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:35:798 [140603723568896] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:35:798 [140603723568896] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:35:798 [140603723568896] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:35:798 [140603723568896] 3 hub: SSL handshake done fd=33: SSL negotiation finished successfully Mar 30 15:21:35:798 [140603723568896] 3 hub: (x509_verify_cert_subjectAltName) - extracted subjectAltName (localhost) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:35:798 [140603723568896] 3 hub: (x509_verify_cert_subjectAltName) - using regex matching... Mar 30 15:21:35:798 [140603723568896] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:35:798 [140603723568896] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:35:798 [140603723568896] 3 hub: TSESS-P-18 connected to 10.x.x.x on socket 33 Mar 30 15:21:35:798 [140603723568896] 3 hub: CTRL TunnelB SSL_SESSION() reused for TSESS-P-18 Mar 30 15:21:35:834 [140603740354304] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:35:834 [140603740354304] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:35:834 [140603740354304] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:35:834 [140603740354304] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:35:834 [140603740354304] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:35:834 [140603740354304] 3 hub: SSL handshake done fd=34: SSL negotiation finished successfully Mar 30 15:21:35:834 [140603740354304] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:35:834 [140603740354304] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:35:834 [140603740354304] 3 hub: TSESS-A-19 connected to 10.x.x.x on socket 34 Mar 30 15:21:35:834 [140603740354304] 3 hub: CTRL Secondary SSL_SESSION() reused for TSESS-P-19 Mar 30 15:21:35:835 [140603740354304] 3 hub: CTRL Secondary new active TSESS-A-4 added Mar 30 15:21:35:835 [140603740354304] 3 hub: CTRL Secondary added new active session (v7) (0 running, 3 available) Mar 30 15:21:35:835 [140603740354304] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=35: before/connect initialization Mar 30 15:21:35:835 [140603740354304] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:21:35:835 [140603740354304] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:21:35:837 [140603723568896] 3 hub: CTRL TunnelB new passive TSESS-P-5 added Mar 30 15:21:35:837 [140603723568896] 3 hub: CTRL TunnelB added new passive session (0 running, 3 available) Mar 30 15:21:35:845 [140603740354304] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:21:35:845 [140603740354304] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:21:35:845 [140603740354304] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:21:35:845 [140603740354304] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:21:35:845 [140603740354304] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:21:35:845 [140603740354304] 3 hub: SSL handshake done fd=35: SSL negotiation finished successfully Mar 30 15:21:35:845 [140603740354304] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:21:35:845 [140603740354304] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:21:35:845 [140603740354304] 3 hub: TSESS-P-20 connected to 10.x.x.x on socket 35 Mar 30 15:21:35:845 [140603740354304] 3 hub: CTRL Secondary SSL_SESSION() reused for TSESS-P-20 Mar 30 15:21:35:887 [140603740354304] 3 hub: CTRL Secondary new passive TSESS-P-5 added Mar 30 15:21:35:887 [140603740354304] 3 hub: CTRL Secondary added new passive session (0 running, 3 available) Mar 30 15:21:36:683 [140605547460672] 4 hub: medium timeout Mar 30 15:21:36:684 [140605007062784] 1 hub: (Metric Plugin): Created 0 QOS_BASELINE messages to send. Mar 30 15:21:36:774 [140605547460672] 3 hub: robotup - ROBOT_NAME (10.x.x.x) is a valid controller Mar 30 15:21:36:774 [140605547460672] 1 hub: robotup - ROBOT_NAME 10.x.x.x 48000 (reachable via 127.0.0.1 48000) [9.20SHF9 [Build 9.20SHF9.290, Feb 5 2020]] Mar 30 15:21:36:774 [140605547460672] 2 hub: ch_robotlist /DOMAIN/TunnelBHA/ROBOT_NAME 0 Mar 30 15:21:36:774 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:21:36:774 [140605547460672] 2 hub: nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME 10.x.x.x/48000 Mar 30 15:21:36:777 [140605547460672] 2 hub: login from ctrl 127.0.0.1/52872 Mar 30 15:21:36:777 [140605547460672] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:21:36:777 [140605547460672] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:21:36:777 [140605547460672] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:21:36:777 [140605547460672] 3 hub: IsIPInList ip=10.x.x.x, list=|||10.x.x.x|, found=1 Mar 30 15:21:36:777 [140605547460672] 1 hub: login - probe=controller permissions=read ip=10.x.x.x Mar 30 15:21:36:777 [140605547460672] 1 hub: Login: succeeded for controller, ip = 10.x.x.x Mar 30 15:21:36:777 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:21:36:778 [140605547460672] 3 hub: probelist - ROBOT_NAME (127.0.0.1) is a valid controller Mar 30 15:21:36:778 [140605547460672] 2 hub: nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME 127.0.0.1/48000 Mar 30 15:21:36:778 [140605547460672] 2 hub: internal alarm - Robot ROBOT_NAME is running, 0, ROBOT_NAME Mar 30 15:21:36:778 [140605547460672] 3 hub: robot probelist update - ROBOT_NAME 10.x.x.x - status change with robot or probes detected, last_change=1585596096, status=0 Mar 30 15:21:36:778 [140605547460672] 3 hub: robot probelist update - ROBOT_NAME 10.x.x.x - robot configuration file reread detected, last_config_reread=1585596094 Mar 30 15:21:36:778 [140605547460672] 3 hub: robot probelist update - ROBOT_NAME 10.x.x.x - send robot change event Mar 30 15:21:36:778 [140605547460672] 2 hub: ch_probe /DOMAIN/TunnelBHA/ROBOT_NAME Mar 30 15:21:36:778 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:21:36:778 [140605547460672] 1 hub: robot probelist update - probelist - ROBOT_NAME 127.0.0.1 (s:5326) Mar 30 15:21:36:778 [140605547460672] 2 hub: nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/controller 127.0.0.1/48000 Mar 30 15:21:36:778 [140605547460672] 2 hub: probe port added: ROBOT_NAME controller 10.x.x.x 48000 Mar 30 15:21:36:778 [140605547460672] 2 hub: nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:21:36:778 [140605547460672] 2 hub: probe port added: ROBOT_NAME hub 10.x.x.x 48002 Mar 30 15:21:36:778 [140605547460672] 2 hub: nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/spooler 127.0.0.1/48001 Mar 30 15:21:36:778 [140605547460672] 2 hub: probe port added: ROBOT_NAME spooler 10.x.x.x 48001 Mar 30 15:21:36:778 [140605547460672] 3 hub: check_if_primary...: result = 1 Mar 30 15:21:36:937 [140605547460672] 2 hub: verify_login from ctrl 127.0.0.1/52880 Mar 30 15:21:36:937 [140605547460672] 1 hub: verify - [target] prid=controller cmd=port_list ip=10.x.x.x perm=1 Mar 30 15:21:36:937 [140605547460672] 1 hub: verify - filter access denied ip=10.x.x.x Mar 30 15:21:36:938 [140605547460672] 2 hub: login from ctrl 127.0.0.1/52886 Mar 30 15:21:36:938 [140605547460672] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:21:36:938 [140605547460672] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:21:36:938 [140605547460672] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:21:36:938 [140605547460672] 3 hub: IsIPInList ip=10.x.x.x, list=|||10.x.x.x|, found=1 Mar 30 15:21:36:938 [140605547460672] 1 hub: login - probe=distsrv permissions=admin ip=10.x.x.x Mar 30 15:21:36:938 [140605547460672] 1 hub: Login: succeeded for distsrv, ip = 10.x.x.x Mar 30 15:21:36:938 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:21:36:939 [140605547460672] 2 hub: verify_login from ctrl 127.0.0.1/52888 Mar 30 15:21:36:939 [140605547460672] 1 hub: verify - [target] prid=controller cmd=port_list ip=10.x.x.x perm=1 Mar 30 15:21:36:939 [140605547460672] 1 hub: verify - [source] id=distsrv ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:39:040 [140605547460672] 2 hub: verify_login from ctrl 127.0.0.1/52908 Mar 30 15:21:39:040 [140605547460672] 1 hub: verify - [target] prid=controller cmd=probe_list ip=10.x.x.x perm=1 Mar 30 15:21:39:040 [140605547460672] 1 hub: verify - filter access denied ip=10.x.x.x Mar 30 15:21:39:041 [140605547460672] 2 hub: login from ctrl 127.0.0.1/52912 Mar 30 15:21:39:041 [140605547460672] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:21:39:041 [140605547460672] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:21:39:041 [140605547460672] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:21:39:041 [140605547460672] 3 hub: IsIPInList ip=10.x.x.x, list=|||10.x.x.x|, found=1 Mar 30 15:21:39:041 [140605547460672] 1 hub: login - probe=cdm permissions=write ip=10.x.x.x Mar 30 15:21:39:041 [140605547460672] 1 hub: Login: succeeded for cdm, ip = 10.x.x.x Mar 30 15:21:39:041 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:21:39:041 [140605547460672] 2 hub: verify_login from ctrl 127.0.0.1/52914 Mar 30 15:21:39:041 [140605547460672] 1 hub: verify - [target] prid=controller cmd=probe_list ip=10.x.x.x perm=1 Mar 30 15:21:39:041 [140605547460672] 1 hub: verify - [source] id=cdm ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:39:050 [140604570838784] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:21:39:050 [140604570838784] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:21:39:050 [140604570838784] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:21:39:050 [140604570838784] 5 hub: (Metric Plugin): cached QOS_DEFINITION cdm QOS_PROC_QUEUE_LEN (Processes procs) Mar 30 15:21:39:050 [140604570838784] 5 hub: (Metric Plugin): Processing pipeline finished, 1 messages output Mar 30 15:21:39:050 [140604570838784] 3 hub: nimSession:0x7fe114001fd0:11:127.0.0.1/41438:127.0.0.1/41438 - needs cross-thread locking Mar 30 15:21:39:051 [140604998670080] 5 hub: (Metric Plugin): cached QOS_DEFINITION cdm QOS_MEMORY_PERC_USAGE (Percent %) Mar 30 15:21:39:051 [140604998670080] 5 hub: (Metric Plugin): Processing pipeline finished, 1 messages output Mar 30 15:21:39:051 [140604998670080] 3 hub: nimSession:0x7fe114001fd0:11:127.0.0.1/41440:127.0.0.1/41440 - needs cross-thread locking Mar 30 15:21:39:051 [140605015455488] 5 hub: (Metric Plugin): cached QOS_DEFINITION cdm QOS_DISK_USAGE_PERC (Percent %) Mar 30 15:21:39:051 [140605015455488] 5 hub: (Metric Plugin): Processing pipeline finished, 1 messages output Mar 30 15:21:39:051 [140605015455488] 3 hub: nimSession:0x7fe114001fd0:11:127.0.0.1/41442:127.0.0.1/41442 - needs cross-thread locking Mar 30 15:21:39:052 [140604965099264] 5 hub: (Metric Plugin): cached QOS_DEFINITION cdm QOS_CPU_USAGE (Percent %) Mar 30 15:21:39:052 [140604965099264] 5 hub: (Metric Plugin): Processing pipeline finished, 1 messages output Mar 30 15:21:39:052 [140604965099264] 3 hub: nimSession:0x7fe114001fd0:11:127.0.0.1/41444:127.0.0.1/41444 - needs cross-thread locking Mar 30 15:21:40:064 [140604612802304] 5 hub: (Metric Plugin): Processing pipeline finished, 1 messages output Mar 30 15:21:40:064 [140604612802304] 3 hub: nimSession:0x7fe114001fd0:11:127.0.0.1/41450:127.0.0.1/41450 - needs cross-thread locking Mar 30 15:21:40:072 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:21:40:072 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:21:40:072 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60458 Mar 30 15:21:40:072 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60458' delegated to TSESS-A-5 [0000] Mar 30 15:21:40:073 [140603715176192] 3 hub: TSESS-A-5-0 [TunnelB] started (local) in tid=3665774336 Mar 30 15:21:40:078 [140603715176192] 5 hub: TSESS-A-5-0 [TunnelB] local 100 'get_info' Mar 30 15:21:40:078 [140603715176192] 1 hub: TSESS-A-5-0 [TunnelB] sent 100 'get_info' (165 bytes, 0 ms) Mar 30 15:21:40:078 [140603715176192] 1 hub: TSESS-A-5-0 [TunnelB] ssl 299 '-' (51 bytes) Mar 30 15:21:40:078 [140603715176192] 1 hub: TSESS-A-5-0 [TunnelB] received 299 '-' (59 bytes) Mar 30 15:21:40:080 [140605547460672] 2 hub: login from ctrl 127.0.0.1/52942 Mar 30 15:21:40:080 [140605547460672] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:21:40:080 [140605547460672] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:21:40:080 [140605547460672] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:21:40:080 [140605547460672] 3 hub: IsIPInList ip=10.x.x.x, list=|||10.x.x.x|, found=1 Mar 30 15:21:40:080 [140605547460672] 1 hub: login - probe=HA permissions=admin ip=10.x.x.x Mar 30 15:21:40:080 [140605547460672] 1 hub: Login: succeeded for HA, ip = 10.x.x.x Mar 30 15:21:40:080 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:21:40:084 [140603715176192] 1 hub: TSESS-A-5-0 [TunnelB] message not routable, ending conversation Mar 30 15:21:40:084 [140603715176192] 5 hub: TSESS-A-5-0 [TunnelB] suspended Mar 30 15:21:40:133 [140604587624192] 5 hub: (Metric Plugin): Processing pipeline finished, 1 messages output Mar 30 15:21:40:133 [140604587624192] 3 hub: nimSession:0x7fe114001fd0:11:127.0.0.1/41466:127.0.0.1/41466 - needs cross-thread locking Mar 30 15:21:40:134 [140604579231488] 5 hub: (Metric Plugin): Processing pipeline finished, 1 messages output Mar 30 15:21:40:134 [140604579231488] 3 hub: nimSession:0x7fe114001fd0:11:127.0.0.1/41468:127.0.0.1/41468 - needs cross-thread locking Mar 30 15:21:40:135 [140604570838784] 5 hub: (Metric Plugin): cdm is not a configured probe, skipping Mar 30 15:21:40:135 [140604570838784] 5 hub: (Metric Plugin): Processing pipeline finished, 1 messages output Mar 30 15:21:40:135 [140604570838784] 3 hub: nimSession:0x7fe114001fd0:11:127.0.0.1/41470:127.0.0.1/41470 - needs cross-thread locking Mar 30 15:21:40:135 [140604998670080] 5 hub: (Metric Plugin): cdm is not a configured probe, skipping Mar 30 15:21:40:135 [140604998670080] 5 hub: (Metric Plugin): Processing pipeline finished, 1 messages output Mar 30 15:21:40:135 [140604998670080] 3 hub: nimSession:0x7fe114004b10:36:127.0.0.1/41472:127.0.0.1/41472 - needs cross-thread locking Mar 30 15:21:40:136 [140605015455488] 5 hub: (Metric Plugin): Processing pipeline finished, 1 messages output Mar 30 15:21:40:136 [140605015455488] 3 hub: nimSession:0x7fe114001fd0:11:127.0.0.1/41474:127.0.0.1/41474 - needs cross-thread locking Mar 30 15:21:40:136 [140604965099264] 5 hub: (Metric Plugin): Processing pipeline finished, 1 messages output Mar 30 15:21:40:136 [140604965099264] 3 hub: nimSession:0x7fe114001fd0:11:127.0.0.1/41476:127.0.0.1/41476 - needs cross-thread locking Mar 30 15:21:40:137 [140604612802304] 5 hub: (Metric Plugin): Processing pipeline finished, 1 messages output Mar 30 15:21:40:137 [140604612802304] 3 hub: nimSession:0x7fe114001fd0:11:127.0.0.1/41478:127.0.0.1/41478 - needs cross-thread locking Mar 30 15:21:40:138 [140604604409600] 5 hub: (Metric Plugin): cdm is not a configured probe, skipping Mar 30 15:21:40:138 [140604604409600] 5 hub: (Metric Plugin): Processing pipeline finished, 1 messages output Mar 30 15:21:40:138 [140604604409600] 3 hub: nimSession:0x7fe114001fd0:11:127.0.0.1/41480:127.0.0.1/41480 - needs cross-thread locking Mar 30 15:21:40:585 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60482 Mar 30 15:21:40:585 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60482' delegated to TSESS-A-5 [140603715176192] Mar 30 15:21:40:585 [140603715176192] 3 hub: TSESS-A-5-1 [TunnelB] restarted (for local) Mar 30 15:21:40:590 [140603715176192] 5 hub: TSESS-A-5-1 [TunnelB] local 100 'get_info' Mar 30 15:21:40:590 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:40:590 [140603715176192] 1 hub: TSESS-A-5-1 [TunnelB] sent 100 'get_info' (312 bytes, 0 ms) Mar 30 15:21:40:601 [140603715176192] 1 hub: TSESS-A-5-1 [TunnelB] ssl 200 '-' (1277 bytes) Mar 30 15:21:40:601 [140603715176192] 3 hub: TSESS-A-5-1 [TunnelB] is paired with [P-298-1] Mar 30 15:21:40:601 [140603715176192] 1 hub: TSESS-A-5-1 [TunnelB] received 200 '-' (1285 bytes) Mar 30 15:21:40:606 [140603715176192] 5 hub: TSESS-A-5-1 [TunnelB] local 100 '_close' Mar 30 15:21:40:606 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:40:606 [140603715176192] 1 hub: TSESS-A-5-1 [TunnelB] sent 100 '_close' (310 bytes, 0 ms) Mar 30 15:21:40:606 [140603715176192] 3 hub: TSESS-A-5-1 [TunnelB] local session closed, suspending Mar 30 15:21:40:806 [140603715176192] 5 hub: TSESS-A-5-1 [TunnelB] suspended Mar 30 15:21:41:603 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60486 Mar 30 15:21:41:603 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60486' delegated to TSESS-A-5 [140603715176192] Mar 30 15:21:41:603 [140603715176192] 3 hub: TSESS-A-5-2 [TunnelB] restarted (for local) Mar 30 15:21:41:608 [140603715176192] 5 hub: TSESS-A-5-2 [TunnelB] local 100 'get_info' Mar 30 15:21:41:608 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:41:608 [140603715176192] 1 hub: TSESS-A-5-2 [TunnelB] sent 100 'get_info' (312 bytes, 0 ms) Mar 30 15:21:41:619 [140603715176192] 1 hub: TSESS-A-5-2 [TunnelB] ssl 200 '-' (1277 bytes) Mar 30 15:21:41:619 [140603715176192] 3 hub: TSESS-A-5-2 [TunnelB] is paired with [P-298-2] Mar 30 15:21:41:619 [140603715176192] 1 hub: TSESS-A-5-2 [TunnelB] received 200 '-' (1285 bytes) Mar 30 15:21:41:620 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60488 Mar 30 15:21:41:620 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60488' delegated to TSESS-A-10 [0000] Mar 30 15:21:41:620 [140603706783488] 3 hub: TSESS-A-10-0 [TunnelB] started (local) in tid=3657381632 Mar 30 15:21:41:624 [140603715176192] 5 hub: TSESS-A-5-2 [TunnelB] local 100 '_close' Mar 30 15:21:41:624 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:41:624 [140603715176192] 1 hub: TSESS-A-5-2 [TunnelB] sent 100 '_close' (310 bytes, 0 ms) Mar 30 15:21:41:624 [140603715176192] 3 hub: TSESS-A-5-2 [TunnelB] local session closed, suspending Mar 30 15:21:41:625 [140603706783488] 5 hub: TSESS-A-10-0 [TunnelB] local 100 '_status' Mar 30 15:21:41:625 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:41:626 [140603706783488] 1 hub: TSESS-A-10-0 [TunnelB] sent 100 '_status' (312 bytes, 0 ms) Mar 30 15:21:41:636 [140603706783488] 1 hub: TSESS-A-10-0 [TunnelB] ssl 200 '-' (279 bytes) Mar 30 15:21:41:636 [140603706783488] 3 hub: TSESS-A-10-0 [TunnelB] is paired with [P-300-0] Mar 30 15:21:41:636 [140603706783488] 1 hub: TSESS-A-10-0 [TunnelB] received 200 '-' (287 bytes) Mar 30 15:21:41:636 [140604587624192] 5 hub: (Metric Plugin): Processing pipeline finished, 1 messages output Mar 30 15:21:41:637 [140604587624192] 3 hub: nimSession:0x7fe114001fd0:11:127.0.0.1/41490:127.0.0.1/41490 - needs cross-thread locking Mar 30 15:21:41:641 [140603706783488] 5 hub: TSESS-A-10-0 [TunnelB] local 100 '_close' Mar 30 15:21:41:641 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:41:641 [140603706783488] 1 hub: TSESS-A-10-0 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:21:41:641 [140603706783488] 3 hub: TSESS-A-10-0 [TunnelB] local session closed, suspending Mar 30 15:21:41:824 [140603715176192] 5 hub: TSESS-A-5-2 [TunnelB] suspended Mar 30 15:21:41:841 [140603706783488] 5 hub: TSESS-A-10-0 [TunnelB] suspended Mar 30 15:21:41:940 [140605547460672] 2 hub: verify_login from ctrl 127.0.0.1/53062 Mar 30 15:21:41:940 [140605547460672] 1 hub: verify - [target] prid=controller cmd=probe_list ip=10.x.x.x perm=1 Mar 30 15:21:41:940 [140605547460672] 1 hub: verify - filter access denied ip=10.x.x.x Mar 30 15:21:41:985 [140605547460672] 2 hub: login from ctrl 127.0.0.1/53078 Mar 30 15:21:41:985 [140605547460672] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:21:41:985 [140605547460672] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:21:41:985 [140605547460672] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:21:41:985 [140605547460672] 3 hub: IsIPInList ip=10.x.x.x, list=|||10.x.x.x|, found=1 Mar 30 15:21:41:985 [140605547460672] 1 hub: login - probe=ppm permissions=admin ip=10.x.x.x Mar 30 15:21:41:985 [140605547460672] 1 hub: Login: succeeded for ppm, ip = 10.x.x.x Mar 30 15:21:41:985 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:21:45:199 [140605547460672] 2 hub: login from ctrl 127.0.0.1/53092 Mar 30 15:21:45:199 [140605547460672] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:21:45:199 [140605547460672] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:21:45:199 [140605547460672] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:21:45:199 [140605547460672] 3 hub: IsIPInList ip=10.x.x.x, list=|||10.x.x.x|, found=1 Mar 30 15:21:45:199 [140605547460672] 1 hub: login - probe=automated_deployment_engine permissions=admin ip=10.x.x.x Mar 30 15:21:45:199 [140605547460672] 1 hub: Login: succeeded for automated_deployment_engine, ip = 10.x.x.x Mar 30 15:21:45:199 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:21:45:205 [140605547460672] 1 hub: Subscribe request received from 127.0.0.1/53098 Mar 30 15:21:45:693 [140604562446080] 1 hub: Processing new subscribe request from '127.0.0.1/53098' Mar 30 15:21:45:693 [140604562446080] 5 hub: ciOpen - cache path: /opt/nimsoft/niscache Mar 30 15:21:45:693 [140604562446080] 2 hub: ciSave - saving CI [CA8C0209FA13F39614C4B3B81A221CD39] Mar 30 15:21:45:693 [140604562446080] 2 hub: ciSaveMetric - saving MET [M4DAE4ACC6B68F91B1F3CB1BFD13E2F3A] 10.1:10 Mar 30 15:21:45:693 [140604562446080] 2 hub: internal alarm - Number of subscribers is below the threshold., 0, subs Mar 30 15:21:45:693 [140604562446080] 4 hub: ciClose - [CA8C0209FA13F39614C4B3B81A221CD39] Mar 30 15:21:45:693 [140604562446080] 3 hub: add_subscriber id=NimListenHandler sub=0x7fe0c80008c0 Mar 30 15:21:45:693 [140604562446080] 5 hub: subscription request header: Size=329 Mar 30 15:21:45:693 [140604562446080] 5 hub: frm PDS_PCH 16 127.0.0.1/53098 Mar 30 15:21:45:693 [140604562446080] 5 hub: mtype PDS_PCH 4 100 Mar 30 15:21:45:693 [140604562446080] 5 hub: charset PDS_PCH 6 UTF-8 Mar 30 15:21:45:693 [140604562446080] 5 hub: cmd PDS_PCH 10 subscribe Mar 30 15:21:45:693 [140604562446080] 5 hub: addr PDS_PCH 35 /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:21:45:693 [140604562446080] 5 hub: seq PDS_I 2 0 Mar 30 15:21:45:693 [140604562446080] 5 hub: ts PDS_I 11 1585596105 Mar 30 15:21:45:693 [140604562446080] 5 hub: sid PDS_PCH 169 /3at54YuKAXpMRs/F7Xp9AADMTU4NTYxNzcwNQFUdW5uZWxCSEEBMTcxNzU5MTI1NQExMDAuNjUuMy45NAEwOjUwOjU2OjgzOjQ4OmZlATEBYXV0b21hdGVkX2RlcGxveW1lbnRfZW5naW5lATMBfDEwMC42NS4zLjk0fAE= Mar 30 15:21:45:693 [140604562446080] 5 hub: subscription request data : Size=83 Mar 30 15:21:45:693 [140604562446080] 5 hub: subject PDS_PCH 20 AutomatedDeployment Mar 30 15:21:45:693 [140604562446080] 5 hub: id PDS_PCH 17 NimListenHandler Mar 30 15:21:45:693 [140604562446080] 5 hub: heartbeat_interval PDS_I 2 2 Mar 30 15:21:45:693 [140604562446080] 1 hub: Add subscriber to temporary queue 'AutomatedDeployment' (requested by '127.0.0.1/53098') Mar 30 15:21:45:693 [140604562446080] 3 hub: newRoute name=t_0 pr=0x7fe0c8001a50 Mar 30 15:21:45:693 [140604562446080] 2 hub: TMP route 't_0' created for subject 'AutomatedDeployment' Mar 30 15:21:45:693 [140604562446080] 5 hub: subscriber did not request a reply_timeout, using default of 180 Mar 30 15:21:45:693 [140603698390784] 2 hub: Queue management thread for 't_0' started Mar 30 15:21:45:693 [140603698390784] 1 hub: Queue 't_0' will start reading at file sequence no. 0 Mar 30 15:21:45:693 [140603698390784] 1 hub: Queue 't_0' will start writing at file sequence no. 0 Mar 30 15:21:45:693 [140604562446080] 3 hub: nimSession:0x213e830:12:127.0.0.1/53098:127.0.0.1/53098 - needs cross-thread locking Mar 30 15:21:45:693 [140604562446080] 1 hub: Subscriber 'NimListenHandler' at '127.0.0.1/53098' attached to queue 't_0' (subject:AutomatedDeployment requested bulk:0, granted bulk:0, minimum bulk:0, wait:0, heartbeat: 2, reply timeout: 180), time used: 0 ms Mar 30 15:21:45:693 [140604562446080] 2 hub: Subscriber 'NimListenHandler' attached to queue 't_0' sub=0x7fe0c80008c0 pr=0x7fe0c8001a50 nims=0x213e830 sfd=12 Mar 30 15:21:45:693 [140603698390784] 1 hub: Queue 't_0' opened file '0.sds' for writing Mar 30 15:21:45:694 [140603698390784] 1 hub: Queue 't_0' now reading from file '0.sds' (contains 0 msgs, starting @ pos 0) Mar 30 15:21:45:694 [140603689998080] 2 hub: Queue sender thread for 't_0' started Mar 30 15:21:45:696 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:21:46:206 [140605547460672] 4 hub: medium timeout Mar 30 15:21:46:206 [140605547460672] 1 hub: saving robotfile Mar 30 15:21:46:206 [140605547460672] 3 hub: robot check in - add robot to check list - added ROBOT_NAME Mar 30 15:21:46:206 [140605547460672] 1 hub: robot check in - started worker thread Mar 30 15:21:46:206 [140605547460672] 3 hub: next_check [1585596396] now [1585596106] Mar 30 15:21:46:206 [140603203516160] 2 hub: robot check in thread - started Mar 30 15:21:46:206 [140603203516160] 1 hub: robot check in - sent to ROBOT_NAME 10.x.x.x:48000, reachable at 127.0.0.1:48000 Mar 30 15:21:46:206 [140603203516160] 4 hub: CONNECT: 0x7fe0b4000e50(39) 127.0.0.1/44204->127.0.0.1/48000 Mar 30 15:21:46:207 [140603203516160] 3 hub: send request to robot - cmd='checkin', addr='/DOMAIN/TunnelBHA/ROBOT_NAME/controller', rc=0(OK) Mar 30 15:21:46:207 [140603203516160] 2 hub: robot check in thread - stopped Mar 30 15:21:46:685 [140605283890944] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:21:46:685 [140605283890944] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:21:46:685 [140605283890944] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:21:46:685 [140605283890944] 3 hub: hubhash_write - 0 hubs saved to cache Mar 30 15:21:46:685 [140605283890944] 5 hub: hublist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:21:46:685 [140605283890944] 5 hub: hublist->Secondary[+]: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:21:46:685 [140605283890944] 5 hub: hublist->Secondary[+]: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:21:46:685 [140605283890944] 5 hub: hublist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:21:46:686 [140605283890944] 4 hub: CONNECT: 0x7fe130002a70(39) 127.0.0.1/46762->127.0.0.1/48007 Mar 30 15:21:46:686 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/46762 Mar 30 15:21:46:686 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/46762' delegated to TSESS-A-6 [0000] Mar 30 15:21:46:686 [140603203516160] 3 hub: TSESS-A-6-0 [Secondary] started (local) in tid=3154114304 Mar 30 15:21:46:691 [140603203516160] 5 hub: TSESS-A-6-0 [Secondary] local 100 'hubup' Mar 30 15:21:46:691 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:46:691 [140603203516160] 1 hub: TSESS-A-6-0 [Secondary] sent 100 'hubup' (1534 bytes, 0 ms) Mar 30 15:21:46:701 [140603203516160] 1 hub: TSESS-A-6-0 [Secondary] ssl 200 '-' (453 bytes) Mar 30 15:21:46:701 [140603203516160] 3 hub: TSESS-A-6-0 [Secondary] is paired with [P-2120-0] Mar 30 15:21:46:701 [140603203516160] 1 hub: TSESS-A-6-0 [Secondary] received 200 '-' (461 bytes) Mar 30 15:21:46:701 [140605283890944] 1 hub: hubup succeeded - /DOMAIN/Secondary 10.x.x.x/48002 (reachable via 127.0.0.1:48007) Mar 30 15:21:46:701 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:21:46:701 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:21:46:701 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:21:46:701 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:21:46:701 [140605283890944] 5 hub: hubinfo_update - topology - origin UPDATED - old origin= , new origin= Secondary - addr=/DOMAIN/Secondary/ROBOT_NAME/hub Mar 30 15:21:46:701 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary CHANGED proximity=0 status=0 secver=217 Mar 30 15:21:46:701 [140605283890944] 2 hub: ch_hublist /DOMAIN/Secondary/ROBOT_NAME/hub Mar 30 15:21:46:701 [140605283890944] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:21:46:701 [140605283890944] 5 hub: hublist->TunnelB[+]: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:21:46:701 [140605283890944] 5 hub: hublist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:21:46:701 [140605283890944] 5 hub: hublist->TunnelB[+]: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:21:46:701 [140605283890944] 5 hub: hublist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:21:46:702 [140605283890944] 4 hub: CONNECT: 0x7fe130002eb0(39) 127.0.0.1/60626->127.0.0.1/48009 Mar 30 15:21:46:702 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60626 Mar 30 15:21:46:702 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60626' delegated to TSESS-A-5 [140603715176192] Mar 30 15:21:46:702 [140603715176192] 3 hub: TSESS-A-5-3 [TunnelB] restarted (for local) Mar 30 15:21:46:706 [140603203516160] 5 hub: TSESS-A-6-0 [Secondary] local 100 '_close' Mar 30 15:21:46:706 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:46:706 [140603203516160] 1 hub: TSESS-A-6-0 [Secondary] sent 100 '_close' (293 bytes, 0 ms) Mar 30 15:21:46:706 [140603203516160] 3 hub: TSESS-A-6-0 [Secondary] local session closed, suspending Mar 30 15:21:46:707 [140603715176192] 5 hub: TSESS-A-5-3 [TunnelB] local 100 'hubup' Mar 30 15:21:46:707 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:46:707 [140603715176192] 1 hub: TSESS-A-5-3 [TunnelB] sent 100 'hubup' (1623 bytes, 0 ms) Mar 30 15:21:46:713 [140603715176192] 1 hub: TSESS-A-5-3 [TunnelB] ssl 200 '-' (442 bytes) Mar 30 15:21:46:713 [140603715176192] 3 hub: TSESS-A-5-3 [TunnelB] is paired with [P-298-3] Mar 30 15:21:46:713 [140603715176192] 1 hub: TSESS-A-5-3 [TunnelB] received 200 '-' (450 bytes) Mar 30 15:21:46:713 [140605283890944] 1 hub: hubup succeeded - /DOMAIN/TunnelB 10.x.x.x/48002 (reachable via 127.0.0.1:48009) Mar 30 15:21:46:713 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:21:46:713 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:21:46:713 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:21:46:713 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:21:46:713 [140605283890944] 5 hub: hubinfo_update - topology - origin UPDATED - old origin= , new origin= TunnelB - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:21:46:713 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB CHANGED proximity=0 status=0 secver=217 Mar 30 15:21:46:713 [140605283890944] 2 hub: ch_hublist /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:21:46:713 [140605283890944] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:21:46:713 [140605283890944] 5 hub: hublist->SecondaryHA[+]: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:21:46:713 [140605283890944] 5 hub: hublist->SecondaryHA[+]: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:21:46:713 [140605283890944] 5 hub: hublist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:21:46:713 [140605283890944] 5 hub: hublist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:21:46:713 [140605283890944] 4 hub: CONNECT: 0x7fe1300043f0(39) 127.0.0.1/39814->127.0.0.1/48008 Mar 30 15:21:46:714 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/39814 Mar 30 15:21:46:714 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/39814' delegated to TSESS-A-3 [0000] Mar 30 15:21:46:714 [140603195123456] 3 hub: TSESS-A-3-0 [SecondaryHA] started (local) in tid=3145721600 Mar 30 15:21:46:718 [140603715176192] 5 hub: TSESS-A-5-3 [TunnelB] local 100 '_close' Mar 30 15:21:46:718 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:46:718 [140603715176192] 1 hub: TSESS-A-5-3 [TunnelB] sent 100 '_close' (291 bytes, 0 ms) Mar 30 15:21:46:718 [140603715176192] 3 hub: TSESS-A-5-3 [TunnelB] local session closed, suspending Mar 30 15:21:46:719 [140603195123456] 5 hub: TSESS-A-3-0 [SecondaryHA] local 100 'hubup' Mar 30 15:21:46:719 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:46:719 [140603195123456] 1 hub: TSESS-A-3-0 [SecondaryHA] sent 100 'hubup' (1699 bytes, 0 ms) Mar 30 15:21:46:729 [140603195123456] 1 hub: TSESS-A-3-0 [SecondaryHA] ssl 200 '-' (457 bytes) Mar 30 15:21:46:729 [140603195123456] 3 hub: TSESS-A-3-0 [SecondaryHA] is paired with [P-1947-0] Mar 30 15:21:46:729 [140603195123456] 1 hub: TSESS-A-3-0 [SecondaryHA] received 200 '-' (465 bytes) Mar 30 15:21:46:729 [140605283890944] 1 hub: hubup succeeded - /DOMAIN/SecondaryHA 10.x.x.x/48002 (reachable via 127.0.0.1:48008) Mar 30 15:21:46:729 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:21:46:729 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:21:46:729 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:21:46:729 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:21:46:729 [140605283890944] 5 hub: hubinfo_update - topology - origin UPDATED - old origin= , new origin= SecondaryHA - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub Mar 30 15:21:46:729 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA CHANGED proximity=0 status=0 secver=217 Mar 30 15:21:46:729 [140605283890944] 2 hub: ch_hublist /DOMAIN/SecondaryHA/ROBOT_NAME/hub Mar 30 15:21:46:729 [140605283890944] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:21:46:729 [140605283890944] 1 hub: PERF: hubup_check_interval: 0 sec hubhash_size: 4 Mar 30 15:21:46:729 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:21:46:729 [140605283890944] 2 hub: hubupdatelist->Secondary[+]: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:21:46:729 [140605283890944] 2 hub: hubupdatelist->Secondary[+]: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:21:46:729 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:21:46:730 [140605283890944] 4 hub: CONNECT: 0x7fe130003a50(39) 127.0.0.1/46768->127.0.0.1/48007 Mar 30 15:21:46:730 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/46768 Mar 30 15:21:46:730 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/46768' delegated to TSESS-A-12 [0000] Mar 30 15:21:46:730 [140603186730752] 3 hub: TSESS-A-12-0 [Secondary] started (local) in tid=3137328896 Mar 30 15:21:46:734 [140603195123456] 5 hub: TSESS-A-3-0 [SecondaryHA] local 100 '_close' Mar 30 15:21:46:734 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:46:734 [140603195123456] 1 hub: TSESS-A-3-0 [SecondaryHA] sent 100 '_close' (295 bytes, 0 ms) Mar 30 15:21:46:734 [140603195123456] 3 hub: TSESS-A-3-0 [SecondaryHA] local session closed, suspending Mar 30 15:21:46:735 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] local 100 'hubup' Mar 30 15:21:46:735 [140603186730752] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:46:735 [140603186730752] 1 hub: TSESS-A-12-0 [Secondary] sent 100 'hubup' (1702 bytes, 0 ms) Mar 30 15:21:46:745 [140603186730752] 1 hub: TSESS-A-12-0 [Secondary] ssl 200 '-' (453 bytes) Mar 30 15:21:46:745 [140603186730752] 3 hub: TSESS-A-12-0 [Secondary] is paired with [P-2122-0] Mar 30 15:21:46:745 [140603186730752] 1 hub: TSESS-A-12-0 [Secondary] received 200 '-' (461 bytes) Mar 30 15:21:46:745 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:21:46:745 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:21:46:745 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:21:46:745 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:21:46:745 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary same proximity=0 status=0 secver=217 Mar 30 15:21:46:745 [140605283890944] 2 hub: hubupdatelist->TunnelB[+]: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:21:46:745 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:21:46:745 [140605283890944] 2 hub: hubupdatelist->TunnelB[+]: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:21:46:745 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:21:46:746 [140605283890944] 4 hub: CONNECT: 0x7fe130003970(39) 127.0.0.1/60632->127.0.0.1/48009 Mar 30 15:21:46:746 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60632 Mar 30 15:21:46:746 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60632' delegated to TSESS-A-10 [140603706783488] Mar 30 15:21:46:746 [140603706783488] 3 hub: TSESS-A-10-1 [TunnelB] restarted (for local) Mar 30 15:21:46:750 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] local 100 '_close' Mar 30 15:21:46:750 [140603186730752] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:46:750 [140603186730752] 1 hub: TSESS-A-12-0 [Secondary] sent 100 '_close' (294 bytes, 0 ms) Mar 30 15:21:46:750 [140603186730752] 3 hub: TSESS-A-12-0 [Secondary] local session closed, suspending Mar 30 15:21:46:751 [140603706783488] 5 hub: TSESS-A-10-1 [TunnelB] local 100 'hubup' Mar 30 15:21:46:751 [140603706783488] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:46:751 [140603706783488] 1 hub: TSESS-A-10-1 [TunnelB] sent 100 'hubup' (1710 bytes, 0 ms) Mar 30 15:21:46:761 [140603706783488] 1 hub: TSESS-A-10-1 [TunnelB] ssl 200 '-' (442 bytes) Mar 30 15:21:46:761 [140603706783488] 3 hub: TSESS-A-10-1 [TunnelB] is paired with [P-300-1] Mar 30 15:21:46:761 [140603706783488] 1 hub: TSESS-A-10-1 [TunnelB] received 200 '-' (450 bytes) Mar 30 15:21:46:761 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:21:46:761 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:21:46:761 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:21:46:761 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:21:46:761 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB same proximity=0 status=0 secver=217 Mar 30 15:21:46:761 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[+]: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:21:46:761 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[+]: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:21:46:761 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:21:46:762 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:21:46:762 [140605283890944] 4 hub: CONNECT: 0x7fe130003960(39) 127.0.0.1/39820->127.0.0.1/48008 Mar 30 15:21:46:762 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/39820 Mar 30 15:21:46:762 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/39820' delegated to TSESS-A-9 [0000] Mar 30 15:21:46:762 [140603178338048] 3 hub: TSESS-A-9-0 [SecondaryHA] started (local) in tid=3128936192 Mar 30 15:21:46:766 [140603706783488] 5 hub: TSESS-A-10-1 [TunnelB] local 100 '_close' Mar 30 15:21:46:766 [140603706783488] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:46:766 [140603706783488] 1 hub: TSESS-A-10-1 [TunnelB] sent 100 '_close' (292 bytes, 0 ms) Mar 30 15:21:46:766 [140603706783488] 3 hub: TSESS-A-10-1 [TunnelB] local session closed, suspending Mar 30 15:21:46:767 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] local 100 'hubup' Mar 30 15:21:46:767 [140603178338048] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:46:767 [140603178338048] 1 hub: TSESS-A-9-0 [SecondaryHA] sent 100 'hubup' (1699 bytes, 0 ms) Mar 30 15:21:46:777 [140603178338048] 1 hub: TSESS-A-9-0 [SecondaryHA] ssl 200 '-' (457 bytes) Mar 30 15:21:46:777 [140603178338048] 3 hub: TSESS-A-9-0 [SecondaryHA] is paired with [P-1949-0] Mar 30 15:21:46:777 [140603178338048] 1 hub: TSESS-A-9-0 [SecondaryHA] received 200 '-' (465 bytes) Mar 30 15:21:46:777 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:21:46:777 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:21:46:777 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:21:46:777 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:21:46:777 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA same proximity=0 status=0 secver=217 Mar 30 15:21:46:777 [140605283890944] 5 hub: mark_hubup_list_updated Mar 30 15:21:46:777 [140605283890944] 5 hub: hublist chnaged reset: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:21:46:777 [140605283890944] 5 hub: hublist chnaged reset: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:21:46:777 [140605283890944] 5 hub: hublist chnaged reset: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:21:46:777 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelBHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:21:46:777 [140605283890944] 5 hub: mark_hubup_list_updated end Mar 30 15:21:46:777 [140605283890944] 1 hub: PERF: hubup_check_for_updates: 0 sec hubhash_size: 4 Mar 30 15:21:46:777 [140605283890944] 1 hub: PERF: hubi_run_command: 0 sec hubhash_size: 4 Mar 30 15:21:46:778 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:21:46:778 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:21:46:778 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:21:46:779 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:21:46:779 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:21:46:779 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:21:46:779 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:21:46:780 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:21:46:780 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:21:46:781 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:21:46:781 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:21:46:781 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:21:46:781 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:21:46:782 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:21:46:782 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] local 100 '_close' Mar 30 15:21:46:782 [140603178338048] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:46:783 [140603178338048] 1 hub: TSESS-A-9-0 [SecondaryHA] sent 100 '_close' (295 bytes, 0 ms) Mar 30 15:21:46:783 [140603178338048] 3 hub: TSESS-A-9-0 [SecondaryHA] local session closed, suspending Mar 30 15:21:46:783 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:21:46:783 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:21:46:784 [140605283890944] 1 hub: PERF: hubi_check_security: 0 sec hubhash_size: 4 Mar 30 15:21:46:907 [140603203516160] 5 hub: TSESS-A-6-0 [Secondary] suspended Mar 30 15:21:46:918 [140603715176192] 5 hub: TSESS-A-5-3 [TunnelB] suspended Mar 30 15:21:46:935 [140603195123456] 5 hub: TSESS-A-3-0 [SecondaryHA] suspended Mar 30 15:21:46:951 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:21:46:953 [140605547460672] 2 hub: verify_login from ctrl 127.0.0.1/53130 Mar 30 15:21:46:953 [140605547460672] 1 hub: verify - [target] prid=controller cmd=probe_list ip=10.x.x.x perm=1 Mar 30 15:21:46:953 [140605547460672] 1 hub: verify - [source] id=ppm ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:46:967 [140603706783488] 5 hub: TSESS-A-10-1 [TunnelB] suspended Mar 30 15:21:46:983 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:21:47:499 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:21:47:955 [140605547460672] 3 hub: probelist - ROBOT_NAME (127.0.0.1) is a valid controller Mar 30 15:21:47:955 [140605547460672] 2 hub: nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME 127.0.0.1/48000 Mar 30 15:21:47:955 [140605547460672] 2 hub: internal alarm - Robot ROBOT_NAME is running, 0, ROBOT_NAME Mar 30 15:21:47:955 [140605547460672] 3 hub: robot probelist update - ROBOT_NAME 10.x.x.x - status change with robot or probes detected, last_change=1585596105, status=0 Mar 30 15:21:47:955 [140605547460672] 3 hub: robot probelist update - ROBOT_NAME 10.x.x.x - send robot change event Mar 30 15:21:47:955 [140605547460672] 2 hub: ch_probe /DOMAIN/TunnelBHA/ROBOT_NAME Mar 30 15:21:47:955 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:21:47:955 [140605547460672] 1 hub: robot probelist update - probelist - ROBOT_NAME 127.0.0.1 (s:6200) Mar 30 15:21:47:955 [140605547460672] 2 hub: nns(-):/DOMAIN/TunnelBHA/ROBOT_NAME/spooler Mar 30 15:21:47:955 [140605547460672] 2 hub: nns(-):/DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:21:47:955 [140605547460672] 2 hub: nns(-):/DOMAIN/TunnelBHA/ROBOT_NAME/controller Mar 30 15:21:47:955 [140605547460672] 2 hub: nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/controller 127.0.0.1/48000 Mar 30 15:21:47:955 [140605547460672] 2 hub: probe port added: ROBOT_NAME controller 10.x.x.x 48000 Mar 30 15:21:47:955 [140605547460672] 2 hub: nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:21:47:955 [140605547460672] 2 hub: probe port added: ROBOT_NAME hub 10.x.x.x 48002 Mar 30 15:21:47:955 [140605547460672] 2 hub: nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/spooler 127.0.0.1/48001 Mar 30 15:21:47:955 [140605547460672] 2 hub: probe port added: ROBOT_NAME spooler 10.x.x.x 48001 Mar 30 15:21:47:955 [140605547460672] 2 hub: nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/distsrv 127.0.0.1/48010 Mar 30 15:21:47:955 [140605547460672] 2 hub: probe port added: ROBOT_NAME distsrv 10.x.x.x 48010 Mar 30 15:21:47:955 [140605547460672] 2 hub: nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hdb 127.0.0.1/48011 Mar 30 15:21:47:955 [140605547460672] 2 hub: probe port added: ROBOT_NAME hdb 10.x.x.x 48011 Mar 30 15:21:47:955 [140605547460672] 2 hub: nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/cdm 127.0.0.1/48012 Mar 30 15:21:47:955 [140605547460672] 2 hub: probe port added: ROBOT_NAME cdm 10.x.x.x 48012 Mar 30 15:21:47:955 [140605547460672] 2 hub: nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/HA 127.0.0.1/48013 Mar 30 15:21:47:955 [140605547460672] 2 hub: probe port added: ROBOT_NAME HA 10.x.x.x 48013 Mar 30 15:21:47:955 [140605547460672] 2 hub: nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/ppm 127.0.0.1/48014 Mar 30 15:21:47:955 [140605547460672] 2 hub: probe port added: ROBOT_NAME ppm 10.x.x.x 48014 Mar 30 15:21:47:955 [140605547460672] 2 hub: nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/automated_deployment_engine 127.0.0.1/48015 Mar 30 15:21:47:955 [140605547460672] 2 hub: probe port added: ROBOT_NAME automated_deployment_engine 10.x.x.x 48015 Mar 30 15:21:47:955 [140605547460672] 3 hub: check_if_primary...: result = 1 Mar 30 15:21:48:957 [140605007062784] 5 hub: Reclaiming post session [0x7fe114001fd0] from '127.0.0.1/41656' Mar 30 15:21:49:301 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:21:51:104 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:21:51:648 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:21:51:648 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:21:51:648 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60662 Mar 30 15:21:51:648 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60662' delegated to TSESS-A-5 [140603715176192] Mar 30 15:21:51:648 [140603715176192] 3 hub: TSESS-A-5-4 [TunnelB] restarted (for local) Mar 30 15:21:51:653 [140603715176192] 5 hub: TSESS-A-5-4 [TunnelB] local 100 'get_info' Mar 30 15:21:51:653 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:51:654 [140603715176192] 1 hub: TSESS-A-5-4 [TunnelB] sent 100 'get_info' (312 bytes, 0 ms) Mar 30 15:21:51:664 [140603715176192] 1 hub: TSESS-A-5-4 [TunnelB] ssl 200 '-' (1277 bytes) Mar 30 15:21:51:664 [140603715176192] 3 hub: TSESS-A-5-4 [TunnelB] is paired with [P-298-4] Mar 30 15:21:51:664 [140603715176192] 1 hub: TSESS-A-5-4 [TunnelB] received 200 '-' (1285 bytes) Mar 30 15:21:51:665 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60664 Mar 30 15:21:51:665 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60664' delegated to TSESS-A-10 [140603706783488] Mar 30 15:21:51:665 [140603706783488] 3 hub: TSESS-A-10-2 [TunnelB] restarted (for local) Mar 30 15:21:51:669 [140603715176192] 5 hub: TSESS-A-5-4 [TunnelB] local 100 '_close' Mar 30 15:21:51:669 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:51:669 [140603715176192] 1 hub: TSESS-A-5-4 [TunnelB] sent 100 '_close' (310 bytes, 0 ms) Mar 30 15:21:51:669 [140603715176192] 3 hub: TSESS-A-5-4 [TunnelB] local session closed, suspending Mar 30 15:21:51:670 [140603706783488] 5 hub: TSESS-A-10-2 [TunnelB] local 100 '_status' Mar 30 15:21:51:670 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:51:670 [140603706783488] 1 hub: TSESS-A-10-2 [TunnelB] sent 100 '_status' (312 bytes, 0 ms) Mar 30 15:21:51:680 [140603706783488] 1 hub: TSESS-A-10-2 [TunnelB] ssl 200 '-' (279 bytes) Mar 30 15:21:51:680 [140603706783488] 3 hub: TSESS-A-10-2 [TunnelB] is paired with [P-300-2] Mar 30 15:21:51:680 [140603706783488] 1 hub: TSESS-A-10-2 [TunnelB] received 200 '-' (287 bytes) Mar 30 15:21:51:685 [140603706783488] 5 hub: TSESS-A-10-2 [TunnelB] local 100 '_close' Mar 30 15:21:51:686 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:21:51:686 [140603706783488] 1 hub: TSESS-A-10-2 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:21:51:686 [140603706783488] 3 hub: TSESS-A-10-2 [TunnelB] local session closed, suspending Mar 30 15:21:51:870 [140603715176192] 5 hub: TSESS-A-5-4 [TunnelB] suspended Mar 30 15:21:51:886 [140603706783488] 5 hub: TSESS-A-10-2 [TunnelB] suspended Mar 30 15:21:53:808 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:21:55:610 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:21:56:653 [140605547460672] 4 hub: medium timeout Mar 30 15:21:56:653 [140605547460672] 1 hub: robot check in - result from ROBOT_NAME remove=0, status=OK Mar 30 15:21:56:653 [140605547460672] 1 hub: robot check in - next run Mon Mar 30 15:44:56 2020 Mar 30 15:21:56:653 [140605547460672] 3 hub: next_check [1585596396] now [1585596116] Mar 30 15:21:56:907 [140603203516160] 5 hub: TSESS-A-6-0 [Secondary] suspended Mar 30 15:21:56:935 [140603195123456] 5 hub: TSESS-A-3-0 [SecondaryHA] suspended Mar 30 15:21:56:951 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:21:56:983 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:21:57:413 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:21:59:216 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:01:018 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:01:691 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:22:01:691 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:22:01:691 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60670 Mar 30 15:22:01:691 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60670' delegated to TSESS-A-5 [140603715176192] Mar 30 15:22:01:691 [140603715176192] 3 hub: TSESS-A-5-5 [TunnelB] restarted (for local) Mar 30 15:22:01:697 [140603715176192] 5 hub: TSESS-A-5-5 [TunnelB] local 100 'get_info' Mar 30 15:22:01:697 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:01:697 [140603715176192] 1 hub: TSESS-A-5-5 [TunnelB] sent 100 'get_info' (312 bytes, 0 ms) Mar 30 15:22:01:707 [140603715176192] 1 hub: TSESS-A-5-5 [TunnelB] ssl 200 '-' (1277 bytes) Mar 30 15:22:01:707 [140603715176192] 3 hub: TSESS-A-5-5 [TunnelB] is paired with [P-298-5] Mar 30 15:22:01:707 [140603715176192] 1 hub: TSESS-A-5-5 [TunnelB] received 200 '-' (1285 bytes) Mar 30 15:22:01:707 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60672 Mar 30 15:22:01:707 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60672' delegated to TSESS-A-10 [140603706783488] Mar 30 15:22:01:707 [140603706783488] 3 hub: TSESS-A-10-3 [TunnelB] restarted (for local) Mar 30 15:22:01:712 [140603715176192] 5 hub: TSESS-A-5-5 [TunnelB] local 100 '_close' Mar 30 15:22:01:712 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:01:712 [140603715176192] 1 hub: TSESS-A-5-5 [TunnelB] sent 100 '_close' (310 bytes, 0 ms) Mar 30 15:22:01:712 [140603715176192] 3 hub: TSESS-A-5-5 [TunnelB] local session closed, suspending Mar 30 15:22:01:712 [140603706783488] 5 hub: TSESS-A-10-3 [TunnelB] local 100 '_status' Mar 30 15:22:01:712 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:01:712 [140603706783488] 1 hub: TSESS-A-10-3 [TunnelB] sent 100 '_status' (312 bytes, 0 ms) Mar 30 15:22:01:723 [140603706783488] 1 hub: TSESS-A-10-3 [TunnelB] ssl 200 '-' (279 bytes) Mar 30 15:22:01:723 [140603706783488] 3 hub: TSESS-A-10-3 [TunnelB] is paired with [P-300-3] Mar 30 15:22:01:723 [140603706783488] 1 hub: TSESS-A-10-3 [TunnelB] received 200 '-' (287 bytes) Mar 30 15:22:01:728 [140603706783488] 5 hub: TSESS-A-10-3 [TunnelB] local 100 '_close' Mar 30 15:22:01:728 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:01:728 [140603706783488] 1 hub: TSESS-A-10-3 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:22:01:728 [140603706783488] 3 hub: TSESS-A-10-3 [TunnelB] local session closed, suspending Mar 30 15:22:01:912 [140603715176192] 5 hub: TSESS-A-5-5 [TunnelB] suspended Mar 30 15:22:01:928 [140603706783488] 5 hub: TSESS-A-10-3 [TunnelB] suspended Mar 30 15:22:03:722 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:04:695 [140605547460672] 3 hub: check_session - session 0x20f9050 (127.0.0.1/48002) is a server Mar 30 15:22:04:695 [140605547460672] 2 hub: Profile - clayk 38 bytes (OK) Mar 30 15:22:05:524 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:06:697 [140605547460672] 4 hub: medium timeout Mar 30 15:22:06:697 [140605547460672] 3 hub: next_check [1585596396] now [1585596126] Mar 30 15:22:06:907 [140603203516160] 5 hub: TSESS-A-6-0 [Secondary] suspended Mar 30 15:22:06:935 [140603195123456] 5 hub: TSESS-A-3-0 [SecondaryHA] suspended Mar 30 15:22:06:951 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:22:06:983 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:22:07:327 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:09:129 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:11:734 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:22:11:734 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:22:11:735 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60678 Mar 30 15:22:11:735 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60678' delegated to TSESS-A-5 [140603715176192] Mar 30 15:22:11:735 [140603715176192] 3 hub: TSESS-A-5-6 [TunnelB] restarted (for local) Mar 30 15:22:11:740 [140603715176192] 5 hub: TSESS-A-5-6 [TunnelB] local 100 'get_info' Mar 30 15:22:11:740 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:11:740 [140603715176192] 1 hub: TSESS-A-5-6 [TunnelB] sent 100 'get_info' (312 bytes, 0 ms) Mar 30 15:22:11:750 [140603715176192] 1 hub: TSESS-A-5-6 [TunnelB] ssl 200 '-' (1277 bytes) Mar 30 15:22:11:750 [140603715176192] 3 hub: TSESS-A-5-6 [TunnelB] is paired with [P-298-6] Mar 30 15:22:11:750 [140603715176192] 1 hub: TSESS-A-5-6 [TunnelB] received 200 '-' (1285 bytes) Mar 30 15:22:11:750 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60680 Mar 30 15:22:11:750 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60680' delegated to TSESS-A-10 [140603706783488] Mar 30 15:22:11:750 [140603706783488] 3 hub: TSESS-A-10-4 [TunnelB] restarted (for local) Mar 30 15:22:11:755 [140603715176192] 5 hub: TSESS-A-5-6 [TunnelB] local 100 '_close' Mar 30 15:22:11:755 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:11:755 [140603715176192] 1 hub: TSESS-A-5-6 [TunnelB] sent 100 '_close' (310 bytes, 0 ms) Mar 30 15:22:11:755 [140603715176192] 3 hub: TSESS-A-5-6 [TunnelB] local session closed, suspending Mar 30 15:22:11:755 [140603706783488] 5 hub: TSESS-A-10-4 [TunnelB] local 100 '_status' Mar 30 15:22:11:755 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:11:756 [140603706783488] 1 hub: TSESS-A-10-4 [TunnelB] sent 100 '_status' (312 bytes, 0 ms) Mar 30 15:22:11:766 [140603706783488] 1 hub: TSESS-A-10-4 [TunnelB] ssl 200 '-' (279 bytes) Mar 30 15:22:11:766 [140603706783488] 3 hub: TSESS-A-10-4 [TunnelB] is paired with [P-300-4] Mar 30 15:22:11:766 [140603706783488] 1 hub: TSESS-A-10-4 [TunnelB] received 200 '-' (287 bytes) Mar 30 15:22:11:771 [140603706783488] 5 hub: TSESS-A-10-4 [TunnelB] local 100 '_close' Mar 30 15:22:11:771 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:11:771 [140603706783488] 1 hub: TSESS-A-10-4 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:22:11:771 [140603706783488] 3 hub: TSESS-A-10-4 [TunnelB] local session closed, suspending Mar 30 15:22:11:831 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:11:955 [140603715176192] 5 hub: TSESS-A-5-6 [TunnelB] suspended Mar 30 15:22:11:971 [140603706783488] 5 hub: TSESS-A-10-4 [TunnelB] suspended Mar 30 15:22:13:634 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:15:436 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:16:740 [140605547460672] 4 hub: medium timeout Mar 30 15:22:16:740 [140605547460672] 3 hub: next_check [1585596396] now [1585596136] Mar 30 15:22:16:815 [140605283890944] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:22:16:815 [140605283890944] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:22:16:816 [140605283890944] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:22:16:816 [140605283890944] 3 hub: hubhash_write - 0 hubs saved to cache Mar 30 15:22:16:816 [140605283890944] 1 hub: PERF: hubup_check_interval: 0 sec hubhash_size: 4 Mar 30 15:22:16:816 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:22:16:816 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:22:16:816 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:22:16:816 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:22:16:816 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/46824->127.0.0.1/48007 Mar 30 15:22:16:816 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/46824 Mar 30 15:22:16:816 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/46824' delegated to TSESS-A-6 [140603203516160] Mar 30 15:22:16:816 [140603203516160] 3 hub: TSESS-A-6-1 [Secondary] restarted (for local) Mar 30 15:22:16:821 [140603203516160] 5 hub: TSESS-A-6-1 [Secondary] local 100 'hubup' Mar 30 15:22:16:821 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:16:821 [140603203516160] 1 hub: TSESS-A-6-1 [Secondary] sent 100 'hubup' (686 bytes, 0 ms) Mar 30 15:22:16:831 [140603203516160] 1 hub: TSESS-A-6-1 [Secondary] ssl 200 '-' (453 bytes) Mar 30 15:22:16:831 [140603203516160] 3 hub: TSESS-A-6-1 [Secondary] is paired with [P-2120-1] Mar 30 15:22:16:831 [140603203516160] 1 hub: TSESS-A-6-1 [Secondary] received 200 '-' (461 bytes) Mar 30 15:22:16:832 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:22:16:832 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:22:16:832 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:22:16:832 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:22:16:832 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary same proximity=0 status=0 secver=217 Mar 30 15:22:16:832 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:22:16:832 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:22:16:832 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:22:16:832 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:22:16:832 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/60688->127.0.0.1/48009 Mar 30 15:22:16:832 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60688 Mar 30 15:22:16:832 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60688' delegated to TSESS-A-5 [140603715176192] Mar 30 15:22:16:832 [140603715176192] 3 hub: TSESS-A-5-7 [TunnelB] restarted (for local) Mar 30 15:22:16:837 [140603203516160] 5 hub: TSESS-A-6-1 [Secondary] local 100 '_close' Mar 30 15:22:16:837 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:16:837 [140603203516160] 1 hub: TSESS-A-6-1 [Secondary] sent 100 '_close' (293 bytes, 0 ms) Mar 30 15:22:16:837 [140603203516160] 3 hub: TSESS-A-6-1 [Secondary] local session closed, suspending Mar 30 15:22:16:837 [140603715176192] 5 hub: TSESS-A-5-7 [TunnelB] local 100 'hubup' Mar 30 15:22:16:837 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:16:837 [140603715176192] 1 hub: TSESS-A-5-7 [TunnelB] sent 100 'hubup' (684 bytes, 0 ms) Mar 30 15:22:16:847 [140603715176192] 1 hub: TSESS-A-5-7 [TunnelB] ssl 200 '-' (442 bytes) Mar 30 15:22:16:847 [140603715176192] 3 hub: TSESS-A-5-7 [TunnelB] is paired with [P-298-7] Mar 30 15:22:16:847 [140603715176192] 1 hub: TSESS-A-5-7 [TunnelB] received 200 '-' (450 bytes) Mar 30 15:22:16:847 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:22:16:847 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:22:16:847 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:22:16:847 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:22:16:847 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB same proximity=0 status=0 secver=217 Mar 30 15:22:16:847 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:22:16:847 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:22:16:847 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:22:16:847 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:22:16:848 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/39876->127.0.0.1/48008 Mar 30 15:22:16:848 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/39876 Mar 30 15:22:16:848 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/39876' delegated to TSESS-A-3 [140603195123456] Mar 30 15:22:16:848 [140603195123456] 3 hub: TSESS-A-3-1 [SecondaryHA] restarted (for local) Mar 30 15:22:16:852 [140603715176192] 5 hub: TSESS-A-5-7 [TunnelB] local 100 '_close' Mar 30 15:22:16:852 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:16:852 [140603715176192] 1 hub: TSESS-A-5-7 [TunnelB] sent 100 '_close' (291 bytes, 0 ms) Mar 30 15:22:16:852 [140603715176192] 3 hub: TSESS-A-5-7 [TunnelB] local session closed, suspending Mar 30 15:22:16:853 [140603195123456] 5 hub: TSESS-A-3-1 [SecondaryHA] local 100 'hubup' Mar 30 15:22:16:853 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:16:853 [140603195123456] 1 hub: TSESS-A-3-1 [SecondaryHA] sent 100 'hubup' (688 bytes, 0 ms) Mar 30 15:22:16:863 [140603195123456] 1 hub: TSESS-A-3-1 [SecondaryHA] ssl 200 '-' (457 bytes) Mar 30 15:22:16:863 [140603195123456] 3 hub: TSESS-A-3-1 [SecondaryHA] is paired with [P-1947-1] Mar 30 15:22:16:863 [140603195123456] 1 hub: TSESS-A-3-1 [SecondaryHA] received 200 '-' (465 bytes) Mar 30 15:22:16:863 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:22:16:863 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:22:16:863 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:22:16:863 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:22:16:863 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA same proximity=0 status=0 secver=217 Mar 30 15:22:16:863 [140605283890944] 5 hub: mark_hubup_list_updated Mar 30 15:22:16:863 [140605283890944] 5 hub: hublist changed: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:22:16:863 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:22:16:863 [140605283890944] 5 hub: hublist changed: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:22:16:863 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelBHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:22:16:863 [140605283890944] 5 hub: mark_hubup_list_updated end Mar 30 15:22:16:863 [140605283890944] 1 hub: PERF: hubup_check_for_updates: 0 sec hubhash_size: 4 Mar 30 15:22:16:863 [140605283890944] 1 hub: PERF: hubi_run_command: 0 sec hubhash_size: 4 Mar 30 15:22:16:863 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:22:16:864 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:22:16:864 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:22:16:864 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:22:16:865 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:22:16:865 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:22:16:865 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:22:16:866 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:22:16:866 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:22:16:866 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:22:16:866 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:22:16:867 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:22:16:867 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:22:16:868 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:22:16:868 [140603195123456] 5 hub: TSESS-A-3-1 [SecondaryHA] local 100 '_close' Mar 30 15:22:16:868 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:16:868 [140603195123456] 1 hub: TSESS-A-3-1 [SecondaryHA] sent 100 '_close' (295 bytes, 0 ms) Mar 30 15:22:16:868 [140603195123456] 3 hub: TSESS-A-3-1 [SecondaryHA] local session closed, suspending Mar 30 15:22:16:868 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:22:16:869 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:22:16:870 [140605283890944] 1 hub: PERF: hubi_check_security: 0 sec hubhash_size: 4 Mar 30 15:22:16:951 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:22:16:983 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:22:17:037 [140603203516160] 5 hub: TSESS-A-6-1 [Secondary] suspended Mar 30 15:22:17:053 [140603715176192] 5 hub: TSESS-A-5-7 [TunnelB] suspended Mar 30 15:22:17:069 [140603195123456] 5 hub: TSESS-A-3-1 [SecondaryHA] suspended Mar 30 15:22:17:239 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:19:042 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:21:746 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:21:778 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:22:21:778 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:22:21:778 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60696 Mar 30 15:22:21:778 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60696' delegated to TSESS-A-5 [140603715176192] Mar 30 15:22:21:778 [140603715176192] 3 hub: TSESS-A-5-8 [TunnelB] restarted (for local) Mar 30 15:22:21:784 [140603715176192] 5 hub: TSESS-A-5-8 [TunnelB] local 100 'get_info' Mar 30 15:22:21:784 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:21:784 [140603715176192] 1 hub: TSESS-A-5-8 [TunnelB] sent 100 'get_info' (312 bytes, 0 ms) Mar 30 15:22:21:794 [140603715176192] 1 hub: TSESS-A-5-8 [TunnelB] ssl 200 '-' (1277 bytes) Mar 30 15:22:21:794 [140603715176192] 3 hub: TSESS-A-5-8 [TunnelB] is paired with [P-298-8] Mar 30 15:22:21:794 [140603715176192] 1 hub: TSESS-A-5-8 [TunnelB] received 200 '-' (1285 bytes) Mar 30 15:22:21:794 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60698 Mar 30 15:22:21:794 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60698' delegated to TSESS-A-10 [140603706783488] Mar 30 15:22:21:794 [140603706783488] 3 hub: TSESS-A-10-5 [TunnelB] restarted (for local) Mar 30 15:22:21:799 [140603715176192] 5 hub: TSESS-A-5-8 [TunnelB] local 100 '_close' Mar 30 15:22:21:799 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:21:799 [140603715176192] 1 hub: TSESS-A-5-8 [TunnelB] sent 100 '_close' (310 bytes, 0 ms) Mar 30 15:22:21:799 [140603715176192] 3 hub: TSESS-A-5-8 [TunnelB] local session closed, suspending Mar 30 15:22:21:800 [140603706783488] 5 hub: TSESS-A-10-5 [TunnelB] local 100 '_status' Mar 30 15:22:21:800 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:21:800 [140603706783488] 1 hub: TSESS-A-10-5 [TunnelB] sent 100 '_status' (312 bytes, 0 ms) Mar 30 15:22:21:810 [140603706783488] 1 hub: TSESS-A-10-5 [TunnelB] ssl 200 '-' (279 bytes) Mar 30 15:22:21:810 [140603706783488] 3 hub: TSESS-A-10-5 [TunnelB] is paired with [P-300-5] Mar 30 15:22:21:810 [140603706783488] 1 hub: TSESS-A-10-5 [TunnelB] received 200 '-' (287 bytes) Mar 30 15:22:21:815 [140603706783488] 5 hub: TSESS-A-10-5 [TunnelB] local 100 '_close' Mar 30 15:22:21:815 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:21:815 [140603706783488] 1 hub: TSESS-A-10-5 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:22:21:815 [140603706783488] 3 hub: TSESS-A-10-5 [TunnelB] local session closed, suspending Mar 30 15:22:22:000 [140603715176192] 5 hub: TSESS-A-5-8 [TunnelB] suspended Mar 30 15:22:22:015 [140603706783488] 5 hub: TSESS-A-10-5 [TunnelB] suspended Mar 30 15:22:23:548 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:25:351 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:26:784 [140605547460672] 4 hub: medium timeout Mar 30 15:22:26:784 [140605547460672] 3 hub: next_check [1585596396] now [1585596146] Mar 30 15:22:26:951 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:22:26:983 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:22:27:037 [140603203516160] 5 hub: TSESS-A-6-1 [Secondary] suspended Mar 30 15:22:27:069 [140603195123456] 5 hub: TSESS-A-3-1 [SecondaryHA] suspended Mar 30 15:22:27:154 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:29:858 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:31:660 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:31:821 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:22:31:821 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:22:31:822 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60704 Mar 30 15:22:31:822 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60704' delegated to TSESS-A-5 [140603715176192] Mar 30 15:22:31:822 [140603715176192] 3 hub: TSESS-A-5-9 [TunnelB] restarted (for local) Mar 30 15:22:31:827 [140603715176192] 5 hub: TSESS-A-5-9 [TunnelB] local 100 'get_info' Mar 30 15:22:31:827 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:31:827 [140603715176192] 1 hub: TSESS-A-5-9 [TunnelB] sent 100 'get_info' (312 bytes, 0 ms) Mar 30 15:22:31:837 [140603715176192] 1 hub: TSESS-A-5-9 [TunnelB] ssl 200 '-' (1277 bytes) Mar 30 15:22:31:837 [140603715176192] 3 hub: TSESS-A-5-9 [TunnelB] is paired with [P-298-9] Mar 30 15:22:31:837 [140603715176192] 1 hub: TSESS-A-5-9 [TunnelB] received 200 '-' (1285 bytes) Mar 30 15:22:31:838 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60706 Mar 30 15:22:31:838 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60706' delegated to TSESS-A-10 [140603706783488] Mar 30 15:22:31:838 [140603706783488] 3 hub: TSESS-A-10-6 [TunnelB] restarted (for local) Mar 30 15:22:31:843 [140603715176192] 5 hub: TSESS-A-5-9 [TunnelB] local 100 '_close' Mar 30 15:22:31:843 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:31:843 [140603715176192] 1 hub: TSESS-A-5-9 [TunnelB] sent 100 '_close' (310 bytes, 0 ms) Mar 30 15:22:31:843 [140603715176192] 3 hub: TSESS-A-5-9 [TunnelB] local session closed, suspending Mar 30 15:22:31:843 [140603706783488] 5 hub: TSESS-A-10-6 [TunnelB] local 100 '_status' Mar 30 15:22:31:843 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:31:843 [140603706783488] 1 hub: TSESS-A-10-6 [TunnelB] sent 100 '_status' (312 bytes, 0 ms) Mar 30 15:22:31:853 [140603706783488] 1 hub: TSESS-A-10-6 [TunnelB] ssl 200 '-' (279 bytes) Mar 30 15:22:31:853 [140603706783488] 3 hub: TSESS-A-10-6 [TunnelB] is paired with [P-300-6] Mar 30 15:22:31:853 [140603706783488] 1 hub: TSESS-A-10-6 [TunnelB] received 200 '-' (287 bytes) Mar 30 15:22:31:858 [140603706783488] 5 hub: TSESS-A-10-6 [TunnelB] local 100 '_close' Mar 30 15:22:31:858 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:31:858 [140603706783488] 1 hub: TSESS-A-10-6 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:22:31:858 [140603706783488] 3 hub: TSESS-A-10-6 [TunnelB] local session closed, suspending Mar 30 15:22:32:043 [140603715176192] 5 hub: TSESS-A-5-9 [TunnelB] suspended Mar 30 15:22:32:059 [140603706783488] 5 hub: TSESS-A-10-6 [TunnelB] suspended Mar 30 15:22:32:456 [140603723568896] 5 hub: TSESS-P-7 not internal command - thread will handle. mtype=100; sfd:22 Mar 30 15:22:32:456 [140603169945344] 3 hub: TSESS-P-7-0 [TunnelB] started (remote) in tid=3120543488 Mar 30 15:22:32:456 [140603169945344] 2 hub: TSESS-P-7 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:32:32 Mar 30 15:22:32:456 [140603169945344] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:22:32:457 [140603169945344] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:22:32:457 [140603169945344] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:22:32:457 [140603169945344] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:22:32:457 [140603169945344] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:22:32:457 [140603169945344] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:22:32:457 [140603169945344] 4 hub: CONNECT: 0x7fe0a8000c80(39) 127.0.0.1/53186->127.0.0.1/48002 Mar 30 15:22:32:457 [140603169945344] 3 hub: TSESS-P-7-0 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:22:32:457 [140603169945344] 1 hub: TSESS-P-7-0 [TunnelB] received 100 'hubup' (672 bytes) Mar 30 15:22:32:457 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53186 (s:378) Mar 30 15:22:32:457 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/53186] Mar 30 15:22:32:457 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:22:32:457 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:22:32:457 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/53186 Mar 30 15:22:32:457 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/53186 same proximity=0 status=0 secver=217 Mar 30 15:22:32:462 [140603169945344] 5 hub: TSESS-P-7-0 [TunnelB] local 200 'OK' Mar 30 15:22:32:462 [140603169945344] 1 hub: TSESS-P-7-0 [TunnelB] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:22:32:472 [140603169945344] 1 hub: TSESS-P-7-0 [TunnelB] ssl 100 '_close' (283 bytes) Mar 30 15:22:32:472 [140603169945344] 3 hub: TSESS-P-7-0 [TunnelB] is paired with [A-299-0] Mar 30 15:22:32:472 [140603169945344] 1 hub: TSESS-P-7-0 [TunnelB] received 100 '_close' (295 bytes) Mar 30 15:22:32:477 [140603169945344] 1 hub: TSESS-P-7-0 [TunnelB] remote session closed Mar 30 15:22:32:477 [140603169945344] 3 hub: TSESS-P-7-0 [TunnelB] exiting msg loop (remote close) Mar 30 15:22:32:477 [140603169945344] 5 hub: TSESS-P-7-0 [TunnelB] suspended Mar 30 15:22:32:524 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:22:32:524 [140603161552640] 3 hub: TSESS-P-4-0 [SecondaryHA] started (remote) in tid=3112150784 Mar 30 15:22:32:524 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:32:32 Mar 30 15:22:32:524 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:22:32:524 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:22:32:524 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:22:32:524 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:22:32:524 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:22:32:524 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:22:32:524 [140603161552640] 4 hub: CONNECT: 0x7fe09c000c60(39) 127.0.0.1/53188->127.0.0.1/48002 Mar 30 15:22:32:524 [140603161552640] 3 hub: TSESS-P-4-0 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:22:32:524 [140603161552640] 1 hub: TSESS-P-4-0 [SecondaryHA] received 100 'hubup' (687 bytes) Mar 30 15:22:32:524 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53188 (s:392) Mar 30 15:22:32:524 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/53188] Mar 30 15:22:32:524 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:22:32:524 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:22:32:524 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/53188 Mar 30 15:22:32:524 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/53188 same proximity=0 status=0 secver=217 Mar 30 15:22:32:525 [140603740354304] 5 hub: TSESS-P-8 not internal command - thread will handle. mtype=100; sfd:23 Mar 30 15:22:32:525 [140603153159936] 3 hub: TSESS-P-8-0 [Secondary] started (remote) in tid=3103758080 Mar 30 15:22:32:525 [140603153159936] 2 hub: TSESS-P-8 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:32:32 Mar 30 15:22:32:525 [140603153159936] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:22:32:525 [140603153159936] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:22:32:525 [140603153159936] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:22:32:525 [140603153159936] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:22:32:525 [140603153159936] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:22:32:525 [140603153159936] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:22:32:525 [140603153159936] 4 hub: CONNECT: 0x7fe0a0000be0(41) 127.0.0.1/53190->127.0.0.1/48002 Mar 30 15:22:32:525 [140603153159936] 3 hub: TSESS-P-8-0 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:22:32:525 [140603153159936] 1 hub: TSESS-P-8-0 [Secondary] received 100 'hubup' (683 bytes) Mar 30 15:22:32:525 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53190 (s:388) Mar 30 15:22:32:525 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/53190] Mar 30 15:22:32:525 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:22:32:525 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:22:32:525 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/53190 Mar 30 15:22:32:525 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/53190 same proximity=0 status=0 secver=217 Mar 30 15:22:32:529 [140603161552640] 5 hub: TSESS-P-4-0 [SecondaryHA] local 200 'OK' Mar 30 15:22:32:529 [140603161552640] 1 hub: TSESS-P-4-0 [SecondaryHA] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:22:32:530 [140603153159936] 5 hub: TSESS-P-8-0 [Secondary] local 200 'OK' Mar 30 15:22:32:530 [140603153159936] 1 hub: TSESS-P-8-0 [Secondary] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:22:32:540 [140603161552640] 1 hub: TSESS-P-4-0 [SecondaryHA] ssl 100 '_close' (300 bytes) Mar 30 15:22:32:540 [140603161552640] 3 hub: TSESS-P-4-0 [SecondaryHA] is paired with [A-1948-0] Mar 30 15:22:32:540 [140603161552640] 1 hub: TSESS-P-4-0 [SecondaryHA] received 100 '_close' (296 bytes) Mar 30 15:22:32:540 [140603153159936] 1 hub: TSESS-P-8-0 [Secondary] ssl 100 '_close' (296 bytes) Mar 30 15:22:32:540 [140603153159936] 3 hub: TSESS-P-8-0 [Secondary] is paired with [A-2121-0] Mar 30 15:22:32:540 [140603153159936] 1 hub: TSESS-P-8-0 [Secondary] received 100 '_close' (296 bytes) Mar 30 15:22:32:545 [140603161552640] 1 hub: TSESS-P-4-0 [SecondaryHA] remote session closed Mar 30 15:22:32:545 [140603161552640] 3 hub: TSESS-P-4-0 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:22:32:545 [140603161552640] 5 hub: TSESS-P-4-0 [SecondaryHA] suspended Mar 30 15:22:32:545 [140603153159936] 1 hub: TSESS-P-8-0 [Secondary] remote session closed Mar 30 15:22:32:545 [140603153159936] 3 hub: TSESS-P-8-0 [Secondary] exiting msg loop (remote close) Mar 30 15:22:32:546 [140603153159936] 5 hub: TSESS-P-8-0 [Secondary] suspended Mar 30 15:22:33:462 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:33:513 [140603723568896] 5 hub: TSESS-P-7 not internal command - thread will handle. mtype=100; sfd:22 Mar 30 15:22:33:513 [140603723568896] 2 hub: CTRL TunnelB request 'hubup' passed to TSESS-P-7 [140603169945344] Mar 30 15:22:33:513 [140603169945344] 3 hub: TSESS-P-7-1 [TunnelB] restarted (for remote) Mar 30 15:22:33:513 [140603169945344] 2 hub: TSESS-P-7 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:32:33 Mar 30 15:22:33:513 [140603169945344] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:22:33:513 [140603169945344] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:22:33:513 [140603169945344] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:22:33:513 [140603169945344] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:22:33:513 [140603169945344] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:22:33:513 [140603169945344] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:22:33:514 [140603169945344] 4 hub: CONNECT: 0x7fe0a8000d20(39) 127.0.0.1/53192->127.0.0.1/48002 Mar 30 15:22:33:514 [140603169945344] 3 hub: TSESS-P-7-1 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:22:33:514 [140603169945344] 1 hub: TSESS-P-7-1 [TunnelB] received 100 'hubup' (672 bytes) Mar 30 15:22:33:514 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53192 (s:378) Mar 30 15:22:33:514 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/53192] Mar 30 15:22:33:514 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:22:33:514 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:22:33:514 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/53192 Mar 30 15:22:33:514 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/53192 same proximity=0 status=0 secver=217 Mar 30 15:22:33:519 [140603169945344] 5 hub: TSESS-P-7-1 [TunnelB] local 200 'OK' Mar 30 15:22:33:519 [140603169945344] 1 hub: TSESS-P-7-1 [TunnelB] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:22:33:529 [140603169945344] 1 hub: TSESS-P-7-1 [TunnelB] ssl 100 '_close' (283 bytes) Mar 30 15:22:33:529 [140603169945344] 3 hub: TSESS-P-7-1 [TunnelB] is paired with [A-299-1] Mar 30 15:22:33:529 [140603169945344] 1 hub: TSESS-P-7-1 [TunnelB] received 100 '_close' (295 bytes) Mar 30 15:22:33:534 [140603169945344] 1 hub: TSESS-P-7-1 [TunnelB] remote session closed Mar 30 15:22:33:534 [140603169945344] 3 hub: TSESS-P-7-1 [TunnelB] exiting msg loop (remote close) Mar 30 15:22:33:534 [140603169945344] 5 hub: TSESS-P-7-1 [TunnelB] suspended Mar 30 15:22:33:632 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:22:33:632 [140603731961600] 2 hub: CTRL SecondaryHA request 'hubup' passed to TSESS-P-4 [140603161552640] Mar 30 15:22:33:633 [140603161552640] 3 hub: TSESS-P-4-1 [SecondaryHA] restarted (for remote) Mar 30 15:22:33:633 [140603740354304] 5 hub: TSESS-P-8 not internal command - thread will handle. mtype=100; sfd:23 Mar 30 15:22:33:633 [140603740354304] 2 hub: CTRL Secondary request 'hubup' passed to TSESS-P-8 [140603153159936] Mar 30 15:22:33:633 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:32:33 Mar 30 15:22:33:633 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:22:33:633 [140603153159936] 3 hub: TSESS-P-8-1 [Secondary] restarted (for remote) Mar 30 15:22:33:633 [140603153159936] 2 hub: TSESS-P-8 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:32:33 Mar 30 15:22:33:633 [140603153159936] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:22:33:633 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:22:33:633 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:22:33:633 [140603153159936] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:22:33:633 [140603153159936] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:22:33:633 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:22:33:633 [140603153159936] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:22:33:633 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:22:33:633 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:22:33:633 [140603153159936] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:22:33:633 [140603153159936] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:22:33:633 [140603161552640] 4 hub: CONNECT: 0x7fe09c000c60(40) 127.0.0.1/53194->127.0.0.1/48002 Mar 30 15:22:33:633 [140603153159936] 4 hub: CONNECT: 0x7fe0a0000be0(39) 127.0.0.1/53196->127.0.0.1/48002 Mar 30 15:22:33:633 [140603153159936] 3 hub: TSESS-P-8-1 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:22:33:633 [140603161552640] 3 hub: TSESS-P-4-1 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:22:33:633 [140603153159936] 1 hub: TSESS-P-8-1 [Secondary] received 100 'hubup' (683 bytes) Mar 30 15:22:33:633 [140603161552640] 1 hub: TSESS-P-4-1 [SecondaryHA] received 100 'hubup' (687 bytes) Mar 30 15:22:33:633 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53196 (s:388) Mar 30 15:22:33:633 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/53196] Mar 30 15:22:33:633 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:22:33:633 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:22:33:633 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/53196 Mar 30 15:22:33:633 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/53196 same proximity=0 status=0 secver=217 Mar 30 15:22:33:633 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53194 (s:392) Mar 30 15:22:33:633 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/53194] Mar 30 15:22:33:633 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:22:33:633 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:22:33:633 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/53194 Mar 30 15:22:33:633 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/53194 same proximity=0 status=0 secver=217 Mar 30 15:22:33:638 [140603153159936] 5 hub: TSESS-P-8-1 [Secondary] local 200 'OK' Mar 30 15:22:33:638 [140603161552640] 5 hub: TSESS-P-4-1 [SecondaryHA] local 200 'OK' Mar 30 15:22:33:638 [140603153159936] 1 hub: TSESS-P-8-1 [Secondary] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:22:33:638 [140603161552640] 1 hub: TSESS-P-4-1 [SecondaryHA] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:22:33:648 [140603153159936] 1 hub: TSESS-P-8-1 [Secondary] ssl 100 '_close' (296 bytes) Mar 30 15:22:33:649 [140603153159936] 3 hub: TSESS-P-8-1 [Secondary] is paired with [A-2121-1] Mar 30 15:22:33:648 [140603161552640] 1 hub: TSESS-P-4-1 [SecondaryHA] ssl 100 '_close' (300 bytes) Mar 30 15:22:33:649 [140603153159936] 1 hub: TSESS-P-8-1 [Secondary] received 100 '_close' (296 bytes) Mar 30 15:22:33:649 [140603161552640] 3 hub: TSESS-P-4-1 [SecondaryHA] is paired with [A-1948-1] Mar 30 15:22:33:649 [140603161552640] 1 hub: TSESS-P-4-1 [SecondaryHA] received 100 '_close' (296 bytes) Mar 30 15:22:33:654 [140603153159936] 1 hub: TSESS-P-8-1 [Secondary] remote session closed Mar 30 15:22:33:654 [140603153159936] 3 hub: TSESS-P-8-1 [Secondary] exiting msg loop (remote close) Mar 30 15:22:33:654 [140603161552640] 1 hub: TSESS-P-4-1 [SecondaryHA] remote session closed Mar 30 15:22:33:654 [140603153159936] 5 hub: TSESS-P-8-1 [Secondary] suspended Mar 30 15:22:33:654 [140603161552640] 3 hub: TSESS-P-4-1 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:22:33:654 [140603161552640] 5 hub: TSESS-P-4-1 [SecondaryHA] suspended Mar 30 15:22:34:650 [140605547460672] 3 hub: check_session - session 0x20f9050 (127.0.0.1/48002) is a server Mar 30 15:22:34:650 [140605547460672] 2 hub: Profile - clayk 38 bytes (OK) Mar 30 15:22:34:745 [140604042360576] 5 hub: There are 0 objects waiting on 8 threads Mar 30 15:22:35:265 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:36:652 [140605547460672] 4 hub: medium timeout Mar 30 15:22:36:652 [140605547460672] 3 hub: next_check [1585596396] now [1585596156] Mar 30 15:22:36:849 [140603740354304] 5 hub: TSESS-P-8 not internal command - thread will handle. mtype=100; sfd:23 Mar 30 15:22:36:849 [140603740354304] 2 hub: CTRL Secondary request 'license_list' passed to TSESS-P-8 [140603153159936] Mar 30 15:22:36:849 [140603153159936] 3 hub: TSESS-P-8-2 [Secondary] restarted (for remote) Mar 30 15:22:36:849 [140603153159936] 2 hub: TSESS-P-8 SID issued to id=distsrv hub=Secondary ip=10.x.x.x type=1 expire=2020-03-30 20:25:09 Mar 30 15:22:36:849 [140603153159936] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:22:36:849 [140603153159936] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:22:36:849 [140603153159936] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:22:36:849 [140603153159936] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/distsrv Mar 30 15:22:36:849 [140603153159936] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/distsrv 127.0.0.1/48010 Mar 30 15:22:36:849 [140603153159936] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/distsrv 127.0.0.1/48010 - 0 ms (nns) Mar 30 15:22:36:849 [140603153159936] 4 hub: CONNECT: 0x7fe0a0000d90(39) 127.0.0.1/41892->127.0.0.1/48010 Mar 30 15:22:36:849 [140603153159936] 3 hub: TSESS-P-8-2 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/distsrv (127.0.0.1/48010) Mar 30 15:22:36:849 [140603153159936] 1 hub: TSESS-P-8-2 [Secondary] received 100 'license_list' (314 bytes) Mar 30 15:22:36:850 [140605547460672] 2 hub: verify_login from ctrl 127.0.0.1/53204 Mar 30 15:22:36:850 [140605547460672] 1 hub: verify - [target] prid=distsrv cmd=license_list ip=127.0.0.1 perm=1 Mar 30 15:22:36:850 [140605547460672] 1 hub: verify - [source] id=distsrv ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:36:855 [140603153159936] 5 hub: TSESS-P-8-2 [Secondary] local 200 'OK' Mar 30 15:22:36:855 [140603153159936] 1 hub: TSESS-P-8-2 [Secondary] sent 200 'OK' (133 bytes, 0 ms) Mar 30 15:22:36:865 [140603153159936] 1 hub: TSESS-P-8-2 [Secondary] ssl 100 '_close' (327 bytes) Mar 30 15:22:36:865 [140603153159936] 3 hub: TSESS-P-8-2 [Secondary] is paired with [A-2121-2] Mar 30 15:22:36:865 [140603153159936] 1 hub: TSESS-P-8-2 [Secondary] received 100 '_close' (307 bytes) Mar 30 15:22:36:870 [140603153159936] 1 hub: TSESS-P-8-2 [Secondary] remote session closed Mar 30 15:22:36:870 [140603153159936] 3 hub: TSESS-P-8-2 [Secondary] exiting msg loop (remote close) Mar 30 15:22:36:870 [140603153159936] 5 hub: TSESS-P-8-2 [Secondary] suspended Mar 30 15:22:36:951 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:22:36:983 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:22:37:037 [140603203516160] 5 hub: TSESS-A-6-1 [Secondary] suspended Mar 30 15:22:37:068 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:37:069 [140603195123456] 5 hub: TSESS-A-3-1 [SecondaryHA] suspended Mar 30 15:22:37:734 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:22:37:734 [140603731961600] 2 hub: CTRL SecondaryHA request 'hubup' passed to TSESS-P-4 [140603161552640] Mar 30 15:22:37:734 [140603161552640] 3 hub: TSESS-P-4-2 [SecondaryHA] restarted (for remote) Mar 30 15:22:37:734 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:32:37 Mar 30 15:22:37:734 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:22:37:734 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:22:37:734 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:22:37:734 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:22:37:734 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:22:37:734 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:22:37:734 [140603161552640] 4 hub: CONNECT: 0x7fe09c000c80(39) 127.0.0.1/53206->127.0.0.1/48002 Mar 30 15:22:37:734 [140603161552640] 3 hub: TSESS-P-4-2 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:22:37:734 [140603161552640] 1 hub: TSESS-P-4-2 [SecondaryHA] received 100 'hubup' (687 bytes) Mar 30 15:22:37:734 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53206 (s:392) Mar 30 15:22:37:734 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/53206] Mar 30 15:22:37:734 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:22:37:734 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:22:37:734 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/53206 Mar 30 15:22:37:734 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/53206 same proximity=0 status=0 secver=217 Mar 30 15:22:37:739 [140603161552640] 5 hub: TSESS-P-4-2 [SecondaryHA] local 200 'OK' Mar 30 15:22:37:739 [140603161552640] 1 hub: TSESS-P-4-2 [SecondaryHA] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:22:37:746 [140603740354304] 5 hub: TSESS-P-8 not internal command - thread will handle. mtype=100; sfd:23 Mar 30 15:22:37:747 [140603740354304] 2 hub: CTRL Secondary request 'hubup' passed to TSESS-P-8 [140603153159936] Mar 30 15:22:37:747 [140603153159936] 3 hub: TSESS-P-8-3 [Secondary] restarted (for remote) Mar 30 15:22:37:747 [140603153159936] 2 hub: TSESS-P-8 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:32:37 Mar 30 15:22:37:747 [140603153159936] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:22:37:747 [140603153159936] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:22:37:747 [140603153159936] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:22:37:747 [140603153159936] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:22:37:747 [140603153159936] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:22:37:747 [140603153159936] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:22:37:747 [140603153159936] 4 hub: CONNECT: 0x7fe0a0000d90(42) 127.0.0.1/53208->127.0.0.1/48002 Mar 30 15:22:37:747 [140603153159936] 3 hub: TSESS-P-8-3 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:22:37:747 [140603153159936] 1 hub: TSESS-P-8-3 [Secondary] received 100 'hubup' (683 bytes) Mar 30 15:22:37:747 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53208 (s:388) Mar 30 15:22:37:747 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/53208] Mar 30 15:22:37:747 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:22:37:747 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:22:37:747 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/53208 Mar 30 15:22:37:747 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/53208 same proximity=0 status=0 secver=217 Mar 30 15:22:37:749 [140603161552640] 1 hub: TSESS-P-4-2 [SecondaryHA] ssl 100 '_close' (300 bytes) Mar 30 15:22:37:749 [140603161552640] 3 hub: TSESS-P-4-2 [SecondaryHA] is paired with [A-1948-2] Mar 30 15:22:37:749 [140603161552640] 1 hub: TSESS-P-4-2 [SecondaryHA] received 100 '_close' (296 bytes) Mar 30 15:22:37:752 [140603153159936] 5 hub: TSESS-P-8-3 [Secondary] local 200 'OK' Mar 30 15:22:37:752 [140603153159936] 1 hub: TSESS-P-8-3 [Secondary] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:22:37:755 [140603161552640] 1 hub: TSESS-P-4-2 [SecondaryHA] remote session closed Mar 30 15:22:37:755 [140603161552640] 3 hub: TSESS-P-4-2 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:22:37:755 [140603161552640] 5 hub: TSESS-P-4-2 [SecondaryHA] suspended Mar 30 15:22:37:762 [140603153159936] 1 hub: TSESS-P-8-3 [Secondary] ssl 100 '_close' (296 bytes) Mar 30 15:22:37:762 [140603153159936] 3 hub: TSESS-P-8-3 [Secondary] is paired with [A-2121-3] Mar 30 15:22:37:762 [140603153159936] 1 hub: TSESS-P-8-3 [Secondary] received 100 '_close' (296 bytes) Mar 30 15:22:37:767 [140603153159936] 1 hub: TSESS-P-8-3 [Secondary] remote session closed Mar 30 15:22:37:767 [140603153159936] 3 hub: TSESS-P-8-3 [Secondary] exiting msg loop (remote close) Mar 30 15:22:37:768 [140603153159936] 5 hub: TSESS-P-8-3 [Secondary] suspended Mar 30 15:22:38:573 [140603723568896] 5 hub: TSESS-P-7 not internal command - thread will handle. mtype=100; sfd:22 Mar 30 15:22:38:573 [140603723568896] 2 hub: CTRL TunnelB request 'hubup' passed to TSESS-P-7 [140603169945344] Mar 30 15:22:38:573 [140603169945344] 3 hub: TSESS-P-7-2 [TunnelB] restarted (for remote) Mar 30 15:22:38:573 [140603169945344] 2 hub: TSESS-P-7 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:32:38 Mar 30 15:22:38:573 [140603169945344] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:22:38:574 [140603169945344] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:22:38:574 [140603169945344] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:22:38:574 [140603169945344] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:22:38:574 [140603169945344] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:22:38:574 [140603169945344] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:22:38:574 [140603169945344] 4 hub: CONNECT: 0x7fe0a8000d20(39) 127.0.0.1/53210->127.0.0.1/48002 Mar 30 15:22:38:574 [140603169945344] 3 hub: TSESS-P-7-2 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:22:38:574 [140603169945344] 1 hub: TSESS-P-7-2 [TunnelB] received 100 'hubup' (672 bytes) Mar 30 15:22:38:574 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53210 (s:378) Mar 30 15:22:38:574 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/53210] Mar 30 15:22:38:574 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:22:38:574 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:22:38:574 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/53210 Mar 30 15:22:38:574 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/53210 same proximity=0 status=0 secver=217 Mar 30 15:22:38:579 [140603169945344] 5 hub: TSESS-P-7-2 [TunnelB] local 200 'OK' Mar 30 15:22:38:579 [140603169945344] 1 hub: TSESS-P-7-2 [TunnelB] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:22:38:589 [140603169945344] 1 hub: TSESS-P-7-2 [TunnelB] ssl 100 '_close' (283 bytes) Mar 30 15:22:38:589 [140603169945344] 3 hub: TSESS-P-7-2 [TunnelB] is paired with [A-299-2] Mar 30 15:22:38:589 [140603169945344] 1 hub: TSESS-P-7-2 [TunnelB] received 100 '_close' (295 bytes) Mar 30 15:22:38:594 [140603169945344] 1 hub: TSESS-P-7-2 [TunnelB] remote session closed Mar 30 15:22:38:594 [140603169945344] 3 hub: TSESS-P-7-2 [TunnelB] exiting msg loop (remote close) Mar 30 15:22:38:594 [140603169945344] 5 hub: TSESS-P-7-2 [TunnelB] suspended Mar 30 15:22:38:866 [140603740354304] 5 hub: TSESS-P-8 not internal command - thread will handle. mtype=100; sfd:23 Mar 30 15:22:38:866 [140603740354304] 2 hub: CTRL Secondary request 'archive_list' passed to TSESS-P-8 [140603153159936] Mar 30 15:22:38:866 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] restarted (for remote) Mar 30 15:22:38:866 [140603153159936] 2 hub: TSESS-P-8 SID issued to id=distsrv hub=Secondary ip=10.x.x.x type=1 expire=2020-03-30 20:25:09 Mar 30 15:22:38:866 [140603153159936] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:22:38:866 [140603153159936] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:22:38:866 [140603153159936] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:22:38:866 [140603153159936] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/distsrv Mar 30 15:22:38:866 [140603153159936] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/distsrv 127.0.0.1/48010 Mar 30 15:22:38:866 [140603153159936] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/distsrv 127.0.0.1/48010 - 0 ms (nns) Mar 30 15:22:38:866 [140603153159936] 4 hub: CONNECT: 0x7fe0a0000d90(39) 127.0.0.1/41904->127.0.0.1/48010 Mar 30 15:22:38:866 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/distsrv (127.0.0.1/48010) Mar 30 15:22:38:866 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (355 bytes) Mar 30 15:22:38:867 [140605547460672] 2 hub: verify_login from ctrl 127.0.0.1/53216 Mar 30 15:22:38:867 [140605547460672] 1 hub: verify - [target] prid=distsrv cmd=archive_list ip=127.0.0.1 perm=1 Mar 30 15:22:38:867 [140605547460672] 1 hub: verify - [source] id=distsrv ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:38:871 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:38:871 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (275 bytes, 0 ms) Mar 30 15:22:39:771 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:39:881 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (373 bytes) Mar 30 15:22:39:881 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:22:39:881 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (353 bytes) Mar 30 15:22:39:886 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:39:886 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (286 bytes, 0 ms) Mar 30 15:22:40:903 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (392 bytes) Mar 30 15:22:40:903 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:22:40:903 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (372 bytes) Mar 30 15:22:40:908 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:40:908 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (302 bytes, 0 ms) Mar 30 15:22:41:574 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:41:864 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:22:41:864 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:22:41:864 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60744 Mar 30 15:22:41:864 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60744' delegated to TSESS-A-5 [140603715176192] Mar 30 15:22:41:864 [140603715176192] 3 hub: TSESS-A-5-10 [TunnelB] restarted (for local) Mar 30 15:22:41:870 [140603715176192] 5 hub: TSESS-A-5-10 [TunnelB] local 100 'get_info' Mar 30 15:22:41:870 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:41:870 [140603715176192] 1 hub: TSESS-A-5-10 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:22:41:880 [140603715176192] 1 hub: TSESS-A-5-10 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:22:41:880 [140603715176192] 3 hub: TSESS-A-5-10 [TunnelB] is paired with [P-298-10] Mar 30 15:22:41:880 [140603715176192] 1 hub: TSESS-A-5-10 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:22:41:880 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60746 Mar 30 15:22:41:880 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60746' delegated to TSESS-A-10 [140603706783488] Mar 30 15:22:41:880 [140603706783488] 3 hub: TSESS-A-10-7 [TunnelB] restarted (for local) Mar 30 15:22:41:885 [140603715176192] 5 hub: TSESS-A-5-10 [TunnelB] local 100 '_close' Mar 30 15:22:41:885 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:41:885 [140603715176192] 1 hub: TSESS-A-5-10 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:22:41:885 [140603715176192] 3 hub: TSESS-A-5-10 [TunnelB] local session closed, suspending Mar 30 15:22:41:885 [140603706783488] 5 hub: TSESS-A-10-7 [TunnelB] local 100 '_status' Mar 30 15:22:41:885 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:41:885 [140603706783488] 1 hub: TSESS-A-10-7 [TunnelB] sent 100 '_status' (312 bytes, 0 ms) Mar 30 15:22:41:896 [140603706783488] 1 hub: TSESS-A-10-7 [TunnelB] ssl 200 '-' (279 bytes) Mar 30 15:22:41:896 [140603706783488] 3 hub: TSESS-A-10-7 [TunnelB] is paired with [P-300-7] Mar 30 15:22:41:896 [140603706783488] 1 hub: TSESS-A-10-7 [TunnelB] received 200 '-' (287 bytes) Mar 30 15:22:41:901 [140603706783488] 5 hub: TSESS-A-10-7 [TunnelB] local 100 '_close' Mar 30 15:22:41:901 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:41:901 [140603706783488] 1 hub: TSESS-A-10-7 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:22:41:901 [140603706783488] 3 hub: TSESS-A-10-7 [TunnelB] local session closed, suspending Mar 30 15:22:41:910 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (369 bytes) Mar 30 15:22:41:911 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:22:41:911 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (349 bytes) Mar 30 15:22:41:916 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:41:916 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (258 bytes, 0 ms) Mar 30 15:22:42:085 [140603715176192] 5 hub: TSESS-A-5-10 [TunnelB] suspended Mar 30 15:22:42:101 [140603706783488] 5 hub: TSESS-A-10-7 [TunnelB] suspended Mar 30 15:22:42:923 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (370 bytes) Mar 30 15:22:42:923 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:22:42:923 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (350 bytes) Mar 30 15:22:42:928 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:42:928 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (267 bytes, 0 ms) Mar 30 15:22:43:376 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:43:936 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (373 bytes) Mar 30 15:22:43:936 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:22:43:936 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (353 bytes) Mar 30 15:22:43:941 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:43:941 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (270 bytes, 0 ms) Mar 30 15:22:44:948 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (368 bytes) Mar 30 15:22:44:948 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:22:44:948 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (348 bytes) Mar 30 15:22:44:953 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:44:953 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (264 bytes, 0 ms) Mar 30 15:22:45:179 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:45:958 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (369 bytes) Mar 30 15:22:45:958 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:22:45:959 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (349 bytes) Mar 30 15:22:45:964 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:45:964 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (259 bytes, 0 ms) Mar 30 15:22:46:897 [140605283890944] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:22:46:897 [140605283890944] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:22:46:897 [140605283890944] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:22:46:897 [140605283890944] 1 hub: PERF: hubup_check_interval: 0 sec hubhash_size: 4 Mar 30 15:22:46:897 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:22:46:897 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:22:46:897 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:22:46:897 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:22:46:897 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(40) 127.0.0.1/46890->127.0.0.1/48007 Mar 30 15:22:46:898 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/46890 Mar 30 15:22:46:898 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/46890' delegated to TSESS-A-6 [140603203516160] Mar 30 15:22:46:898 [140603203516160] 3 hub: TSESS-A-6-2 [Secondary] restarted (for local) Mar 30 15:22:46:903 [140603203516160] 5 hub: TSESS-A-6-2 [Secondary] local 100 'hubup' Mar 30 15:22:46:903 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:46:903 [140603203516160] 1 hub: TSESS-A-6-2 [Secondary] sent 100 'hubup' (686 bytes, 0 ms) Mar 30 15:22:46:913 [140603203516160] 1 hub: TSESS-A-6-2 [Secondary] ssl 200 '-' (453 bytes) Mar 30 15:22:46:913 [140603203516160] 3 hub: TSESS-A-6-2 [Secondary] is paired with [P-2120-2] Mar 30 15:22:46:913 [140603203516160] 1 hub: TSESS-A-6-2 [Secondary] received 200 '-' (461 bytes) Mar 30 15:22:46:913 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:22:46:913 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:22:46:913 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:22:46:913 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:22:46:913 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary same proximity=0 status=0 secver=217 Mar 30 15:22:46:913 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:22:46:913 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:22:46:913 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:22:46:913 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:22:46:913 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(40) 127.0.0.1/60754->127.0.0.1/48009 Mar 30 15:22:46:913 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60754 Mar 30 15:22:46:913 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60754' delegated to TSESS-A-5 [140603715176192] Mar 30 15:22:46:914 [140603715176192] 3 hub: TSESS-A-5-11 [TunnelB] restarted (for local) Mar 30 15:22:46:918 [140603203516160] 5 hub: TSESS-A-6-2 [Secondary] local 100 '_close' Mar 30 15:22:46:918 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:46:918 [140603203516160] 1 hub: TSESS-A-6-2 [Secondary] sent 100 '_close' (293 bytes, 0 ms) Mar 30 15:22:46:918 [140603203516160] 3 hub: TSESS-A-6-2 [Secondary] local session closed, suspending Mar 30 15:22:46:919 [140603715176192] 5 hub: TSESS-A-5-11 [TunnelB] local 100 'hubup' Mar 30 15:22:46:919 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:46:919 [140603715176192] 1 hub: TSESS-A-5-11 [TunnelB] sent 100 'hubup' (685 bytes, 0 ms) Mar 30 15:22:46:929 [140603715176192] 1 hub: TSESS-A-5-11 [TunnelB] ssl 200 '-' (443 bytes) Mar 30 15:22:46:929 [140603715176192] 3 hub: TSESS-A-5-11 [TunnelB] is paired with [P-298-11] Mar 30 15:22:46:929 [140603715176192] 1 hub: TSESS-A-5-11 [TunnelB] received 200 '-' (451 bytes) Mar 30 15:22:46:929 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:22:46:929 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:22:46:929 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:22:46:929 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:22:46:929 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB same proximity=0 status=0 secver=217 Mar 30 15:22:46:929 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:22:46:929 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:22:46:929 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:22:46:929 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:22:46:929 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(40) 127.0.0.1/39942->127.0.0.1/48008 Mar 30 15:22:46:929 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/39942 Mar 30 15:22:46:929 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/39942' delegated to TSESS-A-3 [140603195123456] Mar 30 15:22:46:929 [140603195123456] 3 hub: TSESS-A-3-2 [SecondaryHA] restarted (for local) Mar 30 15:22:46:934 [140603715176192] 5 hub: TSESS-A-5-11 [TunnelB] local 100 '_close' Mar 30 15:22:46:934 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:46:934 [140603715176192] 1 hub: TSESS-A-5-11 [TunnelB] sent 100 '_close' (292 bytes, 0 ms) Mar 30 15:22:46:934 [140603715176192] 3 hub: TSESS-A-5-11 [TunnelB] local session closed, suspending Mar 30 15:22:46:935 [140603195123456] 5 hub: TSESS-A-3-2 [SecondaryHA] local 100 'hubup' Mar 30 15:22:46:935 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:46:935 [140603195123456] 1 hub: TSESS-A-3-2 [SecondaryHA] sent 100 'hubup' (688 bytes, 0 ms) Mar 30 15:22:46:942 [140605547460672] 4 hub: medium timeout Mar 30 15:22:46:942 [140605547460672] 3 hub: next_check [1585596396] now [1585596166] Mar 30 15:22:46:945 [140603195123456] 1 hub: TSESS-A-3-2 [SecondaryHA] ssl 200 '-' (457 bytes) Mar 30 15:22:46:945 [140603195123456] 3 hub: TSESS-A-3-2 [SecondaryHA] is paired with [P-1947-2] Mar 30 15:22:46:945 [140603195123456] 1 hub: TSESS-A-3-2 [SecondaryHA] received 200 '-' (465 bytes) Mar 30 15:22:46:945 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:22:46:945 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:22:46:945 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:22:46:945 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:22:46:945 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA same proximity=0 status=0 secver=217 Mar 30 15:22:46:945 [140605283890944] 5 hub: mark_hubup_list_updated Mar 30 15:22:46:945 [140605283890944] 5 hub: hublist changed: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:22:46:945 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:22:46:945 [140605283890944] 5 hub: hublist changed: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:22:46:945 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelBHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:22:46:945 [140605283890944] 5 hub: mark_hubup_list_updated end Mar 30 15:22:46:945 [140605283890944] 1 hub: PERF: hubup_check_for_updates: 0 sec hubhash_size: 4 Mar 30 15:22:46:945 [140605283890944] 1 hub: PERF: hubi_run_command: 0 sec hubhash_size: 4 Mar 30 15:22:46:945 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:22:46:946 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:22:46:946 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:22:46:946 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:22:46:946 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:22:46:947 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:22:46:947 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:22:46:948 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:22:46:948 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:22:46:948 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:22:46:948 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:22:46:949 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:22:46:949 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:22:46:950 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:22:46:950 [140603195123456] 5 hub: TSESS-A-3-2 [SecondaryHA] local 100 '_close' Mar 30 15:22:46:950 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:46:950 [140603195123456] 1 hub: TSESS-A-3-2 [SecondaryHA] sent 100 '_close' (295 bytes, 0 ms) Mar 30 15:22:46:950 [140603195123456] 3 hub: TSESS-A-3-2 [SecondaryHA] local session closed, suspending Mar 30 15:22:46:950 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:22:46:951 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:22:46:952 [140605283890944] 1 hub: PERF: hubi_check_security: 0 sec hubhash_size: 4 Mar 30 15:22:46:952 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:22:46:962 [140605547460672] 2 hub: verify_login from ctrl 127.0.0.1/53252 Mar 30 15:22:46:962 [140605547460672] 1 hub: verify - [target] prid=controller cmd=probe_list ip=10.x.x.x perm=1 Mar 30 15:22:46:962 [140605547460672] 1 hub: verify - [source] id=ppm ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:46:975 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (369 bytes) Mar 30 15:22:46:975 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:22:46:975 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (349 bytes) Mar 30 15:22:46:981 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:46:981 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (254 bytes, 0 ms) Mar 30 15:22:46:984 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:22:47:119 [140603203516160] 5 hub: TSESS-A-6-2 [Secondary] suspended Mar 30 15:22:47:135 [140603715176192] 5 hub: TSESS-A-5-11 [TunnelB] suspended Mar 30 15:22:47:150 [140603195123456] 5 hub: TSESS-A-3-2 [SecondaryHA] suspended Mar 30 15:22:47:755 [140603161552640] 5 hub: TSESS-P-4-2 [SecondaryHA] suspended Mar 30 15:22:47:883 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:47:982 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (373 bytes) Mar 30 15:22:47:982 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:22:47:982 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (353 bytes) Mar 30 15:22:47:987 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:47:987 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (264 bytes, 0 ms) Mar 30 15:22:48:594 [140603169945344] 5 hub: TSESS-P-7-2 [TunnelB] suspended Mar 30 15:22:48:992 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (377 bytes) Mar 30 15:22:48:992 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:22:48:992 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (357 bytes) Mar 30 15:22:48:998 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:48:998 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (274 bytes, 0 ms) Mar 30 15:22:49:685 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:50:008 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (370 bytes) Mar 30 15:22:50:008 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:22:50:008 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (350 bytes) Mar 30 15:22:50:013 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:50:013 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (256 bytes, 0 ms) Mar 30 15:22:51:034 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (378 bytes) Mar 30 15:22:51:035 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:22:51:035 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (358 bytes) Mar 30 15:22:51:040 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:51:040 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (291 bytes, 0 ms) Mar 30 15:22:51:488 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:51:908 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:22:51:908 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:22:51:908 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60780 Mar 30 15:22:51:908 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60780' delegated to TSESS-A-5 [140603715176192] Mar 30 15:22:51:908 [140603715176192] 3 hub: TSESS-A-5-12 [TunnelB] restarted (for local) Mar 30 15:22:51:913 [140603715176192] 5 hub: TSESS-A-5-12 [TunnelB] local 100 'get_info' Mar 30 15:22:51:913 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:51:913 [140603715176192] 1 hub: TSESS-A-5-12 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:22:51:924 [140603715176192] 1 hub: TSESS-A-5-12 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:22:51:924 [140603715176192] 3 hub: TSESS-A-5-12 [TunnelB] is paired with [P-298-12] Mar 30 15:22:51:924 [140603715176192] 1 hub: TSESS-A-5-12 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:22:51:924 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60782 Mar 30 15:22:51:924 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60782' delegated to TSESS-A-10 [140603706783488] Mar 30 15:22:51:924 [140603706783488] 3 hub: TSESS-A-10-8 [TunnelB] restarted (for local) Mar 30 15:22:51:929 [140603715176192] 5 hub: TSESS-A-5-12 [TunnelB] local 100 '_close' Mar 30 15:22:51:929 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:51:929 [140603715176192] 1 hub: TSESS-A-5-12 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:22:51:929 [140603715176192] 3 hub: TSESS-A-5-12 [TunnelB] local session closed, suspending Mar 30 15:22:51:929 [140603706783488] 5 hub: TSESS-A-10-8 [TunnelB] local 100 '_status' Mar 30 15:22:51:929 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:51:929 [140603706783488] 1 hub: TSESS-A-10-8 [TunnelB] sent 100 '_status' (312 bytes, 0 ms) Mar 30 15:22:51:939 [140603706783488] 1 hub: TSESS-A-10-8 [TunnelB] ssl 200 '-' (279 bytes) Mar 30 15:22:51:939 [140603706783488] 3 hub: TSESS-A-10-8 [TunnelB] is paired with [P-300-8] Mar 30 15:22:51:939 [140603706783488] 1 hub: TSESS-A-10-8 [TunnelB] received 200 '-' (287 bytes) Mar 30 15:22:51:944 [140603706783488] 5 hub: TSESS-A-10-8 [TunnelB] local 100 '_close' Mar 30 15:22:51:945 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:22:51:945 [140603706783488] 1 hub: TSESS-A-10-8 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:22:51:945 [140603706783488] 3 hub: TSESS-A-10-8 [TunnelB] local session closed, suspending Mar 30 15:22:52:047 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (391 bytes) Mar 30 15:22:52:047 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:22:52:047 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (371 bytes) Mar 30 15:22:52:052 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:52:052 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (345 bytes, 0 ms) Mar 30 15:22:52:129 [140603715176192] 5 hub: TSESS-A-5-12 [TunnelB] suspended Mar 30 15:22:52:145 [140603706783488] 5 hub: TSESS-A-10-8 [TunnelB] suspended Mar 30 15:22:53:056 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (374 bytes) Mar 30 15:22:53:056 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:22:53:056 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (354 bytes) Mar 30 15:22:53:061 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:53:062 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (281 bytes, 0 ms) Mar 30 15:22:53:291 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:54:071 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (377 bytes) Mar 30 15:22:54:071 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:22:54:071 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (357 bytes) Mar 30 15:22:54:076 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:54:076 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (277 bytes, 0 ms) Mar 30 15:22:55:084 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (378 bytes) Mar 30 15:22:55:084 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:22:55:084 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (358 bytes) Mar 30 15:22:55:089 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:55:089 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (279 bytes, 0 ms) Mar 30 15:22:55:093 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:56:096 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (390 bytes) Mar 30 15:22:56:096 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:22:56:096 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (370 bytes) Mar 30 15:22:56:101 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:56:101 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (326 bytes, 0 ms) Mar 30 15:22:56:914 [140605547460672] 4 hub: medium timeout Mar 30 15:22:56:914 [140605547460672] 3 hub: next_check [1585596396] now [1585596176] Mar 30 15:22:56:952 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:22:56:984 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:22:57:108 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (388 bytes) Mar 30 15:22:57:108 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:22:57:108 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (368 bytes) Mar 30 15:22:57:113 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:57:113 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (320 bytes, 0 ms) Mar 30 15:22:57:119 [140603203516160] 5 hub: TSESS-A-6-2 [Secondary] suspended Mar 30 15:22:57:151 [140603195123456] 5 hub: TSESS-A-3-2 [SecondaryHA] suspended Mar 30 15:22:57:755 [140603161552640] 5 hub: TSESS-P-4-2 [SecondaryHA] suspended Mar 30 15:22:57:797 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:22:58:114 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (391 bytes) Mar 30 15:22:58:114 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:22:58:114 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (371 bytes) Mar 30 15:22:58:120 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:58:120 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (317 bytes, 0 ms) Mar 30 15:22:58:595 [140603169945344] 5 hub: TSESS-P-7-2 [TunnelB] suspended Mar 30 15:22:59:130 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (391 bytes) Mar 30 15:22:59:130 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:22:59:130 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (371 bytes) Mar 30 15:22:59:135 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:22:59:135 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (323 bytes, 0 ms) Mar 30 15:22:59:599 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:00:145 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (392 bytes) Mar 30 15:23:00:145 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:00:145 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (372 bytes) Mar 30 15:23:00:150 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:00:150 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (326 bytes, 0 ms) Mar 30 15:23:01:158 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (388 bytes) Mar 30 15:23:01:158 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:01:158 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (368 bytes) Mar 30 15:23:01:163 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:01:163 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (320 bytes, 0 ms) Mar 30 15:23:01:402 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:01:951 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:23:01:951 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:23:01:951 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60788 Mar 30 15:23:01:951 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60788' delegated to TSESS-A-5 [140603715176192] Mar 30 15:23:01:951 [140603715176192] 3 hub: TSESS-A-5-13 [TunnelB] restarted (for local) Mar 30 15:23:01:956 [140603715176192] 5 hub: TSESS-A-5-13 [TunnelB] local 100 'get_info' Mar 30 15:23:01:956 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:01:956 [140603715176192] 1 hub: TSESS-A-5-13 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:23:01:967 [140603715176192] 1 hub: TSESS-A-5-13 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:23:01:967 [140603715176192] 3 hub: TSESS-A-5-13 [TunnelB] is paired with [P-298-13] Mar 30 15:23:01:967 [140603715176192] 1 hub: TSESS-A-5-13 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:23:01:967 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60790 Mar 30 15:23:01:967 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60790' delegated to TSESS-A-10 [140603706783488] Mar 30 15:23:01:967 [140603706783488] 3 hub: TSESS-A-10-9 [TunnelB] restarted (for local) Mar 30 15:23:01:972 [140603715176192] 5 hub: TSESS-A-5-13 [TunnelB] local 100 '_close' Mar 30 15:23:01:972 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:01:972 [140603715176192] 1 hub: TSESS-A-5-13 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:23:01:972 [140603715176192] 3 hub: TSESS-A-5-13 [TunnelB] local session closed, suspending Mar 30 15:23:01:972 [140603706783488] 5 hub: TSESS-A-10-9 [TunnelB] local 100 '_status' Mar 30 15:23:01:972 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:01:972 [140603706783488] 1 hub: TSESS-A-10-9 [TunnelB] sent 100 '_status' (312 bytes, 0 ms) Mar 30 15:23:01:983 [140603706783488] 1 hub: TSESS-A-10-9 [TunnelB] ssl 200 '-' (279 bytes) Mar 30 15:23:01:983 [140603706783488] 3 hub: TSESS-A-10-9 [TunnelB] is paired with [P-300-9] Mar 30 15:23:01:983 [140603706783488] 1 hub: TSESS-A-10-9 [TunnelB] received 200 '-' (287 bytes) Mar 30 15:23:01:988 [140603706783488] 5 hub: TSESS-A-10-9 [TunnelB] local 100 '_close' Mar 30 15:23:01:988 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:01:988 [140603706783488] 1 hub: TSESS-A-10-9 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:23:01:988 [140603706783488] 3 hub: TSESS-A-10-9 [TunnelB] local session closed, suspending Mar 30 15:23:02:169 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (377 bytes) Mar 30 15:23:02:169 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:02:169 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (357 bytes) Mar 30 15:23:02:172 [140603715176192] 5 hub: TSESS-A-5-13 [TunnelB] suspended Mar 30 15:23:02:174 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:02:175 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (283 bytes, 0 ms) Mar 30 15:23:02:188 [140603706783488] 5 hub: TSESS-A-10-9 [TunnelB] suspended Mar 30 15:23:03:175 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (398 bytes) Mar 30 15:23:03:175 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:03:175 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (378 bytes) Mar 30 15:23:03:181 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:03:181 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (336 bytes, 0 ms) Mar 30 15:23:03:205 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:04:189 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (396 bytes) Mar 30 15:23:04:189 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:04:189 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (376 bytes) Mar 30 15:23:04:194 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:04:194 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (328 bytes, 0 ms) Mar 30 15:23:04:954 [140605547460672] 3 hub: check_session - session 0x20f9050 (127.0.0.1/48002) is a server Mar 30 15:23:04:954 [140605547460672] 2 hub: Profile - clayk 38 bytes (OK) Mar 30 15:23:05:008 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:05:201 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (385 bytes) Mar 30 15:23:05:201 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:05:201 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (365 bytes) Mar 30 15:23:05:206 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:05:206 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (327 bytes, 0 ms) Mar 30 15:23:06:214 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (376 bytes) Mar 30 15:23:06:214 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:06:214 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (356 bytes) Mar 30 15:23:06:219 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:06:219 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (294 bytes, 0 ms) Mar 30 15:23:06:952 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:23:06:956 [140605547460672] 4 hub: medium timeout Mar 30 15:23:06:956 [140605547460672] 3 hub: next_check [1585596396] now [1585596186] Mar 30 15:23:06:984 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:23:07:119 [140603203516160] 5 hub: TSESS-A-6-2 [Secondary] suspended Mar 30 15:23:07:151 [140603195123456] 5 hub: TSESS-A-3-2 [SecondaryHA] suspended Mar 30 15:23:07:227 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (375 bytes) Mar 30 15:23:07:227 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:07:227 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (355 bytes) Mar 30 15:23:07:232 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:07:232 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (520 bytes, 0 ms) Mar 30 15:23:07:242 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (375 bytes) Mar 30 15:23:07:242 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:07:242 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (355 bytes) Mar 30 15:23:07:247 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:07:247 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (520 bytes, 0 ms) Mar 30 15:23:07:711 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:07:755 [140603161552640] 5 hub: TSESS-P-4-2 [SecondaryHA] suspended Mar 30 15:23:07:832 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:23:07:832 [140603731961600] 2 hub: CTRL SecondaryHA request 'hubup' passed to TSESS-P-4 [140603161552640] Mar 30 15:23:07:832 [140603161552640] 3 hub: TSESS-P-4-3 [SecondaryHA] restarted (for remote) Mar 30 15:23:07:832 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:33:07 Mar 30 15:23:07:832 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:23:07:832 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:23:07:832 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:23:07:832 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:23:07:832 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:23:07:832 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:23:07:832 [140603161552640] 4 hub: CONNECT: 0x7fe09c000d30(41) 127.0.0.1/53272->127.0.0.1/48002 Mar 30 15:23:07:832 [140603161552640] 3 hub: TSESS-P-4-3 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:23:07:832 [140603161552640] 1 hub: TSESS-P-4-3 [SecondaryHA] received 100 'hubup' (8318 bytes) Mar 30 15:23:07:832 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53272 (s:8023) Mar 30 15:23:07:832 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/53272] Mar 30 15:23:07:833 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/53272 same proximity=0 status=0 secver=217 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-9] domain [DOMAIN] addr [(null)] ip [100.64.1.19] port [48002] robotname [a0f8us052nim749] version [9.20 [Build 9.20.1048, Aug 8 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-9] source [/DOMAIN/SecondaryHA] from [127.0.0.1/53272] Mar 30 15:23:07:833 [140605547460672] 4 hub: hubup source: /DOMAIN/SecondaryHA Mar 30 15:23:07:833 [140605547460672] 4 hub: hubinfo create: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 0x2105350 Mar 30 15:23:07:833 [140605547460672] 1 hub: hubinfo_update - new hubinfo for /DOMAIN/A0F8-USDA-FMMI-9 p=3 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/A0F8-USDA-FMMI-9/a0f8us052nim749/hub, from 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 2 hub: hubinfo_update - DOMAIN/A0F8-USDA-FMMI-9 from=127.0.0.1/53272 CHANGED proximity=3 status=0 secver=217 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update source - lookup /DOMAIN/SecondaryHA ok Mar 30 15:23:07:833 [140605547460672] 2 hub: ch_hublist /DOMAIN/A0F8-USDA-FMMI-9/a0f8us052nim749/hub Mar 30 15:23:07:833 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-4] domain [DOMAIN] addr [(null)] ip [100.64.1.14] port [48002] robotname [a0f8us052nim744] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-4] source [/DOMAIN/SecondaryHA] from [127.0.0.1/53272] Mar 30 15:23:07:833 [140605547460672] 4 hub: hubup source: /DOMAIN/SecondaryHA Mar 30 15:23:07:833 [140605547460672] 4 hub: hubinfo create: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 0x2106190 Mar 30 15:23:07:833 [140605547460672] 1 hub: hubinfo_update - new hubinfo for /DOMAIN/A0F8-USDA-FMMI-4 p=3 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/A0F8-USDA-FMMI-4/a0f8us052nim744/hub, from 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 2 hub: hubinfo_update - DOMAIN/A0F8-USDA-FMMI-4 from=127.0.0.1/53272 CHANGED proximity=3 status=0 secver=217 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update source - lookup /DOMAIN/SecondaryHA ok Mar 30 15:23:07:833 [140605547460672] 2 hub: ch_hublist /DOMAIN/A0F8-USDA-FMMI-4/a0f8us052nim744/hub Mar 30 15:23:07:833 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - name [Primary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [a0a4us052nim701] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Primary] source [/DOMAIN/SecondaryHA] from [127.0.0.1/53272] Mar 30 15:23:07:833 [140605547460672] 4 hub: hubup source: /DOMAIN/SecondaryHA Mar 30 15:23:07:833 [140605547460672] 4 hub: hubinfo create: /DOMAIN/Primary 10.x.x.x/48002 0x21431f0 Mar 30 15:23:07:833 [140605547460672] 1 hub: hubinfo_update - new hubinfo for /DOMAIN/Primary p=1 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Primary/a0a4us052nim701/hub, from 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Primary from=127.0.0.1/53272 CHANGED proximity=1 status=0 secver=217 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update source - lookup /DOMAIN/SecondaryHA ok Mar 30 15:23:07:833 [140605547460672] 2 hub: ch_hublist /DOMAIN/Primary/a0a4us052nim701/hub Mar 30 15:23:07:833 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-5] domain [DOMAIN] addr [(null)] ip [100.64.1.15] port [48002] robotname [a0f8us052nim745] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-5] source [/DOMAIN/SecondaryHA] from [127.0.0.1/53272] Mar 30 15:23:07:833 [140605547460672] 4 hub: hubup source: /DOMAIN/SecondaryHA Mar 30 15:23:07:833 [140605547460672] 4 hub: hubinfo create: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 0x2143690 Mar 30 15:23:07:833 [140605547460672] 1 hub: hubinfo_update - new hubinfo for /DOMAIN/A0F8-USDA-FMMI-5 p=3 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/A0F8-USDA-FMMI-5/a0f8us052nim745/hub, from 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 2 hub: hubinfo_update - DOMAIN/A0F8-USDA-FMMI-5 from=127.0.0.1/53272 CHANGED proximity=3 status=0 secver=217 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update source - lookup /DOMAIN/SecondaryHA ok Mar 30 15:23:07:833 [140605547460672] 2 hub: ch_hublist /DOMAIN/A0F8-USDA-FMMI-5/a0f8us052nim745/hub Mar 30 15:23:07:833 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-6] domain [DOMAIN] addr [(null)] ip [100.64.1.16] port [48002] robotname [a0f8us052nim746] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-6] source [/DOMAIN/SecondaryHA] from [127.0.0.1/53272] Mar 30 15:23:07:833 [140605547460672] 4 hub: hubup source: /DOMAIN/SecondaryHA Mar 30 15:23:07:833 [140605547460672] 4 hub: hubinfo create: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 0x2106a40 Mar 30 15:23:07:833 [140605547460672] 1 hub: hubinfo_update - new hubinfo for /DOMAIN/A0F8-USDA-FMMI-6 p=3 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/A0F8-USDA-FMMI-6/a0f8us052nim746/hub, from 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 2 hub: hubinfo_update - DOMAIN/A0F8-USDA-FMMI-6 from=127.0.0.1/53272 CHANGED proximity=3 status=0 secver=217 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update source - lookup /DOMAIN/SecondaryHA ok Mar 30 15:23:07:833 [140605547460672] 2 hub: ch_hublist /DOMAIN/A0F8-USDA-FMMI-6/a0f8us052nim746/hub Mar 30 15:23:07:833 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-1] domain [DOMAIN] addr [(null)] ip [100.64.1.11] port [48002] robotname [a0f8us052nim741] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-1] source [/DOMAIN/SecondaryHA] from [127.0.0.1/53272] Mar 30 15:23:07:833 [140605547460672] 4 hub: hubup source: /DOMAIN/SecondaryHA Mar 30 15:23:07:833 [140605547460672] 4 hub: hubinfo create: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 0x2147b30 Mar 30 15:23:07:833 [140605547460672] 1 hub: hubinfo_update - new hubinfo for /DOMAIN/A0F8-USDA-FMMI-1 p=2 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/A0F8-USDA-FMMI-1/a0f8us052nim741/hub, from 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 2 hub: hubinfo_update - DOMAIN/A0F8-USDA-FMMI-1 from=127.0.0.1/53272 CHANGED proximity=2 status=0 secver=217 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update source - lookup /DOMAIN/SecondaryHA ok Mar 30 15:23:07:833 [140605547460672] 2 hub: ch_hublist /DOMAIN/A0F8-USDA-FMMI-1/a0f8us052nim741/hub Mar 30 15:23:07:833 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-10] domain [DOMAIN] addr [(null)] ip [100.64.1.20] port [48002] robotname [a0f8us052nim750] version [9.20 [Build 9.20.1048, Aug 8 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-10] source [/DOMAIN/SecondaryHA] from [127.0.0.1/53272] Mar 30 15:23:07:833 [140605547460672] 4 hub: hubup source: /DOMAIN/SecondaryHA Mar 30 15:23:07:833 [140605547460672] 4 hub: hubinfo create: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 0x21487f0 Mar 30 15:23:07:833 [140605547460672] 1 hub: hubinfo_update - new hubinfo for /DOMAIN/A0F8-USDA-FMMI-10 p=3 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/A0F8-USDA-FMMI-10/a0f8us052nim750/hub, from 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 2 hub: hubinfo_update - DOMAIN/A0F8-USDA-FMMI-10 from=127.0.0.1/53272 CHANGED proximity=3 status=0 secver=217 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update source - lookup /DOMAIN/SecondaryHA ok Mar 30 15:23:07:833 [140605547460672] 2 hub: ch_hublist /DOMAIN/A0F8-USDA-FMMI-10/a0f8us052nim750/hub Mar 30 15:23:07:833 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-7] domain [DOMAIN] addr [(null)] ip [100.64.1.17] port [48002] robotname [a0f8us052nim747] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-7] source [/DOMAIN/SecondaryHA] from [127.0.0.1/53272] Mar 30 15:23:07:833 [140605547460672] 4 hub: hubup source: /DOMAIN/SecondaryHA Mar 30 15:23:07:833 [140605547460672] 4 hub: hubinfo create: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 0x2149690 Mar 30 15:23:07:833 [140605547460672] 1 hub: hubinfo_update - new hubinfo for /DOMAIN/A0F8-USDA-FMMI-7 p=3 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/A0F8-USDA-FMMI-7/a0f8us052nim747/hub, from 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 2 hub: hubinfo_update - DOMAIN/A0F8-USDA-FMMI-7 from=127.0.0.1/53272 CHANGED proximity=3 status=0 secver=217 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update source - lookup /DOMAIN/SecondaryHA ok Mar 30 15:23:07:833 [140605547460672] 2 hub: ch_hublist /DOMAIN/A0F8-USDA-FMMI-7/a0f8us052nim747/hub Mar 30 15:23:07:833 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-2] domain [DOMAIN] addr [(null)] ip [100.64.1.12] port [48002] robotname [a0f8us052nim742] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-2] source [/DOMAIN/SecondaryHA] from [127.0.0.1/53272] Mar 30 15:23:07:833 [140605547460672] 4 hub: hubup source: /DOMAIN/SecondaryHA Mar 30 15:23:07:833 [140605547460672] 4 hub: hubinfo create: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 0x2144350 Mar 30 15:23:07:833 [140605547460672] 1 hub: hubinfo_update - new hubinfo for /DOMAIN/A0F8-USDA-FMMI-2 p=3 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/A0F8-USDA-FMMI-2/a0f8us052nim742/hub, from 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 2 hub: hubinfo_update - DOMAIN/A0F8-USDA-FMMI-2 from=127.0.0.1/53272 CHANGED proximity=3 status=0 secver=217 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update source - lookup /DOMAIN/SecondaryHA ok Mar 30 15:23:07:833 [140605547460672] 2 hub: ch_hublist /DOMAIN/A0F8-USDA-FMMI-2/a0f8us052nim742/hub Mar 30 15:23:07:833 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - name [TunnelA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelA] source [/DOMAIN/SecondaryHA] from [127.0.0.1/53272] Mar 30 15:23:07:833 [140605547460672] 4 hub: hubup source: /DOMAIN/SecondaryHA Mar 30 15:23:07:833 [140605547460672] 4 hub: hubinfo create: /DOMAIN/TunnelA 10.x.x.x/48002 0x21490d0 Mar 30 15:23:07:833 [140605547460672] 1 hub: hubinfo_update - new hubinfo for /DOMAIN/TunnelA p=1 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelA/ROBOT_NAME/hub, from 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelA from=127.0.0.1/53272 CHANGED proximity=1 status=0 secver=217 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update source - lookup /DOMAIN/SecondaryHA ok Mar 30 15:23:07:833 [140605547460672] 2 hub: ch_hublist /DOMAIN/TunnelA/ROBOT_NAME/hub Mar 30 15:23:07:833 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - name [PrimaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [a0a4us052nim702] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [PrimaryHA] source [/DOMAIN/SecondaryHA] from [127.0.0.1/53272] Mar 30 15:23:07:833 [140605547460672] 4 hub: hubup source: /DOMAIN/SecondaryHA Mar 30 15:23:07:833 [140605547460672] 4 hub: hubinfo create: /DOMAIN/PrimaryHA 10.x.x.x/48002 0x2149f40 Mar 30 15:23:07:833 [140605547460672] 1 hub: hubinfo_update - new hubinfo for /DOMAIN/PrimaryHA p=1 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/PrimaryHA/a0a4us052nim702/hub, from 127.0.0.1/53272 Mar 30 15:23:07:833 [140605547460672] 2 hub: hubinfo_update - DOMAIN/PrimaryHA from=127.0.0.1/53272 CHANGED proximity=1 status=0 secver=217 Mar 30 15:23:07:833 [140605547460672] 5 hub: hubinfo_update source - lookup /DOMAIN/SecondaryHA ok Mar 30 15:23:07:834 [140605547460672] 2 hub: ch_hublist /DOMAIN/PrimaryHA/a0a4us052nim702/hub Mar 30 15:23:07:834 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:23:07:834 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-11] domain [DOMAIN] addr [(null)] ip [100.64.1.21] port [48002] robotname [a0f8us052nim751] version [9.20 [Build 9.20.1048, Aug 8 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-11] source [/DOMAIN/SecondaryHA] from [127.0.0.1/53272] Mar 30 15:23:07:834 [140605547460672] 4 hub: hubup source: /DOMAIN/SecondaryHA Mar 30 15:23:07:834 [140605547460672] 4 hub: hubinfo create: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 0x214b120 Mar 30 15:23:07:834 [140605547460672] 1 hub: hubinfo_update - new hubinfo for /DOMAIN/A0F8-USDA-FMMI-11 p=3 127.0.0.1/53272 Mar 30 15:23:07:834 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/A0F8-USDA-FMMI-11/a0f8us052nim751/hub, from 127.0.0.1/53272 Mar 30 15:23:07:834 [140605547460672] 2 hub: hubinfo_update - DOMAIN/A0F8-USDA-FMMI-11 from=127.0.0.1/53272 CHANGED proximity=3 status=0 secver=2 Mar 30 15:23:07:834 [140605547460672] 5 hub: hubinfo_update source - lookup /DOMAIN/SecondaryHA ok Mar 30 15:23:07:834 [140605547460672] 2 hub: ch_hublist /DOMAIN/A0F8-USDA-FMMI-11/a0f8us052nim751/hub Mar 30 15:23:07:834 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:23:07:834 [140603740354304] 5 hub: TSESS-P-14 not internal command - thread will handle. mtype=100; sfd:29 Mar 30 15:23:07:834 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-8] domain [DOMAIN] addr [(null)] ip [100.64.1.18] port [48002] robotname [a0f8us052nim748] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-8] source [/DOMAIN/SecondaryHA] from [127.0.0.1/53272] Mar 30 15:23:07:834 [140605547460672] 4 hub: hubup source: /DOMAIN/SecondaryHA Mar 30 15:23:07:834 [140605547460672] 4 hub: hubinfo create: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 0x214bac0 Mar 30 15:23:07:834 [140605547460672] 1 hub: hubinfo_update - new hubinfo for /DOMAIN/A0F8-USDA-FMMI-8 p=3 127.0.0.1/53272 Mar 30 15:23:07:834 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/A0F8-USDA-FMMI-8/a0f8us052nim748/hub, from 127.0.0.1/53272 Mar 30 15:23:07:834 [140605547460672] 2 hub: hubinfo_update - DOMAIN/A0F8-USDA-FMMI-8 from=127.0.0.1/53272 CHANGED proximity=3 status=0 secver=217 Mar 30 15:23:07:834 [140605547460672] 5 hub: hubinfo_update source - lookup /DOMAIN/SecondaryHA ok Mar 30 15:23:07:834 [140605547460672] 2 hub: ch_hublist /DOMAIN/A0F8-USDA-FMMI-8/a0f8us052nim748/hub Mar 30 15:23:07:834 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:23:07:834 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-3] domain [DOMAIN] addr [(null)] ip [100.64.1.13] port [48002] robotname [a0f8us052nim743] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-3] source [/DOMAIN/SecondaryHA] from [127.0.0.1/53272] Mar 30 15:23:07:834 [140605547460672] 4 hub: hubup source: /DOMAIN/SecondaryHA Mar 30 15:23:07:834 [140605547460672] 4 hub: hubinfo create: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 0x214c370 Mar 30 15:23:07:834 [140605547460672] 1 hub: hubinfo_update - new hubinfo for /DOMAIN/A0F8-USDA-FMMI-3 p=3 127.0.0.1/53272 Mar 30 15:23:07:834 [140602666645248] 3 hub: TSESS-P-14-0 [Secondary] started (remote) in tid=2617243392 Mar 30 15:23:07:834 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/A0F8-USDA-FMMI-3/a0f8us052nim743/hub, from 127.0.0.1/53272 Mar 30 15:23:07:834 [140605547460672] 2 hub: hubinfo_update - DOMAIN/A0F8-USDA-FMMI-3 from=127.0.0.1/53272 CHANGED proximity=3 status=0 secver=217 Mar 30 15:23:07:834 [140605547460672] 5 hub: hubinfo_update source - lookup /DOMAIN/SecondaryHA ok Mar 30 15:23:07:834 [140605547460672] 2 hub: ch_hublist /DOMAIN/A0F8-USDA-FMMI-3/a0f8us052nim743/hub Mar 30 15:23:07:834 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:23:07:834 [140602666645248] 2 hub: TSESS-P-14 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:33:07 Mar 30 15:23:07:834 [140602666645248] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:23:07:834 [140602666645248] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:23:07:834 [140602666645248] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:23:07:834 [140602666645248] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:23:07:834 [140602666645248] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:23:07:834 [140602666645248] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:23:07:834 [140603740354304] 3 hub: SSL handshake start from 10.x.x.x/48003 fd=42: before/connect initialization Mar 30 15:23:07:834 [140603740354304] 3 hub: SSL state (connect): before/connect initialization Mar 30 15:23:07:834 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [/DOMAIN/SecondaryHA] from [127.0.0.1/53272] Mar 30 15:23:07:834 [140605547460672] 4 hub: hubup source: /DOMAIN/SecondaryHA Mar 30 15:23:07:834 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:23:07:834 [140605547460672] 5 hub: hubinfo_update - for /DOMAIN/TunnelB/ROBOT_NAME/hub from 127.0.0.1/53272, proximity 1 higher than current 0, ignore update.... Mar 30 15:23:07:834 [140603740354304] 3 hub: SSL state (connect): SSLv3 write client hello A Mar 30 15:23:07:834 [140602666645248] 4 hub: CONNECT: 0x7fe094000be0(43) 127.0.0.1/53276->127.0.0.1/48002 Mar 30 15:23:07:834 [140602666645248] 3 hub: TSESS-P-14-0 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:23:07:834 [140602666645248] 1 hub: TSESS-P-14-0 [Secondary] received 100 'hubup' (8788 bytes) Mar 30 15:23:07:834 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53276 (s:8493) Mar 30 15:23:07:834 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/53276] Mar 30 15:23:07:834 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:23:07:834 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:23:07:834 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/53276 Mar 30 15:23:07:834 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/53276 same proximity=0 status=0 secver=217 Mar 30 15:23:07:834 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-9] domain [DOMAIN] addr [(null)] ip [100.64.1.19] port [48002] robotname [a0f8us052nim749] version [9.20 [Build 9.20.1048, Aug 8 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-9] source [/DOMAIN/Secondary] from [127.0.0.1/53276] Mar 30 15:23:07:834 [140605547460672] 4 hub: hubup source: /DOMAIN/Secondary Mar 30 15:23:07:834 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-9 Mar 30 15:23:07:834 [140605547460672] 5 hub: hubinfo_update - same proximity but different source (/DOMAIN/Secondary) Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-4] domain [DOMAIN] addr [(null)] ip [100.64.1.14] port [48002] robotname [a0f8us052nim744] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-4] source [/DOMAIN/Secondary] from [127.0.0.1/53276] Mar 30 15:23:07:835 [140605547460672] 4 hub: hubup source: /DOMAIN/Secondary Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-4 Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - same proximity but different source (/DOMAIN/Secondary) Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - name [Primary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [a0a4us052nim701] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Primary] source [/DOMAIN/Secondary] from [127.0.0.1/53276] Mar 30 15:23:07:835 [140605547460672] 4 hub: hubup source: /DOMAIN/Secondary Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Primary Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - same proximity but different source (/DOMAIN/Secondary) Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-5] domain [DOMAIN] addr [(null)] ip [100.64.1.15] port [48002] robotname [a0f8us052nim745] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-5] source [/DOMAIN/Secondary] from [127.0.0.1/53276] Mar 30 15:23:07:835 [140605547460672] 4 hub: hubup source: /DOMAIN/Secondary Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-5 Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - same proximity but different source (/DOMAIN/Secondary) Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-1] domain [DOMAIN] addr [(null)] ip [100.64.1.11] port [48002] robotname [a0f8us052nim741] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-1] source [/DOMAIN/Secondary] from [127.0.0.1/53276] Mar 30 15:23:07:835 [140605547460672] 4 hub: hubup source: /DOMAIN/Secondary Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-1 Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - same proximity but different source (/DOMAIN/Secondary) Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-6] domain [DOMAIN] addr [(null)] ip [100.64.1.16] port [48002] robotname [a0f8us052nim746] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-6] source [/DOMAIN/Secondary] from [127.0.0.1/53276] Mar 30 15:23:07:835 [140605547460672] 4 hub: hubup source: /DOMAIN/Secondary Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-6 Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - same proximity but different source (/DOMAIN/Secondary) Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-10] domain [DOMAIN] addr [(null)] ip [100.64.1.20] port [48002] robotname [a0f8us052nim750] version [9.20 [Build 9.20.1048, Aug 8 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-10] source [/DOMAIN/Secondary] from [127.0.0.1/53276] Mar 30 15:23:07:835 [140605547460672] 4 hub: hubup source: /DOMAIN/Secondary Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-10 Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - same proximity but different source (/DOMAIN/Secondary) Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-2] domain [DOMAIN] addr [(null)] ip [100.64.1.12] port [48002] robotname [a0f8us052nim742] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-2] source [/DOMAIN/Secondary] from [127.0.0.1/53276] Mar 30 15:23:07:835 [140605547460672] 4 hub: hubup source: /DOMAIN/Secondary Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-2 Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - same proximity but different source (/DOMAIN/Secondary) Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-7] domain [DOMAIN] addr [(null)] ip [100.64.1.17] port [48002] robotname [a0f8us052nim747] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-7] source [/DOMAIN/Secondary] from [127.0.0.1/53276] Mar 30 15:23:07:835 [140605547460672] 4 hub: hubup source: /DOMAIN/Secondary Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-7 Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - same proximity but different source (/DOMAIN/Secondary) Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - name [TunnelA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelA] source [/DOMAIN/Secondary] from [127.0.0.1/53276] Mar 30 15:23:07:835 [140605547460672] 4 hub: hubup source: /DOMAIN/Secondary Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelA Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - same proximity but different source (/DOMAIN/Secondary) Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - name [PrimaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [a0a4us052nim702] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [PrimaryHA] source [/DOMAIN/Secondary] from [127.0.0.1/53276] Mar 30 15:23:07:835 [140605547460672] 4 hub: hubup source: /DOMAIN/Secondary Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/PrimaryHA Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - same proximity but different source (/DOMAIN/Secondary) Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-11] domain [DOMAIN] addr [(null)] ip [100.64.1.21] port [48002] robotname [a0f8us052nim751] version [9.20 [Build 9.20.1048, Aug 8 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-11] source [/DOMAIN/Secondary] from [127.0.0.1/53276] Mar 30 15:23:07:835 [140605547460672] 4 hub: hubup source: /DOMAIN/Secondary Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-11 Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - same proximity but different source (/DOMAIN/Secondary) Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-3] domain [DOMAIN] addr [(null)] ip [100.64.1.13] port [48002] robotname [a0f8us052nim743] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-3] source [/DOMAIN/Secondary] from [127.0.0.1/53276] Mar 30 15:23:07:835 [140605547460672] 4 hub: hubup source: /DOMAIN/Secondary Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-3 Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - same proximity but different source (/DOMAIN/Secondary) Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-8] domain [DOMAIN] addr [(null)] ip [100.64.1.18] port [48002] robotname [a0f8us052nim748] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-8] source [/DOMAIN/Secondary] from [127.0.0.1/53276] Mar 30 15:23:07:835 [140605547460672] 4 hub: hubup source: /DOMAIN/Secondary Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-8 Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - same proximity but different source (/DOMAIN/Secondary) Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [/DOMAIN/Secondary] from [127.0.0.1/53276] Mar 30 15:23:07:835 [140605547460672] 4 hub: hubup source: /DOMAIN/Secondary Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - for /DOMAIN/SecondaryHA/ROBOT_NAME/hub from 127.0.0.1/53276, proximity 1 higher than current 0, ignore update.... Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [/DOMAIN/Secondary] from [127.0.0.1/53276] Mar 30 15:23:07:835 [140605547460672] 4 hub: hubup source: /DOMAIN/Secondary Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:23:07:835 [140605547460672] 5 hub: hubinfo_update - for /DOMAIN/TunnelB/ROBOT_NAME/hub from 127.0.0.1/53276, proximity 1 higher than current 0, ignore update.... Mar 30 15:23:07:838 [140603161552640] 5 hub: TSESS-P-4-3 [SecondaryHA] local 200 'OK' Mar 30 15:23:07:838 [140603161552640] 1 hub: TSESS-P-4-3 [SecondaryHA] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:23:07:839 [140602666645248] 5 hub: TSESS-P-14-0 [Secondary] local 200 'OK' Mar 30 15:23:07:840 [140602666645248] 1 hub: TSESS-P-14-0 [Secondary] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:23:07:848 [140603161552640] 1 hub: TSESS-P-4-3 [SecondaryHA] ssl 100 '_close' (300 bytes) Mar 30 15:23:07:848 [140603161552640] 3 hub: TSESS-P-4-3 [SecondaryHA] is paired with [A-1948-3] Mar 30 15:23:07:848 [140603161552640] 1 hub: TSESS-P-4-3 [SecondaryHA] received 100 '_close' (296 bytes) Mar 30 15:23:07:850 [140602666645248] 1 hub: TSESS-P-14-0 [Secondary] ssl 100 '_close' (296 bytes) Mar 30 15:23:07:850 [140602666645248] 3 hub: TSESS-P-14-0 [Secondary] is paired with [A-2123-0] Mar 30 15:23:07:850 [140602666645248] 1 hub: TSESS-P-14-0 [Secondary] received 100 '_close' (296 bytes) Mar 30 15:23:07:853 [140603161552640] 1 hub: TSESS-P-4-3 [SecondaryHA] remote session closed Mar 30 15:23:07:853 [140603161552640] 3 hub: TSESS-P-4-3 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:23:07:853 [140603161552640] 5 hub: TSESS-P-4-3 [SecondaryHA] suspended Mar 30 15:23:07:855 [140602666645248] 1 hub: TSESS-P-14-0 [Secondary] remote session closed Mar 30 15:23:07:855 [140602666645248] 3 hub: TSESS-P-14-0 [Secondary] exiting msg loop (remote close) Mar 30 15:23:07:855 [140602666645248] 5 hub: TSESS-P-14-0 [Secondary] suspended Mar 30 15:23:07:911 [140603731961600] 5 hub: TSESS-P-11 not internal command - thread will handle. mtype=100; sfd:26 Mar 30 15:23:07:911 [140602658252544] 3 hub: TSESS-P-11-0 [SecondaryHA] started (remote) in tid=2608850688 Mar 30 15:23:07:911 [140602658252544] 2 hub: TSESS-P-11 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:33:07 Mar 30 15:23:07:911 [140602658252544] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:23:07:911 [140602658252544] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:23:07:911 [140602658252544] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:23:07:911 [140602658252544] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:23:07:911 [140602658252544] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:23:07:911 [140602658252544] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:23:07:911 [140602658252544] 4 hub: CONNECT: 0x7fe08c000c60(40) 127.0.0.1/53278->127.0.0.1/48002 Mar 30 15:23:07:911 [140602658252544] 3 hub: TSESS-P-11-0 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:23:07:912 [140602658252544] 1 hub: TSESS-P-11-0 [SecondaryHA] received 100 'hubup' (687 bytes) Mar 30 15:23:07:912 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53278 (s:392) Mar 30 15:23:07:912 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/53278] Mar 30 15:23:07:912 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:23:07:912 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:23:07:912 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/53278 Mar 30 15:23:07:912 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/53278 same proximity=0 status=0 secver=217 Mar 30 15:23:07:917 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] local 200 'OK' Mar 30 15:23:07:917 [140602658252544] 1 hub: TSESS-P-11-0 [SecondaryHA] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:23:07:927 [140602658252544] 1 hub: TSESS-P-11-0 [SecondaryHA] ssl 100 '_close' (300 bytes) Mar 30 15:23:07:927 [140602658252544] 3 hub: TSESS-P-11-0 [SecondaryHA] is paired with [A-1950-0] Mar 30 15:23:07:927 [140602658252544] 1 hub: TSESS-P-11-0 [SecondaryHA] received 100 '_close' (296 bytes) Mar 30 15:23:07:932 [140602658252544] 1 hub: TSESS-P-11-0 [SecondaryHA] remote session closed Mar 30 15:23:07:932 [140602658252544] 3 hub: TSESS-P-11-0 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:23:07:932 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:23:07:934 [140603740354304] 3 hub: SSL state (connect): SSLv3 read server hello A Mar 30 15:23:07:934 [140603740354304] 3 hub: SSL state (connect): SSLv3 read finished A Mar 30 15:23:07:934 [140603740354304] 3 hub: SSL state (connect): SSLv3 write change cipher spec A Mar 30 15:23:07:934 [140603740354304] 3 hub: SSL state (connect): SSLv3 write finished A Mar 30 15:23:07:934 [140603740354304] 3 hub: SSL state (connect): SSLv3 flush data Mar 30 15:23:07:934 [140603740354304] 3 hub: SSL handshake done fd=42: SSL negotiation finished successfully Mar 30 15:23:07:934 [140603740354304] 1 hub: (ssl_verify_cert_hostname) - SSL certificate subjectAltName doesn't match given hostname(10.x.x.x) Mar 30 15:23:07:934 [140603740354304] 3 hub: (x509_verify_cert_commonName) - extracted commonName (10.x.x.x) from cert. Comparing with hostname (10.x.x.x) Mar 30 15:23:07:934 [140603740354304] 3 hub: TSESS-P-21 connected to 10.x.x.x on socket 42 Mar 30 15:23:07:934 [140603740354304] 3 hub: CTRL Secondary SSL_SESSION() reused for TSESS-P-21 Mar 30 15:23:07:935 [140603740354304] 3 hub: CTRL Secondary new passive TSESS-P-6 added Mar 30 15:23:07:935 [140603740354304] 3 hub: CTRL Secondary added new passive session (2 running, 2 available) Mar 30 15:23:07:935 [140603740354304] 5 hub: TSESS-P-20 not internal command - thread will handle. mtype=100; sfd:35 Mar 30 15:23:07:935 [140602649859840] 3 hub: TSESS-P-20-0 [Secondary] started (remote) in tid=2600457984 Mar 30 15:23:07:935 [140602649859840] 2 hub: TSESS-P-20 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:33:07 Mar 30 15:23:07:935 [140602649859840] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:23:07:935 [140602649859840] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:23:07:935 [140602649859840] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:23:07:935 [140602649859840] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:23:07:935 [140602649859840] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:23:07:935 [140602649859840] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:23:07:935 [140602649859840] 4 hub: CONNECT: 0x7fe090000be0(41) 127.0.0.1/53280->127.0.0.1/48002 Mar 30 15:23:07:935 [140602649859840] 3 hub: TSESS-P-20-0 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:23:07:935 [140602649859840] 1 hub: TSESS-P-20-0 [Secondary] received 100 'hubup' (683 bytes) Mar 30 15:23:07:935 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53280 (s:388) Mar 30 15:23:07:935 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/53280] Mar 30 15:23:07:935 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:23:07:935 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:23:07:935 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/53280 Mar 30 15:23:07:935 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/53280 same proximity=0 status=0 secver=217 Mar 30 15:23:07:940 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] local 200 'OK' Mar 30 15:23:07:940 [140602649859840] 1 hub: TSESS-P-20-0 [Secondary] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:23:07:951 [140602649859840] 1 hub: TSESS-P-20-0 [Secondary] ssl 100 '_close' (296 bytes) Mar 30 15:23:07:951 [140602649859840] 3 hub: TSESS-P-20-0 [Secondary] is paired with [A-2125-0] Mar 30 15:23:07:951 [140602649859840] 1 hub: TSESS-P-20-0 [Secondary] received 100 '_close' (296 bytes) Mar 30 15:23:07:956 [140602649859840] 1 hub: TSESS-P-20-0 [Secondary] remote session closed Mar 30 15:23:07:956 [140602649859840] 3 hub: TSESS-P-20-0 [Secondary] exiting msg loop (remote close) Mar 30 15:23:07:956 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:23:08:255 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (385 bytes) Mar 30 15:23:08:255 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:08:255 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (365 bytes) Mar 30 15:23:08:260 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:08:260 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (309 bytes, 0 ms) Mar 30 15:23:08:595 [140603169945344] 5 hub: TSESS-P-7-2 [TunnelB] suspended Mar 30 15:23:08:642 [140603723568896] 5 hub: TSESS-P-7 not internal command - thread will handle. mtype=100; sfd:22 Mar 30 15:23:08:643 [140603723568896] 2 hub: CTRL TunnelB request 'hubup' passed to TSESS-P-7 [140603169945344] Mar 30 15:23:08:643 [140603169945344] 3 hub: TSESS-P-7-3 [TunnelB] restarted (for remote) Mar 30 15:23:08:643 [140603169945344] 2 hub: TSESS-P-7 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:33:08 Mar 30 15:23:08:643 [140603169945344] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:23:08:643 [140603169945344] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:23:08:643 [140603169945344] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:23:08:643 [140603169945344] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:23:08:643 [140603169945344] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:23:08:643 [140603169945344] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:23:08:643 [140603169945344] 4 hub: CONNECT: 0x7fe0a8000d20(41) 127.0.0.1/53282->127.0.0.1/48002 Mar 30 15:23:08:643 [140603169945344] 3 hub: TSESS-P-7-3 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:23:08:643 [140603169945344] 1 hub: TSESS-P-7-3 [TunnelB] received 100 'hubup' (8247 bytes) Mar 30 15:23:08:643 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53282 (s:7953) Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/53282] Mar 30 15:23:08:643 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/53282 Mar 30 15:23:08:643 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/53282 same proximity=0 status=0 secver=217 Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-9] domain [DOMAIN] addr [(null)] ip [100.64.1.19] port [48002] robotname [a0f8us052nim749] version [9.20 [Build 9.20.1048, Aug 8 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-9] source [/DOMAIN/TunnelB] from [127.0.0.1/53282] Mar 30 15:23:08:643 [140605547460672] 4 hub: hubup source: /DOMAIN/TunnelB Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-9 Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - for /DOMAIN/A0F8-USDA-FMMI-9/a0f8us052nim749/hub from 127.0.0.1/53282, proximity 4 higher than current 3, ignore update.... Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-4] domain [DOMAIN] addr [(null)] ip [100.64.1.14] port [48002] robotname [a0f8us052nim744] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-4] source [/DOMAIN/TunnelB] from [127.0.0.1/53282] Mar 30 15:23:08:643 [140605547460672] 4 hub: hubup source: /DOMAIN/TunnelB Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-4 Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - for /DOMAIN/A0F8-USDA-FMMI-4/a0f8us052nim744/hub from 127.0.0.1/53282, proximity 4 higher than current 3, ignore update.... Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - name [Primary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [a0a4us052nim701] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Primary] source [/DOMAIN/TunnelB] from [127.0.0.1/53282] Mar 30 15:23:08:643 [140605547460672] 4 hub: hubup source: /DOMAIN/TunnelB Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Primary Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - for /DOMAIN/Primary/a0a4us052nim701/hub from 127.0.0.1/53282, proximity 2 higher than current 1, ignore update.... Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-5] domain [DOMAIN] addr [(null)] ip [100.64.1.15] port [48002] robotname [a0f8us052nim745] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-5] source [/DOMAIN/TunnelB] from [127.0.0.1/53282] Mar 30 15:23:08:643 [140605547460672] 4 hub: hubup source: /DOMAIN/TunnelB Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-5 Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - for /DOMAIN/A0F8-USDA-FMMI-5/a0f8us052nim745/hub from 127.0.0.1/53282, proximity 4 higher than current 3, ignore update.... Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-1] domain [DOMAIN] addr [(null)] ip [100.64.1.11] port [48002] robotname [a0f8us052nim741] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-1] source [/DOMAIN/TunnelB] from [127.0.0.1/53282] Mar 30 15:23:08:643 [140605547460672] 4 hub: hubup source: /DOMAIN/TunnelB Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-1 Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - for /DOMAIN/A0F8-USDA-FMMI-1/a0f8us052nim741/hub from 127.0.0.1/53282, proximity 3 higher than current 2, ignore update.... Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-6] domain [DOMAIN] addr [(null)] ip [100.64.1.16] port [48002] robotname [a0f8us052nim746] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-6] source [/DOMAIN/TunnelB] from [127.0.0.1/53282] Mar 30 15:23:08:643 [140605547460672] 4 hub: hubup source: /DOMAIN/TunnelB Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-6 Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - for /DOMAIN/A0F8-USDA-FMMI-6/a0f8us052nim746/hub from 127.0.0.1/53282, proximity 4 higher than current 3, ignore update.... Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-10] domain [DOMAIN] addr [(null)] ip [100.64.1.20] port [48002] robotname [a0f8us052nim750] version [9.20 [Build 9.20.1048, Aug 8 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-10] source [/DOMAIN/TunnelB] from [127.0.0.1/53282] Mar 30 15:23:08:643 [140605547460672] 4 hub: hubup source: /DOMAIN/TunnelB Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-10 Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - for /DOMAIN/A0F8-USDA-FMMI-10/a0f8us052nim750/hub from 127.0.0.1/53282, proximity 4 higher than current 3, ignore update.... Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-2] domain [DOMAIN] addr [(null)] ip [100.64.1.12] port [48002] robotname [a0f8us052nim742] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-2] source [/DOMAIN/TunnelB] from [127.0.0.1/53282] Mar 30 15:23:08:643 [140605547460672] 4 hub: hubup source: /DOMAIN/TunnelB Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-2 Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - for /DOMAIN/A0F8-USDA-FMMI-2/a0f8us052nim742/hub from 127.0.0.1/53282, proximity 4 higher than current 3, ignore update.... Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-7] domain [DOMAIN] addr [(null)] ip [100.64.1.17] port [48002] robotname [a0f8us052nim747] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-7] source [/DOMAIN/TunnelB] from [127.0.0.1/53282] Mar 30 15:23:08:643 [140605547460672] 4 hub: hubup source: /DOMAIN/TunnelB Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-7 Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - for /DOMAIN/A0F8-USDA-FMMI-7/a0f8us052nim747/hub from 127.0.0.1/53282, proximity 4 higher than current 3, ignore update.... Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - name [TunnelA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelA] source [/DOMAIN/TunnelB] from [127.0.0.1/53282] Mar 30 15:23:08:643 [140605547460672] 4 hub: hubup source: /DOMAIN/TunnelB Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelA Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - for /DOMAIN/TunnelA/ROBOT_NAME/hub from 127.0.0.1/53282, proximity 2 higher than current 1, ignore update.... Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - name [PrimaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [a0a4us052nim702] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [PrimaryHA] source [/DOMAIN/TunnelB] from [127.0.0.1/53282] Mar 30 15:23:08:643 [140605547460672] 4 hub: hubup source: /DOMAIN/TunnelB Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/PrimaryHA Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - for /DOMAIN/PrimaryHA/a0a4us052nim702/hub from 127.0.0.1/53282, proximity 2 higher than current 1, ignore update.... Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-11] domain [DOMAIN] addr [(null)] ip [100.64.1.21] port [48002] robotname [a0f8us052nim751] version [9.20 [Build 9.20.1048, Aug 8 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-11] source [/DOMAIN/TunnelB] from [127.0.0.1/53282] Mar 30 15:23:08:643 [140605547460672] 4 hub: hubup source: /DOMAIN/TunnelB Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-11 Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - for /DOMAIN/A0F8-USDA-FMMI-11/a0f8us052nim751/hub from 127.0.0.1/53282, proximity 4 higher than current 3, ignore update.... Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-8] domain [DOMAIN] addr [(null)] ip [100.64.1.18] port [48002] robotname [a0f8us052nim748] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-8] source [/DOMAIN/TunnelB] from [127.0.0.1/53282] Mar 30 15:23:08:643 [140605547460672] 4 hub: hubup source: /DOMAIN/TunnelB Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-8 Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - for /DOMAIN/A0F8-USDA-FMMI-8/a0f8us052nim748/hub from 127.0.0.1/53282, proximity 4 higher than current 3, ignore update.... Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-3] domain [DOMAIN] addr [(null)] ip [100.64.1.13] port [48002] robotname [a0f8us052nim743] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-3] source [/DOMAIN/TunnelB] from [127.0.0.1/53282] Mar 30 15:23:08:643 [140605547460672] 4 hub: hubup source: /DOMAIN/TunnelB Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-3 Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - for /DOMAIN/A0F8-USDA-FMMI-3/a0f8us052nim743/hub from 127.0.0.1/53282, proximity 4 higher than current 3, ignore update.... Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [/DOMAIN/TunnelB] from [127.0.0.1/53282] Mar 30 15:23:08:643 [140605547460672] 4 hub: hubup source: /DOMAIN/TunnelB Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:23:08:643 [140605547460672] 5 hub: hubinfo_update - for /DOMAIN/SecondaryHA/ROBOT_NAME/hub from 127.0.0.1/53282, proximity 1 higher than current 0, ignore update.... Mar 30 15:23:08:648 [140603169945344] 5 hub: TSESS-P-7-3 [TunnelB] local 200 'OK' Mar 30 15:23:08:648 [140603169945344] 1 hub: TSESS-P-7-3 [TunnelB] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:23:08:658 [140603169945344] 1 hub: TSESS-P-7-3 [TunnelB] ssl 100 '_close' (283 bytes) Mar 30 15:23:08:658 [140603169945344] 3 hub: TSESS-P-7-3 [TunnelB] is paired with [A-299-3] Mar 30 15:23:08:658 [140603169945344] 1 hub: TSESS-P-7-3 [TunnelB] received 100 '_close' (295 bytes) Mar 30 15:23:08:663 [140603169945344] 1 hub: TSESS-P-7-3 [TunnelB] remote session closed Mar 30 15:23:08:663 [140603169945344] 3 hub: TSESS-P-7-3 [TunnelB] exiting msg loop (remote close) Mar 30 15:23:08:663 [140603169945344] 5 hub: TSESS-P-7-3 [TunnelB] suspended Mar 30 15:23:08:674 [140603723568896] 5 hub: TSESS-P-13 not internal command - thread will handle. mtype=100; sfd:28 Mar 30 15:23:08:675 [140602641467136] 3 hub: TSESS-P-13-0 [TunnelB] started (remote) in tid=2592065280 Mar 30 15:23:08:675 [140602641467136] 2 hub: TSESS-P-13 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:33:08 Mar 30 15:23:08:675 [140602641467136] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:23:08:675 [140602641467136] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:23:08:675 [140602641467136] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:23:08:675 [140602641467136] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:23:08:675 [140602641467136] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:23:08:675 [140602641467136] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:23:08:675 [140602641467136] 4 hub: CONNECT: 0x7fe084000c60(41) 127.0.0.1/53284->127.0.0.1/48002 Mar 30 15:23:08:675 [140602641467136] 3 hub: TSESS-P-13-0 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:23:08:675 [140602641467136] 1 hub: TSESS-P-13-0 [TunnelB] received 100 'hubup' (672 bytes) Mar 30 15:23:08:675 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53284 (s:378) Mar 30 15:23:08:675 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/53284] Mar 30 15:23:08:675 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:23:08:675 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:23:08:675 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/53284 Mar 30 15:23:08:675 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/53284 same proximity=0 status=0 secver=217 Mar 30 15:23:08:680 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] local 200 'OK' Mar 30 15:23:08:680 [140602641467136] 1 hub: TSESS-P-13-0 [TunnelB] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:23:08:690 [140602641467136] 1 hub: TSESS-P-13-0 [TunnelB] ssl 100 '_close' (283 bytes) Mar 30 15:23:08:690 [140602641467136] 3 hub: TSESS-P-13-0 [TunnelB] is paired with [A-301-0] Mar 30 15:23:08:690 [140602641467136] 1 hub: TSESS-P-13-0 [TunnelB] received 100 '_close' (295 bytes) Mar 30 15:23:08:696 [140602641467136] 1 hub: TSESS-P-13-0 [TunnelB] remote session closed Mar 30 15:23:08:696 [140602641467136] 3 hub: TSESS-P-13-0 [TunnelB] exiting msg loop (remote close) Mar 30 15:23:08:696 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:23:09:267 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (376 bytes) Mar 30 15:23:09:267 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:09:267 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (356 bytes) Mar 30 15:23:09:272 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:09:272 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (274 bytes, 0 ms) Mar 30 15:23:09:513 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:10:281 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (384 bytes) Mar 30 15:23:10:281 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:10:281 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (364 bytes) Mar 30 15:23:10:286 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:10:286 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (562 bytes, 0 ms) Mar 30 15:23:10:296 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (384 bytes) Mar 30 15:23:10:296 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:10:296 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (364 bytes) Mar 30 15:23:10:301 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:10:301 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (562 bytes, 0 ms) Mar 30 15:23:11:308 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (383 bytes) Mar 30 15:23:11:308 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:11:308 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (363 bytes) Mar 30 15:23:11:313 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:11:313 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (291 bytes, 0 ms) Mar 30 15:23:11:316 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:11:992 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:23:11:992 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:23:11:992 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60812 Mar 30 15:23:11:992 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60812' delegated to TSESS-A-5 [140603715176192] Mar 30 15:23:11:992 [140603715176192] 3 hub: TSESS-A-5-14 [TunnelB] restarted (for local) Mar 30 15:23:11:997 [140603715176192] 5 hub: TSESS-A-5-14 [TunnelB] local 100 'get_info' Mar 30 15:23:11:997 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:11:998 [140603715176192] 1 hub: TSESS-A-5-14 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:23:12:008 [140603715176192] 1 hub: TSESS-A-5-14 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:23:12:008 [140603715176192] 3 hub: TSESS-A-5-14 [TunnelB] is paired with [P-298-14] Mar 30 15:23:12:008 [140603715176192] 1 hub: TSESS-A-5-14 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:23:12:008 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60814 Mar 30 15:23:12:008 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60814' delegated to TSESS-A-10 [140603706783488] Mar 30 15:23:12:008 [140603706783488] 3 hub: TSESS-A-10-10 [TunnelB] restarted (for local) Mar 30 15:23:12:013 [140603715176192] 5 hub: TSESS-A-5-14 [TunnelB] local 100 '_close' Mar 30 15:23:12:013 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:12:013 [140603715176192] 1 hub: TSESS-A-5-14 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:23:12:013 [140603715176192] 3 hub: TSESS-A-5-14 [TunnelB] local session closed, suspending Mar 30 15:23:12:013 [140603706783488] 5 hub: TSESS-A-10-10 [TunnelB] local 100 '_status' Mar 30 15:23:12:013 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:12:013 [140603706783488] 1 hub: TSESS-A-10-10 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:23:12:024 [140603706783488] 1 hub: TSESS-A-10-10 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:23:12:024 [140603706783488] 3 hub: TSESS-A-10-10 [TunnelB] is paired with [P-300-10] Mar 30 15:23:12:024 [140603706783488] 1 hub: TSESS-A-10-10 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:23:12:029 [140603706783488] 5 hub: TSESS-A-10-10 [TunnelB] local 100 '_close' Mar 30 15:23:12:029 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:12:029 [140603706783488] 1 hub: TSESS-A-10-10 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:23:12:029 [140603706783488] 3 hub: TSESS-A-10-10 [TunnelB] local session closed, suspending Mar 30 15:23:12:213 [140603715176192] 5 hub: TSESS-A-5-14 [TunnelB] suspended Mar 30 15:23:12:229 [140603706783488] 5 hub: TSESS-A-10-10 [TunnelB] suspended Mar 30 15:23:12:319 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (371 bytes) Mar 30 15:23:12:320 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:12:320 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (351 bytes) Mar 30 15:23:12:325 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:12:325 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (272 bytes, 0 ms) Mar 30 15:23:13:118 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:13:332 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (377 bytes) Mar 30 15:23:13:332 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:13:332 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (357 bytes) Mar 30 15:23:13:337 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:13:337 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (271 bytes, 0 ms) Mar 30 15:23:14:339 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (381 bytes) Mar 30 15:23:14:339 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:14:339 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (361 bytes) Mar 30 15:23:14:344 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:14:344 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (526 bytes, 0 ms) Mar 30 15:23:14:354 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (381 bytes) Mar 30 15:23:14:354 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:14:354 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (361 bytes) Mar 30 15:23:14:360 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:14:360 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (526 bytes, 0 ms) Mar 30 15:23:15:367 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (385 bytes) Mar 30 15:23:15:367 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:15:367 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (365 bytes) Mar 30 15:23:15:372 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:15:372 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (730 bytes, 0 ms) Mar 30 15:23:15:382 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (385 bytes) Mar 30 15:23:15:382 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:15:382 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (365 bytes) Mar 30 15:23:15:387 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:15:387 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (730 bytes, 0 ms) Mar 30 15:23:15:397 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (385 bytes) Mar 30 15:23:15:397 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:15:397 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (365 bytes) Mar 30 15:23:15:403 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:15:403 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (730 bytes, 0 ms) Mar 30 15:23:15:822 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:16:409 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (367 bytes) Mar 30 15:23:16:409 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:16:409 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (347 bytes) Mar 30 15:23:16:415 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:16:415 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (444 bytes, 0 ms) Mar 30 15:23:16:425 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (367 bytes) Mar 30 15:23:16:425 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:16:425 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (347 bytes) Mar 30 15:23:16:430 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:16:430 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (444 bytes, 0 ms) Mar 30 15:23:16:952 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:23:16:983 [140605283890944] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:23:16:984 [140605283890944] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:23:16:984 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:23:16:984 [140605283890944] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:23:16:984 [140605283890944] 3 hub: hubhash_write - 0 hubs saved to cache Mar 30 15:23:16:984 [140605283890944] 1 hub: PERF: hubup_check_interval: 0 sec hubhash_size: 18 Mar 30 15:23:16:984 [140605283890944] 2 hub: hubupdatelist->Secondary[+]: /DOMAIN/Primary 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:16:984 [140605283890944] 2 hub: hubupdatelist->Secondary[+]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:16:984 [140605283890944] 2 hub: hubupdatelist->Secondary[+]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:16:984 [140605283890944] 2 hub: hubupdatelist->Secondary[+]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:16:984 [140605283890944] 2 hub: hubupdatelist->Secondary[+]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:16:984 [140605283890944] 2 hub: hubupdatelist->Secondary[+]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:16:984 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:23:16:984 [140605283890944] 2 hub: hubupdatelist->Secondary[+]: /DOMAIN/PrimaryHA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:16:984 [140605283890944] 2 hub: hubupdatelist->Secondary[+]: /DOMAIN/TunnelA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:16:984 [140605283890944] 2 hub: hubupdatelist->Secondary[+]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:16:984 [140605283890944] 2 hub: hubupdatelist->Secondary[+]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:16:985 [140605283890944] 2 hub: hubupdatelist->Secondary[+]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:16:985 [140605283890944] 2 hub: hubupdatelist->Secondary[+]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:16:985 [140605283890944] 2 hub: hubupdatelist->Secondary[+]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:16:985 [140605283890944] 2 hub: hubupdatelist->Secondary[+]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:16:985 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:23:16:985 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:23:16:985 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:23:16:985 [140605283890944] 4 hub: CONNECT: 0x7fe130005d70(41) 127.0.0.1/46954->127.0.0.1/48007 Mar 30 15:23:16:985 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/46954 Mar 30 15:23:16:985 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/46954' delegated to TSESS-A-6 [140603203516160] Mar 30 15:23:16:985 [140603203516160] 3 hub: TSESS-A-6-3 [Secondary] restarted (for local) Mar 30 15:23:16:990 [140603203516160] 5 hub: TSESS-A-6-3 [Secondary] local 100 'hubup' Mar 30 15:23:16:990 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:16:990 [140603203516160] 1 hub: TSESS-A-6-3 [Secondary] sent 100 'hubup' (7786 bytes, 0 ms) Mar 30 15:23:16:997 [140605547460672] 4 hub: medium timeout Mar 30 15:23:16:997 [140605547460672] 3 hub: next_check [1585596396] now [1585596196] Mar 30 15:23:17:000 [140603203516160] 1 hub: TSESS-A-6-3 [Secondary] ssl 200 '-' (453 bytes) Mar 30 15:23:17:000 [140603203516160] 3 hub: TSESS-A-6-3 [Secondary] is paired with [P-2120-3] Mar 30 15:23:17:000 [140603203516160] 1 hub: TSESS-A-6-3 [Secondary] received 200 '-' (461 bytes) Mar 30 15:23:17:000 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:23:17:000 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:23:17:000 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:23:17:000 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:23:17:000 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary same proximity=0 status=0 secver=217 Mar 30 15:23:17:000 [140605283890944] 2 hub: hubupdatelist->TunnelB[+]: /DOMAIN/Primary 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:001 [140605283890944] 2 hub: hubupdatelist->TunnelB[+]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:001 [140605283890944] 2 hub: hubupdatelist->TunnelB[+]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:001 [140605283890944] 2 hub: hubupdatelist->TunnelB[+]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:001 [140605283890944] 2 hub: hubupdatelist->TunnelB[+]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:001 [140605283890944] 2 hub: hubupdatelist->TunnelB[+]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:001 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:23:17:001 [140605283890944] 2 hub: hubupdatelist->TunnelB[+]: /DOMAIN/PrimaryHA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:001 [140605283890944] 2 hub: hubupdatelist->TunnelB[+]: /DOMAIN/TunnelA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:001 [140605283890944] 2 hub: hubupdatelist->TunnelB[+]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:001 [140605283890944] 2 hub: hubupdatelist->TunnelB[+]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:001 [140605283890944] 2 hub: hubupdatelist->TunnelB[+]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:001 [140605283890944] 2 hub: hubupdatelist->TunnelB[+]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:001 [140605283890944] 2 hub: hubupdatelist->TunnelB[+]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:001 [140605283890944] 2 hub: hubupdatelist->TunnelB[+]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:001 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:23:17:001 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:23:17:001 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:23:17:001 [140605283890944] 4 hub: CONNECT: 0x7fe130005d70(40) 127.0.0.1/60818->127.0.0.1/48009 Mar 30 15:23:17:001 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60818 Mar 30 15:23:17:001 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60818' delegated to TSESS-A-5 [140603715176192] Mar 30 15:23:17:001 [140603715176192] 3 hub: TSESS-A-5-15 [TunnelB] restarted (for local) Mar 30 15:23:17:005 [140603203516160] 5 hub: TSESS-A-6-3 [Secondary] local 100 '_close' Mar 30 15:23:17:005 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:17:006 [140603203516160] 1 hub: TSESS-A-6-3 [Secondary] sent 100 '_close' (293 bytes, 0 ms) Mar 30 15:23:17:006 [140603203516160] 3 hub: TSESS-A-6-3 [Secondary] local session closed, suspending Mar 30 15:23:17:006 [140603715176192] 5 hub: TSESS-A-5-15 [TunnelB] local 100 'hubup' Mar 30 15:23:17:006 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:17:006 [140603715176192] 1 hub: TSESS-A-5-15 [TunnelB] sent 100 'hubup' (7785 bytes, 0 ms) Mar 30 15:23:17:016 [140603715176192] 1 hub: TSESS-A-5-15 [TunnelB] ssl 200 '-' (443 bytes) Mar 30 15:23:17:017 [140603715176192] 3 hub: TSESS-A-5-15 [TunnelB] is paired with [P-298-15] Mar 30 15:23:17:017 [140603715176192] 1 hub: TSESS-A-5-15 [TunnelB] received 200 '-' (451 bytes) Mar 30 15:23:17:017 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:23:17:017 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:23:17:017 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:23:17:017 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:23:17:017 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB same proximity=0 status=0 secver=217 Mar 30 15:23:17:017 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:17:017 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:17:017 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:17:017 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:17:017 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:17:017 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:17:017 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:23:17:017 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:17:017 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:17:017 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:17:017 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:17:017 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:17:017 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:17:017 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:17:017 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:17:017 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:23:17:017 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:23:17:017 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:23:17:017 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(40) 127.0.0.1/40006->127.0.0.1/48008 Mar 30 15:23:17:017 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/40006 Mar 30 15:23:17:017 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/40006' delegated to TSESS-A-3 [140603195123456] Mar 30 15:23:17:017 [140603195123456] 3 hub: TSESS-A-3-3 [SecondaryHA] restarted (for local) Mar 30 15:23:17:022 [140603715176192] 5 hub: TSESS-A-5-15 [TunnelB] local 100 '_close' Mar 30 15:23:17:022 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:17:022 [140603715176192] 1 hub: TSESS-A-5-15 [TunnelB] sent 100 '_close' (292 bytes, 0 ms) Mar 30 15:23:17:022 [140603715176192] 3 hub: TSESS-A-5-15 [TunnelB] local session closed, suspending Mar 30 15:23:17:022 [140603195123456] 5 hub: TSESS-A-3-3 [SecondaryHA] local 100 'hubup' Mar 30 15:23:17:022 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:17:022 [140603195123456] 1 hub: TSESS-A-3-3 [SecondaryHA] sent 100 'hubup' (688 bytes, 0 ms) Mar 30 15:23:17:033 [140603195123456] 1 hub: TSESS-A-3-3 [SecondaryHA] ssl 200 '-' (457 bytes) Mar 30 15:23:17:033 [140603195123456] 3 hub: TSESS-A-3-3 [SecondaryHA] is paired with [P-1947-3] Mar 30 15:23:17:033 [140603195123456] 1 hub: TSESS-A-3-3 [SecondaryHA] received 200 '-' (465 bytes) Mar 30 15:23:17:033 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:23:17:033 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:23:17:033 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:23:17:033 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:23:17:033 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA same proximity=0 status=0 secver=217 Mar 30 15:23:17:033 [140605283890944] 5 hub: mark_hubup_list_updated Mar 30 15:23:17:033 [140605283890944] 5 hub: hublist chnaged reset: /DOMAIN/Primary 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:033 [140605283890944] 5 hub: hublist chnaged reset: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:033 [140605283890944] 5 hub: hublist chnaged reset: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:033 [140605283890944] 5 hub: hublist chnaged reset: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:033 [140605283890944] 5 hub: hublist chnaged reset: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:033 [140605283890944] 5 hub: hublist chnaged reset: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:033 [140605283890944] 5 hub: hublist changed: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:23:17:033 [140605283890944] 5 hub: hublist chnaged reset: /DOMAIN/PrimaryHA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:033 [140605283890944] 5 hub: hublist chnaged reset: /DOMAIN/TunnelA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:033 [140605283890944] 5 hub: hublist chnaged reset: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:033 [140605283890944] 5 hub: hublist chnaged reset: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:033 [140605283890944] 5 hub: hublist chnaged reset: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:033 [140605283890944] 5 hub: hublist chnaged reset: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:033 [140605283890944] 5 hub: hublist chnaged reset: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:033 [140605283890944] 5 hub: hublist chnaged reset: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:17:033 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:23:17:033 [140605283890944] 5 hub: hublist changed: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:23:17:033 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelBHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:23:17:033 [140605283890944] 5 hub: mark_hubup_list_updated end Mar 30 15:23:17:033 [140605283890944] 1 hub: PERF: hubup_check_for_updates: 1 sec hubhash_size: 18 Mar 30 15:23:17:033 [140605283890944] 1 hub: PERF: hubi_run_command: 0 sec hubhash_size: 18 Mar 30 15:23:17:033 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:23:17:034 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:23:17:034 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:23:17:034 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:23:17:034 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:23:17:035 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:23:17:035 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:23:17:035 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:23:17:036 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:23:17:036 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:23:17:036 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:23:17:037 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:23:17:037 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:23:17:037 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:23:17:038 [140603195123456] 5 hub: TSESS-A-3-3 [SecondaryHA] local 100 '_close' Mar 30 15:23:17:038 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:17:038 [140603195123456] 1 hub: TSESS-A-3-3 [SecondaryHA] sent 100 '_close' (295 bytes, 0 ms) Mar 30 15:23:17:038 [140603195123456] 3 hub: TSESS-A-3-3 [SecondaryHA] local session closed, suspending Mar 30 15:23:17:038 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:23:17:039 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:23:17:040 [140605283890944] 1 hub: PERF: hubi_check_security: 0 sec hubhash_size: 18 Mar 30 15:23:17:206 [140603203516160] 5 hub: TSESS-A-6-3 [Secondary] suspended Mar 30 15:23:17:222 [140603715176192] 5 hub: TSESS-A-5-15 [TunnelB] suspended Mar 30 15:23:17:238 [140603195123456] 5 hub: TSESS-A-3-3 [SecondaryHA] suspended Mar 30 15:23:17:437 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (381 bytes) Mar 30 15:23:17:437 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:17:437 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (361 bytes) Mar 30 15:23:17:442 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:17:442 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (498 bytes, 0 ms) Mar 30 15:23:17:453 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (381 bytes) Mar 30 15:23:17:453 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:17:453 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (361 bytes) Mar 30 15:23:17:458 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:17:458 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (498 bytes, 0 ms) Mar 30 15:23:17:624 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:17:853 [140603161552640] 5 hub: TSESS-P-4-3 [SecondaryHA] suspended Mar 30 15:23:17:855 [140602666645248] 5 hub: TSESS-P-14-0 [Secondary] suspended Mar 30 15:23:17:932 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:23:17:956 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:23:18:464 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (381 bytes) Mar 30 15:23:18:464 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:18:464 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (361 bytes) Mar 30 15:23:18:469 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:18:469 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (516 bytes, 0 ms) Mar 30 15:23:18:479 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (381 bytes) Mar 30 15:23:18:479 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:18:479 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (361 bytes) Mar 30 15:23:18:485 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:18:485 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (516 bytes, 0 ms) Mar 30 15:23:18:664 [140603169945344] 5 hub: TSESS-P-7-3 [TunnelB] suspended Mar 30 15:23:18:696 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:23:19:427 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:19:491 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (372 bytes) Mar 30 15:23:19:491 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:19:491 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (352 bytes) Mar 30 15:23:19:496 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:19:496 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (655 bytes, 0 ms) Mar 30 15:23:19:506 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (372 bytes) Mar 30 15:23:19:506 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:19:506 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (352 bytes) Mar 30 15:23:19:512 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:19:512 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (655 bytes, 0 ms) Mar 30 15:23:19:523 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (372 bytes) Mar 30 15:23:19:523 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:19:523 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (352 bytes) Mar 30 15:23:19:528 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:19:528 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (655 bytes, 0 ms) Mar 30 15:23:20:537 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (381 bytes) Mar 30 15:23:20:538 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:20:538 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (361 bytes) Mar 30 15:23:20:543 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:20:543 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (496 bytes, 0 ms) Mar 30 15:23:20:553 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (381 bytes) Mar 30 15:23:20:553 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:20:553 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (361 bytes) Mar 30 15:23:20:558 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:20:558 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (496 bytes, 0 ms) Mar 30 15:23:21:034 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:23:21:034 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:23:21:034 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60826 Mar 30 15:23:21:034 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60826' delegated to TSESS-A-5 [140603715176192] Mar 30 15:23:21:034 [140603715176192] 3 hub: TSESS-A-5-16 [TunnelB] restarted (for local) Mar 30 15:23:21:039 [140603715176192] 5 hub: TSESS-A-5-16 [TunnelB] local 100 'get_info' Mar 30 15:23:21:039 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:21:039 [140603715176192] 1 hub: TSESS-A-5-16 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:23:21:050 [140603715176192] 1 hub: TSESS-A-5-16 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:23:21:050 [140603715176192] 3 hub: TSESS-A-5-16 [TunnelB] is paired with [P-298-16] Mar 30 15:23:21:050 [140603715176192] 1 hub: TSESS-A-5-16 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:23:21:050 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60828 Mar 30 15:23:21:050 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60828' delegated to TSESS-A-10 [140603706783488] Mar 30 15:23:21:050 [140603706783488] 3 hub: TSESS-A-10-11 [TunnelB] restarted (for local) Mar 30 15:23:21:055 [140603715176192] 5 hub: TSESS-A-5-16 [TunnelB] local 100 '_close' Mar 30 15:23:21:055 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:21:055 [140603715176192] 1 hub: TSESS-A-5-16 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:23:21:055 [140603715176192] 3 hub: TSESS-A-5-16 [TunnelB] local session closed, suspending Mar 30 15:23:21:055 [140603706783488] 5 hub: TSESS-A-10-11 [TunnelB] local 100 '_status' Mar 30 15:23:21:055 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:21:055 [140603706783488] 1 hub: TSESS-A-10-11 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:23:21:065 [140603706783488] 1 hub: TSESS-A-10-11 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:23:21:065 [140603706783488] 3 hub: TSESS-A-10-11 [TunnelB] is paired with [P-300-11] Mar 30 15:23:21:065 [140603706783488] 1 hub: TSESS-A-10-11 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:23:21:071 [140603706783488] 5 hub: TSESS-A-10-11 [TunnelB] local 100 '_close' Mar 30 15:23:21:071 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:21:071 [140603706783488] 1 hub: TSESS-A-10-11 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:23:21:071 [140603706783488] 3 hub: TSESS-A-10-11 [TunnelB] local session closed, suspending Mar 30 15:23:21:229 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:21:255 [140603715176192] 5 hub: TSESS-A-5-16 [TunnelB] suspended Mar 30 15:23:21:271 [140603706783488] 5 hub: TSESS-A-10-11 [TunnelB] suspended Mar 30 15:23:21:565 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (378 bytes) Mar 30 15:23:21:565 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:21:565 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (358 bytes) Mar 30 15:23:21:571 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:21:571 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (478 bytes, 0 ms) Mar 30 15:23:21:581 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (378 bytes) Mar 30 15:23:21:581 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:21:581 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (358 bytes) Mar 30 15:23:21:586 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:21:586 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (478 bytes, 0 ms) Mar 30 15:23:22:595 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (381 bytes) Mar 30 15:23:22:595 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:22:595 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (361 bytes) Mar 30 15:23:22:601 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:22:601 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (496 bytes, 0 ms) Mar 30 15:23:22:611 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (381 bytes) Mar 30 15:23:22:611 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:22:611 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (361 bytes) Mar 30 15:23:22:616 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:22:616 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (496 bytes, 0 ms) Mar 30 15:23:23:032 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:23:623 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (379 bytes) Mar 30 15:23:23:623 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:23:623 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (359 bytes) Mar 30 15:23:23:628 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:23:628 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (484 bytes, 0 ms) Mar 30 15:23:23:639 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (379 bytes) Mar 30 15:23:23:639 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:23:639 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (359 bytes) Mar 30 15:23:23:644 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:23:644 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (484 bytes, 0 ms) Mar 30 15:23:24:651 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (375 bytes) Mar 30 15:23:24:651 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:24:651 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (355 bytes) Mar 30 15:23:24:656 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:24:656 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (460 bytes, 0 ms) Mar 30 15:23:24:666 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (375 bytes) Mar 30 15:23:24:666 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:24:666 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (355 bytes) Mar 30 15:23:24:671 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:24:671 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (460 bytes, 0 ms) Mar 30 15:23:25:678 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (378 bytes) Mar 30 15:23:25:678 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:25:678 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (358 bytes) Mar 30 15:23:25:683 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:25:683 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (478 bytes, 0 ms) Mar 30 15:23:25:694 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (378 bytes) Mar 30 15:23:25:694 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:25:694 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (358 bytes) Mar 30 15:23:25:699 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:25:699 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (478 bytes, 0 ms) Mar 30 15:23:25:735 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:26:706 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (384 bytes) Mar 30 15:23:26:706 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:26:706 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (364 bytes) Mar 30 15:23:26:712 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:26:712 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (500 bytes, 0 ms) Mar 30 15:23:26:722 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (384 bytes) Mar 30 15:23:26:722 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:26:722 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (364 bytes) Mar 30 15:23:26:727 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:26:727 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (500 bytes, 0 ms) Mar 30 15:23:26:952 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:23:26:984 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:23:27:040 [140605547460672] 4 hub: medium timeout Mar 30 15:23:27:040 [140605547460672] 3 hub: next_check [1585596396] now [1585596207] Mar 30 15:23:27:206 [140603203516160] 5 hub: TSESS-A-6-3 [Secondary] suspended Mar 30 15:23:27:238 [140603195123456] 5 hub: TSESS-A-3-3 [SecondaryHA] suspended Mar 30 15:23:27:537 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:27:730 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (381 bytes) Mar 30 15:23:27:730 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:27:730 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (361 bytes) Mar 30 15:23:27:735 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:27:735 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (496 bytes, 0 ms) Mar 30 15:23:27:745 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (381 bytes) Mar 30 15:23:27:745 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:27:745 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (361 bytes) Mar 30 15:23:27:750 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:27:750 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (496 bytes, 0 ms) Mar 30 15:23:27:853 [140603161552640] 5 hub: TSESS-P-4-3 [SecondaryHA] suspended Mar 30 15:23:27:855 [140602666645248] 5 hub: TSESS-P-14-0 [Secondary] suspended Mar 30 15:23:27:932 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:23:27:956 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:23:28:664 [140603169945344] 5 hub: TSESS-P-7-3 [TunnelB] suspended Mar 30 15:23:28:696 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:23:28:758 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (377 bytes) Mar 30 15:23:28:758 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:28:758 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (357 bytes) Mar 30 15:23:28:763 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:28:763 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (508 bytes, 0 ms) Mar 30 15:23:28:773 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (377 bytes) Mar 30 15:23:28:773 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:28:773 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (357 bytes) Mar 30 15:23:28:778 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:28:778 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (508 bytes, 0 ms) Mar 30 15:23:29:340 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:29:786 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (376 bytes) Mar 30 15:23:29:786 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:29:786 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (356 bytes) Mar 30 15:23:29:791 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:29:791 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (466 bytes, 0 ms) Mar 30 15:23:29:802 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (376 bytes) Mar 30 15:23:29:802 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:29:802 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (356 bytes) Mar 30 15:23:29:807 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:29:807 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (466 bytes, 0 ms) Mar 30 15:23:30:810 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (384 bytes) Mar 30 15:23:30:810 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:30:810 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (364 bytes) Mar 30 15:23:30:815 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:30:815 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (514 bytes, 0 ms) Mar 30 15:23:30:825 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (384 bytes) Mar 30 15:23:30:825 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:30:825 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (364 bytes) Mar 30 15:23:30:831 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:30:831 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (514 bytes, 0 ms) Mar 30 15:23:31:077 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:23:31:077 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:23:31:077 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60834 Mar 30 15:23:31:077 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60834' delegated to TSESS-A-5 [140603715176192] Mar 30 15:23:31:077 [140603715176192] 3 hub: TSESS-A-5-17 [TunnelB] restarted (for local) Mar 30 15:23:31:082 [140603715176192] 5 hub: TSESS-A-5-17 [TunnelB] local 100 'get_info' Mar 30 15:23:31:082 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:31:082 [140603715176192] 1 hub: TSESS-A-5-17 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:23:31:093 [140603715176192] 1 hub: TSESS-A-5-17 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:23:31:093 [140603715176192] 3 hub: TSESS-A-5-17 [TunnelB] is paired with [P-298-17] Mar 30 15:23:31:093 [140603715176192] 1 hub: TSESS-A-5-17 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:23:31:093 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60836 Mar 30 15:23:31:093 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60836' delegated to TSESS-A-10 [140603706783488] Mar 30 15:23:31:093 [140603706783488] 3 hub: TSESS-A-10-12 [TunnelB] restarted (for local) Mar 30 15:23:31:098 [140603715176192] 5 hub: TSESS-A-5-17 [TunnelB] local 100 '_close' Mar 30 15:23:31:098 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:31:098 [140603715176192] 1 hub: TSESS-A-5-17 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:23:31:098 [140603715176192] 3 hub: TSESS-A-5-17 [TunnelB] local session closed, suspending Mar 30 15:23:31:098 [140603706783488] 5 hub: TSESS-A-10-12 [TunnelB] local 100 '_status' Mar 30 15:23:31:098 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:31:098 [140603706783488] 1 hub: TSESS-A-10-12 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:23:31:108 [140603706783488] 1 hub: TSESS-A-10-12 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:23:31:108 [140603706783488] 3 hub: TSESS-A-10-12 [TunnelB] is paired with [P-300-12] Mar 30 15:23:31:108 [140603706783488] 1 hub: TSESS-A-10-12 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:23:31:114 [140603706783488] 5 hub: TSESS-A-10-12 [TunnelB] local 100 '_close' Mar 30 15:23:31:114 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:31:114 [140603706783488] 1 hub: TSESS-A-10-12 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:23:31:114 [140603706783488] 3 hub: TSESS-A-10-12 [TunnelB] local session closed, suspending Mar 30 15:23:31:142 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:31:298 [140603715176192] 5 hub: TSESS-A-5-17 [TunnelB] suspended Mar 30 15:23:31:314 [140603706783488] 5 hub: TSESS-A-10-12 [TunnelB] suspended Mar 30 15:23:31:841 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (372 bytes) Mar 30 15:23:31:841 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:31:841 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (352 bytes) Mar 30 15:23:31:846 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:31:846 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (464 bytes, 0 ms) Mar 30 15:23:31:856 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (372 bytes) Mar 30 15:23:31:856 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:31:856 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (352 bytes) Mar 30 15:23:31:861 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:31:861 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (464 bytes, 0 ms) Mar 30 15:23:32:868 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (380 bytes) Mar 30 15:23:32:868 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:32:869 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (360 bytes) Mar 30 15:23:32:874 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:32:874 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (490 bytes, 0 ms) Mar 30 15:23:32:884 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (380 bytes) Mar 30 15:23:32:884 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:32:884 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (360 bytes) Mar 30 15:23:32:889 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:32:889 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (490 bytes, 0 ms) Mar 30 15:23:33:845 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:33:896 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (379 bytes) Mar 30 15:23:33:896 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:33:896 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (359 bytes) Mar 30 15:23:33:901 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:33:901 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (484 bytes, 0 ms) Mar 30 15:23:33:911 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (379 bytes) Mar 30 15:23:33:911 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:33:911 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (359 bytes) Mar 30 15:23:33:917 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:33:917 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (484 bytes, 0 ms) Mar 30 15:23:34:802 [140604042360576] 5 hub: There are 0 objects waiting on 8 threads Mar 30 15:23:34:923 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (371 bytes) Mar 30 15:23:34:923 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:34:923 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (351 bytes) Mar 30 15:23:34:929 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:34:929 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (440 bytes, 0 ms) Mar 30 15:23:34:939 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (371 bytes) Mar 30 15:23:34:939 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:34:939 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (351 bytes) Mar 30 15:23:34:944 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:34:944 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (440 bytes, 0 ms) Mar 30 15:23:35:081 [140605547460672] 3 hub: check_session - session 0x20f9050 (127.0.0.1/48002) is a server Mar 30 15:23:35:081 [140605547460672] 2 hub: Profile - clayk 38 bytes (OK) Mar 30 15:23:35:648 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:35:946 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (383 bytes) Mar 30 15:23:35:946 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:35:946 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (363 bytes) Mar 30 15:23:35:951 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:35:952 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (508 bytes, 0 ms) Mar 30 15:23:35:962 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (383 bytes) Mar 30 15:23:35:962 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:35:962 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (363 bytes) Mar 30 15:23:35:967 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:35:967 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (508 bytes, 0 ms) Mar 30 15:23:36:952 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:23:36:974 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (371 bytes) Mar 30 15:23:36:974 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:36:974 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (351 bytes) Mar 30 15:23:36:979 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:36:979 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (681 bytes, 0 ms) Mar 30 15:23:36:984 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:23:36:990 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (371 bytes) Mar 30 15:23:36:990 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:36:990 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (351 bytes) Mar 30 15:23:36:995 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:36:995 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (681 bytes, 0 ms) Mar 30 15:23:37:005 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (371 bytes) Mar 30 15:23:37:005 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:37:005 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (351 bytes) Mar 30 15:23:37:010 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:37:010 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (681 bytes, 0 ms) Mar 30 15:23:37:083 [140605547460672] 4 hub: medium timeout Mar 30 15:23:37:083 [140605547460672] 3 hub: next_check [1585596396] now [1585596217] Mar 30 15:23:37:156 [140603740354304] 5 hub: TSESS-P-14 not internal command - thread will handle. mtype=100; sfd:29 Mar 30 15:23:37:156 [140603740354304] 2 hub: CTRL Secondary request 'hubup' passed to TSESS-P-14 [140602666645248] Mar 30 15:23:37:156 [140602666645248] 3 hub: TSESS-P-14-1 [Secondary] restarted (for remote) Mar 30 15:23:37:156 [140602666645248] 2 hub: TSESS-P-14 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:33:37 Mar 30 15:23:37:156 [140602666645248] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:23:37:156 [140602666645248] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:23:37:156 [140602666645248] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:23:37:156 [140602666645248] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:23:37:156 [140602666645248] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:23:37:156 [140602666645248] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:23:37:156 [140602666645248] 4 hub: CONNECT: 0x7fe094000be0(40) 127.0.0.1/53318->127.0.0.1/48002 Mar 30 15:23:37:156 [140602666645248] 3 hub: TSESS-P-14-1 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:23:37:156 [140602666645248] 1 hub: TSESS-P-14-1 [Secondary] received 100 'hubup' (683 bytes) Mar 30 15:23:37:156 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53318 (s:388) Mar 30 15:23:37:156 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/53318] Mar 30 15:23:37:156 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:23:37:156 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:23:37:156 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/53318 Mar 30 15:23:37:156 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/53318 same proximity=0 status=0 secver=217 Mar 30 15:23:37:161 [140602666645248] 5 hub: TSESS-P-14-1 [Secondary] local 200 'OK' Mar 30 15:23:37:161 [140602666645248] 1 hub: TSESS-P-14-1 [Secondary] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:23:37:172 [140602666645248] 1 hub: TSESS-P-14-1 [Secondary] ssl 100 '_close' (296 bytes) Mar 30 15:23:37:172 [140602666645248] 3 hub: TSESS-P-14-1 [Secondary] is paired with [A-2123-1] Mar 30 15:23:37:172 [140602666645248] 1 hub: TSESS-P-14-1 [Secondary] received 100 '_close' (296 bytes) Mar 30 15:23:37:177 [140602666645248] 1 hub: TSESS-P-14-1 [Secondary] remote session closed Mar 30 15:23:37:177 [140602666645248] 3 hub: TSESS-P-14-1 [Secondary] exiting msg loop (remote close) Mar 30 15:23:37:177 [140602666645248] 5 hub: TSESS-P-14-1 [Secondary] suspended Mar 30 15:23:37:206 [140603203516160] 5 hub: TSESS-A-6-3 [Secondary] suspended Mar 30 15:23:37:239 [140603195123456] 5 hub: TSESS-A-3-3 [SecondaryHA] suspended Mar 30 15:23:37:450 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:37:853 [140603161552640] 5 hub: TSESS-P-4-3 [SecondaryHA] suspended Mar 30 15:23:37:932 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:23:37:956 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:23:38:018 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (385 bytes) Mar 30 15:23:38:018 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:38:018 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (365 bytes) Mar 30 15:23:38:023 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:38:023 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (480 bytes, 0 ms) Mar 30 15:23:38:033 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (385 bytes) Mar 30 15:23:38:033 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:38:033 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (365 bytes) Mar 30 15:23:38:039 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:38:039 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (480 bytes, 0 ms) Mar 30 15:23:38:051 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:23:38:051 [140603731961600] 2 hub: CTRL SecondaryHA request 'hubup' passed to TSESS-P-4 [140603161552640] Mar 30 15:23:38:051 [140603161552640] 3 hub: TSESS-P-4-4 [SecondaryHA] restarted (for remote) Mar 30 15:23:38:051 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:33:37 Mar 30 15:23:38:051 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:23:38:051 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:23:38:051 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:23:38:051 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:23:38:051 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:23:38:051 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:23:38:051 [140603161552640] 4 hub: CONNECT: 0x7fe09c000c60(40) 127.0.0.1/53320->127.0.0.1/48002 Mar 30 15:23:38:051 [140603161552640] 3 hub: TSESS-P-4-4 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:23:38:051 [140603161552640] 1 hub: TSESS-P-4-4 [SecondaryHA] received 100 'hubup' (687 bytes) Mar 30 15:23:38:051 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53320 (s:392) Mar 30 15:23:38:052 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/53320] Mar 30 15:23:38:052 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:23:38:052 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:23:38:052 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/53320 Mar 30 15:23:38:052 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/53320 same proximity=0 status=0 secver=217 Mar 30 15:23:38:057 [140603161552640] 5 hub: TSESS-P-4-4 [SecondaryHA] local 200 'OK' Mar 30 15:23:38:057 [140603161552640] 1 hub: TSESS-P-4-4 [SecondaryHA] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:23:38:067 [140603161552640] 1 hub: TSESS-P-4-4 [SecondaryHA] ssl 100 '_close' (300 bytes) Mar 30 15:23:38:067 [140603161552640] 3 hub: TSESS-P-4-4 [SecondaryHA] is paired with [A-1948-4] Mar 30 15:23:38:067 [140603161552640] 1 hub: TSESS-P-4-4 [SecondaryHA] received 100 '_close' (296 bytes) Mar 30 15:23:38:072 [140603161552640] 1 hub: TSESS-P-4-4 [SecondaryHA] remote session closed Mar 30 15:23:38:072 [140603161552640] 3 hub: TSESS-P-4-4 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:23:38:072 [140603161552640] 5 hub: TSESS-P-4-4 [SecondaryHA] suspended Mar 30 15:23:38:664 [140603169945344] 5 hub: TSESS-P-7-3 [TunnelB] suspended Mar 30 15:23:38:696 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:23:38:760 [140603723568896] 5 hub: TSESS-P-7 not internal command - thread will handle. mtype=100; sfd:22 Mar 30 15:23:38:760 [140603723568896] 2 hub: CTRL TunnelB request 'hubup' passed to TSESS-P-7 [140603169945344] Mar 30 15:23:38:760 [140603169945344] 3 hub: TSESS-P-7-4 [TunnelB] restarted (for remote) Mar 30 15:23:38:760 [140603169945344] 2 hub: TSESS-P-7 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:33:38 Mar 30 15:23:38:760 [140603169945344] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:23:38:761 [140603169945344] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:23:38:761 [140603169945344] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:23:38:761 [140603169945344] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:23:38:761 [140603169945344] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:23:38:761 [140603169945344] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:23:38:761 [140603169945344] 4 hub: CONNECT: 0x7fe0a8000d20(41) 127.0.0.1/53322->127.0.0.1/48002 Mar 30 15:23:38:761 [140603169945344] 3 hub: TSESS-P-7-4 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:23:38:761 [140603169945344] 1 hub: TSESS-P-7-4 [TunnelB] received 100 'hubup' (672 bytes) Mar 30 15:23:38:761 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53322 (s:378) Mar 30 15:23:38:761 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/53322] Mar 30 15:23:38:761 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:23:38:761 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:23:38:761 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/53322 Mar 30 15:23:38:761 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/53322 same proximity=0 status=0 secver=217 Mar 30 15:23:38:766 [140603169945344] 5 hub: TSESS-P-7-4 [TunnelB] local 200 'OK' Mar 30 15:23:38:766 [140603169945344] 1 hub: TSESS-P-7-4 [TunnelB] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:23:38:776 [140603169945344] 1 hub: TSESS-P-7-4 [TunnelB] ssl 100 '_close' (283 bytes) Mar 30 15:23:38:776 [140603169945344] 3 hub: TSESS-P-7-4 [TunnelB] is paired with [A-299-4] Mar 30 15:23:38:776 [140603169945344] 1 hub: TSESS-P-7-4 [TunnelB] received 100 '_close' (295 bytes) Mar 30 15:23:38:781 [140603169945344] 1 hub: TSESS-P-7-4 [TunnelB] remote session closed Mar 30 15:23:38:781 [140603169945344] 3 hub: TSESS-P-7-4 [TunnelB] exiting msg loop (remote close) Mar 30 15:23:38:781 [140603169945344] 5 hub: TSESS-P-7-4 [TunnelB] suspended Mar 30 15:23:39:047 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (385 bytes) Mar 30 15:23:39:047 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:39:047 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (365 bytes) Mar 30 15:23:39:052 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:39:052 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (484 bytes, 0 ms) Mar 30 15:23:39:062 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (385 bytes) Mar 30 15:23:39:062 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:39:062 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (365 bytes) Mar 30 15:23:39:067 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:39:067 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (484 bytes, 0 ms) Mar 30 15:23:39:253 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:40:076 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (383 bytes) Mar 30 15:23:40:076 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:40:076 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (363 bytes) Mar 30 15:23:40:081 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:40:081 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (279 bytes, 0 ms) Mar 30 15:23:41:055 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:41:088 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (367 bytes) Mar 30 15:23:41:088 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:41:088 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (347 bytes) Mar 30 15:23:41:093 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:41:093 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (437 bytes, 0 ms) Mar 30 15:23:41:104 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (367 bytes) Mar 30 15:23:41:104 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:41:104 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (347 bytes) Mar 30 15:23:41:109 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:41:109 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (437 bytes, 0 ms) Mar 30 15:23:41:122 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:23:41:122 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:23:41:123 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60850 Mar 30 15:23:41:123 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60850' delegated to TSESS-A-5 [140603715176192] Mar 30 15:23:41:123 [140603715176192] 3 hub: TSESS-A-5-18 [TunnelB] restarted (for local) Mar 30 15:23:41:128 [140603715176192] 5 hub: TSESS-A-5-18 [TunnelB] local 100 'get_info' Mar 30 15:23:41:128 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:41:128 [140603715176192] 1 hub: TSESS-A-5-18 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:23:41:138 [140603715176192] 1 hub: TSESS-A-5-18 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:23:41:138 [140603715176192] 3 hub: TSESS-A-5-18 [TunnelB] is paired with [P-298-18] Mar 30 15:23:41:138 [140603715176192] 1 hub: TSESS-A-5-18 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:23:41:139 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60852 Mar 30 15:23:41:139 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60852' delegated to TSESS-A-10 [140603706783488] Mar 30 15:23:41:139 [140603706783488] 3 hub: TSESS-A-10-13 [TunnelB] restarted (for local) Mar 30 15:23:41:144 [140603715176192] 5 hub: TSESS-A-5-18 [TunnelB] local 100 '_close' Mar 30 15:23:41:144 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:41:144 [140603715176192] 1 hub: TSESS-A-5-18 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:23:41:144 [140603715176192] 3 hub: TSESS-A-5-18 [TunnelB] local session closed, suspending Mar 30 15:23:41:144 [140603706783488] 5 hub: TSESS-A-10-13 [TunnelB] local 100 '_status' Mar 30 15:23:41:144 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:41:144 [140603706783488] 1 hub: TSESS-A-10-13 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:23:41:154 [140603706783488] 1 hub: TSESS-A-10-13 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:23:41:154 [140603706783488] 3 hub: TSESS-A-10-13 [TunnelB] is paired with [P-300-13] Mar 30 15:23:41:154 [140603706783488] 1 hub: TSESS-A-10-13 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:23:41:160 [140603706783488] 5 hub: TSESS-A-10-13 [TunnelB] local 100 '_close' Mar 30 15:23:41:160 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:41:160 [140603706783488] 1 hub: TSESS-A-10-13 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:23:41:160 [140603706783488] 3 hub: TSESS-A-10-13 [TunnelB] local session closed, suspending Mar 30 15:23:41:344 [140603715176192] 5 hub: TSESS-A-5-18 [TunnelB] suspended Mar 30 15:23:41:360 [140603706783488] 5 hub: TSESS-A-10-13 [TunnelB] suspended Mar 30 15:23:42:116 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (380 bytes) Mar 30 15:23:42:116 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:42:117 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (360 bytes) Mar 30 15:23:42:122 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:42:122 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (500 bytes, 0 ms) Mar 30 15:23:42:132 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (380 bytes) Mar 30 15:23:42:132 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:42:132 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (360 bytes) Mar 30 15:23:42:137 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:42:137 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (500 bytes, 0 ms) Mar 30 15:23:43:144 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (387 bytes) Mar 30 15:23:43:144 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:43:144 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (367 bytes) Mar 30 15:23:43:149 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:43:149 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (551 bytes, 0 ms) Mar 30 15:23:43:159 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (387 bytes) Mar 30 15:23:43:160 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:43:160 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (367 bytes) Mar 30 15:23:43:165 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:43:165 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (551 bytes, 0 ms) Mar 30 15:23:43:759 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:44:171 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (390 bytes) Mar 30 15:23:44:171 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:44:171 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (370 bytes) Mar 30 15:23:44:176 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:44:176 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (342 bytes, 0 ms) Mar 30 15:23:45:186 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (386 bytes) Mar 30 15:23:45:186 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:45:186 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (366 bytes) Mar 30 15:23:45:191 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:45:191 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (557 bytes, 0 ms) Mar 30 15:23:45:201 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (386 bytes) Mar 30 15:23:45:201 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:45:201 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (366 bytes) Mar 30 15:23:45:207 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:45:207 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (557 bytes, 0 ms) Mar 30 15:23:45:561 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:46:066 [140605283890944] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:23:46:066 [140605283890944] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:23:46:066 [140605283890944] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:23:46:066 [140605283890944] 1 hub: PERF: hubup_check_interval: 0 sec hubhash_size: 18 Mar 30 15:23:46:066 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:066 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:066 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:066 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:066 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:066 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:066 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:23:46:066 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:066 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:066 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:066 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:066 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:066 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:066 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:066 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:066 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:23:46:066 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:23:46:066 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:23:46:066 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(40) 127.0.0.1/46996->127.0.0.1/48007 Mar 30 15:23:46:066 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/46996 Mar 30 15:23:46:066 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/46996' delegated to TSESS-A-6 [140603203516160] Mar 30 15:23:46:067 [140603203516160] 3 hub: TSESS-A-6-4 [Secondary] restarted (for local) Mar 30 15:23:46:072 [140603203516160] 5 hub: TSESS-A-6-4 [Secondary] local 100 'hubup' Mar 30 15:23:46:072 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:46:072 [140603203516160] 1 hub: TSESS-A-6-4 [Secondary] sent 100 'hubup' (686 bytes, 0 ms) Mar 30 15:23:46:082 [140603203516160] 1 hub: TSESS-A-6-4 [Secondary] ssl 200 '-' (453 bytes) Mar 30 15:23:46:082 [140603203516160] 3 hub: TSESS-A-6-4 [Secondary] is paired with [P-2120-4] Mar 30 15:23:46:082 [140603203516160] 1 hub: TSESS-A-6-4 [Secondary] received 200 '-' (461 bytes) Mar 30 15:23:46:082 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:23:46:082 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:23:46:082 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:23:46:082 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:23:46:082 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary same proximity=0 status=0 secver=217 Mar 30 15:23:46:082 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:082 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:082 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:082 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:082 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:082 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:082 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:23:46:082 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:082 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:082 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:082 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:082 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:082 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:082 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:082 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:082 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:23:46:082 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:23:46:082 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:23:46:082 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(40) 127.0.0.1/60860->127.0.0.1/48009 Mar 30 15:23:46:082 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60860 Mar 30 15:23:46:082 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60860' delegated to TSESS-A-5 [140603715176192] Mar 30 15:23:46:082 [140603715176192] 3 hub: TSESS-A-5-19 [TunnelB] restarted (for local) Mar 30 15:23:46:087 [140603203516160] 5 hub: TSESS-A-6-4 [Secondary] local 100 '_close' Mar 30 15:23:46:087 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:46:087 [140603203516160] 1 hub: TSESS-A-6-4 [Secondary] sent 100 '_close' (293 bytes, 0 ms) Mar 30 15:23:46:087 [140603203516160] 3 hub: TSESS-A-6-4 [Secondary] local session closed, suspending Mar 30 15:23:46:088 [140603715176192] 5 hub: TSESS-A-5-19 [TunnelB] local 100 'hubup' Mar 30 15:23:46:088 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:46:088 [140603715176192] 1 hub: TSESS-A-5-19 [TunnelB] sent 100 'hubup' (685 bytes, 0 ms) Mar 30 15:23:46:098 [140603715176192] 1 hub: TSESS-A-5-19 [TunnelB] ssl 200 '-' (443 bytes) Mar 30 15:23:46:098 [140603715176192] 3 hub: TSESS-A-5-19 [TunnelB] is paired with [P-298-19] Mar 30 15:23:46:098 [140603715176192] 1 hub: TSESS-A-5-19 [TunnelB] received 200 '-' (451 bytes) Mar 30 15:23:46:098 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:23:46:098 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:23:46:098 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:23:46:098 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:23:46:098 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB same proximity=0 status=0 secver=217 Mar 30 15:23:46:098 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:098 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:098 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:098 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:098 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:098 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:098 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:23:46:098 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:098 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:098 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:098 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:098 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:098 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:098 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:098 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:23:46:098 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:23:46:098 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:23:46:098 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:23:46:098 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(40) 127.0.0.1/40048->127.0.0.1/48008 Mar 30 15:23:46:098 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/40048 Mar 30 15:23:46:098 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/40048' delegated to TSESS-A-3 [140603195123456] Mar 30 15:23:46:098 [140603195123456] 3 hub: TSESS-A-3-4 [SecondaryHA] restarted (for local) Mar 30 15:23:46:103 [140603715176192] 5 hub: TSESS-A-5-19 [TunnelB] local 100 '_close' Mar 30 15:23:46:103 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:46:103 [140603715176192] 1 hub: TSESS-A-5-19 [TunnelB] sent 100 '_close' (292 bytes, 0 ms) Mar 30 15:23:46:103 [140603715176192] 3 hub: TSESS-A-5-19 [TunnelB] local session closed, suspending Mar 30 15:23:46:104 [140603195123456] 5 hub: TSESS-A-3-4 [SecondaryHA] local 100 'hubup' Mar 30 15:23:46:104 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:46:104 [140603195123456] 1 hub: TSESS-A-3-4 [SecondaryHA] sent 100 'hubup' (688 bytes, 0 ms) Mar 30 15:23:46:114 [140603195123456] 1 hub: TSESS-A-3-4 [SecondaryHA] ssl 200 '-' (457 bytes) Mar 30 15:23:46:114 [140603195123456] 3 hub: TSESS-A-3-4 [SecondaryHA] is paired with [P-1947-4] Mar 30 15:23:46:114 [140603195123456] 1 hub: TSESS-A-3-4 [SecondaryHA] received 200 '-' (465 bytes) Mar 30 15:23:46:114 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:23:46:114 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:23:46:114 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:23:46:114 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:23:46:114 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA same proximity=0 status=0 secver=217 Mar 30 15:23:46:114 [140605283890944] 5 hub: mark_hubup_list_updated Mar 30 15:23:46:114 [140605283890944] 5 hub: hublist changed: /DOMAIN/Primary 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:46:114 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:46:114 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:46:114 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:46:114 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:46:114 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:46:114 [140605283890944] 5 hub: hublist changed: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:23:46:114 [140605283890944] 5 hub: hublist changed: /DOMAIN/PrimaryHA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:46:114 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:46:114 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:46:114 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:46:114 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:46:114 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:46:114 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:46:114 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:23:46:114 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:23:46:114 [140605283890944] 5 hub: hublist changed: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:23:46:114 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelBHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:23:46:114 [140605283890944] 5 hub: mark_hubup_list_updated end Mar 30 15:23:46:114 [140605283890944] 1 hub: PERF: hubup_check_for_updates: 0 sec hubhash_size: 18 Mar 30 15:23:46:114 [140605283890944] 1 hub: PERF: hubi_run_command: 0 sec hubhash_size: 18 Mar 30 15:23:46:114 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:23:46:115 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:23:46:115 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:23:46:116 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:23:46:116 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:23:46:117 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:23:46:117 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:23:46:117 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:23:46:118 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:23:46:118 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:23:46:118 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:23:46:119 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:23:46:119 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:23:46:120 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:23:46:120 [140603195123456] 5 hub: TSESS-A-3-4 [SecondaryHA] local 100 '_close' Mar 30 15:23:46:120 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:46:120 [140603195123456] 1 hub: TSESS-A-3-4 [SecondaryHA] sent 100 '_close' (295 bytes, 0 ms) Mar 30 15:23:46:120 [140603195123456] 3 hub: TSESS-A-3-4 [SecondaryHA] local session closed, suspending Mar 30 15:23:46:121 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:23:46:121 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:23:46:122 [140605283890944] 1 hub: PERF: hubi_check_security: 0 sec hubhash_size: 18 Mar 30 15:23:46:213 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (392 bytes) Mar 30 15:23:46:213 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:46:213 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (372 bytes) Mar 30 15:23:46:219 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:46:219 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (353 bytes, 0 ms) Mar 30 15:23:46:287 [140603203516160] 5 hub: TSESS-A-6-4 [Secondary] suspended Mar 30 15:23:46:303 [140603715176192] 5 hub: TSESS-A-5-19 [TunnelB] suspended Mar 30 15:23:46:320 [140603195123456] 5 hub: TSESS-A-3-4 [SecondaryHA] suspended Mar 30 15:23:46:952 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:23:46:970 [140605547460672] 2 hub: verify_login from ctrl 127.0.0.1/53358 Mar 30 15:23:46:970 [140605547460672] 1 hub: verify - [target] prid=controller cmd=probe_list ip=10.x.x.x perm=1 Mar 30 15:23:46:970 [140605547460672] 1 hub: verify - [source] id=ppm ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:46:985 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:23:47:177 [140602666645248] 5 hub: TSESS-P-14-1 [Secondary] suspended Mar 30 15:23:47:227 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (382 bytes) Mar 30 15:23:47:227 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:47:227 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (362 bytes) Mar 30 15:23:47:232 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:47:232 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (323 bytes, 0 ms) Mar 30 15:23:47:364 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:47:933 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:23:47:956 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:23:47:971 [140605547460672] 4 hub: medium timeout Mar 30 15:23:47:971 [140605547460672] 3 hub: next_check [1585596396] now [1585596227] Mar 30 15:23:48:072 [140603161552640] 5 hub: TSESS-P-4-4 [SecondaryHA] suspended Mar 30 15:23:48:239 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (391 bytes) Mar 30 15:23:48:239 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:48:240 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (371 bytes) Mar 30 15:23:48:245 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:48:245 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (350 bytes, 0 ms) Mar 30 15:23:48:696 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:23:48:782 [140603169945344] 5 hub: TSESS-P-7-4 [TunnelB] suspended Mar 30 15:23:49:166 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:49:252 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (386 bytes) Mar 30 15:23:49:252 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:49:252 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (366 bytes) Mar 30 15:23:49:258 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:49:258 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (335 bytes, 0 ms) Mar 30 15:23:50:265 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (381 bytes) Mar 30 15:23:50:265 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:50:265 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (361 bytes) Mar 30 15:23:50:270 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:50:270 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (320 bytes, 0 ms) Mar 30 15:23:51:166 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:23:51:166 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:23:51:166 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60888 Mar 30 15:23:51:166 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60888' delegated to TSESS-A-5 [140603715176192] Mar 30 15:23:51:166 [140603715176192] 3 hub: TSESS-A-5-20 [TunnelB] restarted (for local) Mar 30 15:23:51:171 [140603715176192] 5 hub: TSESS-A-5-20 [TunnelB] local 100 'get_info' Mar 30 15:23:51:171 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:51:171 [140603715176192] 1 hub: TSESS-A-5-20 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:23:51:182 [140603715176192] 1 hub: TSESS-A-5-20 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:23:51:182 [140603715176192] 3 hub: TSESS-A-5-20 [TunnelB] is paired with [P-298-20] Mar 30 15:23:51:182 [140603715176192] 1 hub: TSESS-A-5-20 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:23:51:182 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60890 Mar 30 15:23:51:182 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60890' delegated to TSESS-A-10 [140603706783488] Mar 30 15:23:51:182 [140603706783488] 3 hub: TSESS-A-10-14 [TunnelB] restarted (for local) Mar 30 15:23:51:187 [140603715176192] 5 hub: TSESS-A-5-20 [TunnelB] local 100 '_close' Mar 30 15:23:51:187 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:51:187 [140603715176192] 1 hub: TSESS-A-5-20 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:23:51:187 [140603715176192] 3 hub: TSESS-A-5-20 [TunnelB] local session closed, suspending Mar 30 15:23:51:187 [140603706783488] 5 hub: TSESS-A-10-14 [TunnelB] local 100 '_status' Mar 30 15:23:51:187 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:51:187 [140603706783488] 1 hub: TSESS-A-10-14 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:23:51:198 [140603706783488] 1 hub: TSESS-A-10-14 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:23:51:198 [140603706783488] 3 hub: TSESS-A-10-14 [TunnelB] is paired with [P-300-14] Mar 30 15:23:51:198 [140603706783488] 1 hub: TSESS-A-10-14 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:23:51:203 [140603706783488] 5 hub: TSESS-A-10-14 [TunnelB] local 100 '_close' Mar 30 15:23:51:203 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:23:51:203 [140603706783488] 1 hub: TSESS-A-10-14 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:23:51:203 [140603706783488] 3 hub: TSESS-A-10-14 [TunnelB] local session closed, suspending Mar 30 15:23:51:271 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (382 bytes) Mar 30 15:23:51:271 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:51:271 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (362 bytes) Mar 30 15:23:51:276 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:51:276 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (323 bytes, 0 ms) Mar 30 15:23:51:388 [140603715176192] 5 hub: TSESS-A-5-20 [TunnelB] suspended Mar 30 15:23:51:403 [140603706783488] 5 hub: TSESS-A-10-14 [TunnelB] suspended Mar 30 15:23:51:870 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:52:284 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (386 bytes) Mar 30 15:23:52:284 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:52:284 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (366 bytes) Mar 30 15:23:52:289 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:52:289 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (342 bytes, 0 ms) Mar 30 15:23:53:299 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (385 bytes) Mar 30 15:23:53:299 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:53:299 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (365 bytes) Mar 30 15:23:53:304 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:53:304 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (333 bytes, 0 ms) Mar 30 15:23:53:672 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:54:305 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (380 bytes) Mar 30 15:23:54:305 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:54:305 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (360 bytes) Mar 30 15:23:54:310 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:54:310 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (497 bytes, 0 ms) Mar 30 15:23:54:320 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (380 bytes) Mar 30 15:23:54:320 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:54:321 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (360 bytes) Mar 30 15:23:54:326 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:54:326 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (497 bytes, 0 ms) Mar 30 15:23:55:333 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (394 bytes) Mar 30 15:23:55:333 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:55:333 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (374 bytes) Mar 30 15:23:55:338 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:55:338 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (621 bytes, 0 ms) Mar 30 15:23:55:348 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (394 bytes) Mar 30 15:23:55:348 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:55:349 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (374 bytes) Mar 30 15:23:55:354 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:55:354 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (621 bytes, 0 ms) Mar 30 15:23:55:474 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:56:288 [140603203516160] 5 hub: TSESS-A-6-4 [Secondary] suspended Mar 30 15:23:56:320 [140603195123456] 5 hub: TSESS-A-3-4 [SecondaryHA] suspended Mar 30 15:23:56:361 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (370 bytes) Mar 30 15:23:56:361 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:56:361 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (350 bytes) Mar 30 15:23:56:366 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:56:367 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (481 bytes, 0 ms) Mar 30 15:23:56:377 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (370 bytes) Mar 30 15:23:56:377 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:56:377 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (350 bytes) Mar 30 15:23:56:382 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:56:382 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (481 bytes, 0 ms) Mar 30 15:23:56:953 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:23:56:985 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:23:57:172 [140605547460672] 4 hub: medium timeout Mar 30 15:23:57:172 [140605547460672] 3 hub: next_check [1585596396] now [1585596237] Mar 30 15:23:57:177 [140602666645248] 5 hub: TSESS-P-14-1 [Secondary] suspended Mar 30 15:23:57:277 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:57:390 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (393 bytes) Mar 30 15:23:57:390 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:57:390 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (373 bytes) Mar 30 15:23:57:395 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:57:395 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (543 bytes, 0 ms) Mar 30 15:23:57:405 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (393 bytes) Mar 30 15:23:57:405 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:57:405 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (373 bytes) Mar 30 15:23:57:410 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:57:411 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (543 bytes, 0 ms) Mar 30 15:23:57:933 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:23:57:956 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:23:58:072 [140603161552640] 5 hub: TSESS-P-4-4 [SecondaryHA] suspended Mar 30 15:23:58:419 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (381 bytes) Mar 30 15:23:58:420 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:58:420 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (361 bytes) Mar 30 15:23:58:425 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:58:425 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (477 bytes, 0 ms) Mar 30 15:23:58:435 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (381 bytes) Mar 30 15:23:58:435 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:58:435 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (361 bytes) Mar 30 15:23:58:440 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:58:440 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (477 bytes, 0 ms) Mar 30 15:23:58:696 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:23:58:782 [140603169945344] 5 hub: TSESS-P-7-4 [TunnelB] suspended Mar 30 15:23:59:079 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:23:59:448 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (369 bytes) Mar 30 15:23:59:448 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:59:448 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (349 bytes) Mar 30 15:23:59:453 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:59:453 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (593 bytes, 0 ms) Mar 30 15:23:59:464 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (369 bytes) Mar 30 15:23:59:464 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:59:464 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (349 bytes) Mar 30 15:23:59:469 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:59:469 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (593 bytes, 0 ms) Mar 30 15:23:59:479 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (369 bytes) Mar 30 15:23:59:479 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:23:59:479 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (349 bytes) Mar 30 15:23:59:484 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:23:59:484 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (593 bytes, 0 ms) Mar 30 15:24:00:493 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (379 bytes) Mar 30 15:24:00:493 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:00:493 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (359 bytes) Mar 30 15:24:00:498 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:00:498 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (483 bytes, 0 ms) Mar 30 15:24:00:508 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (379 bytes) Mar 30 15:24:00:508 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:00:508 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (359 bytes) Mar 30 15:24:00:513 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:00:514 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (483 bytes, 0 ms) Mar 30 15:24:01:209 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:24:01:209 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:24:01:209 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60896 Mar 30 15:24:01:209 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60896' delegated to TSESS-A-5 [140603715176192] Mar 30 15:24:01:209 [140603715176192] 3 hub: TSESS-A-5-21 [TunnelB] restarted (for local) Mar 30 15:24:01:214 [140603715176192] 5 hub: TSESS-A-5-21 [TunnelB] local 100 'get_info' Mar 30 15:24:01:214 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:01:214 [140603715176192] 1 hub: TSESS-A-5-21 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:24:01:225 [140603715176192] 1 hub: TSESS-A-5-21 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:24:01:225 [140603715176192] 3 hub: TSESS-A-5-21 [TunnelB] is paired with [P-298-21] Mar 30 15:24:01:225 [140603715176192] 1 hub: TSESS-A-5-21 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:24:01:225 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60898 Mar 30 15:24:01:225 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60898' delegated to TSESS-A-10 [140603706783488] Mar 30 15:24:01:225 [140603706783488] 3 hub: TSESS-A-10-15 [TunnelB] restarted (for local) Mar 30 15:24:01:230 [140603715176192] 5 hub: TSESS-A-5-21 [TunnelB] local 100 '_close' Mar 30 15:24:01:230 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:01:230 [140603715176192] 1 hub: TSESS-A-5-21 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:24:01:230 [140603715176192] 3 hub: TSESS-A-5-21 [TunnelB] local session closed, suspending Mar 30 15:24:01:230 [140603706783488] 5 hub: TSESS-A-10-15 [TunnelB] local 100 '_status' Mar 30 15:24:01:230 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:01:230 [140603706783488] 1 hub: TSESS-A-10-15 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:24:01:240 [140603706783488] 1 hub: TSESS-A-10-15 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:24:01:240 [140603706783488] 3 hub: TSESS-A-10-15 [TunnelB] is paired with [P-300-15] Mar 30 15:24:01:240 [140603706783488] 1 hub: TSESS-A-10-15 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:24:01:246 [140603706783488] 5 hub: TSESS-A-10-15 [TunnelB] local 100 '_close' Mar 30 15:24:01:246 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:01:246 [140603706783488] 1 hub: TSESS-A-10-15 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:24:01:246 [140603706783488] 3 hub: TSESS-A-10-15 [TunnelB] local session closed, suspending Mar 30 15:24:01:430 [140603715176192] 5 hub: TSESS-A-5-21 [TunnelB] suspended Mar 30 15:24:01:446 [140603706783488] 5 hub: TSESS-A-10-15 [TunnelB] suspended Mar 30 15:24:01:521 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (368 bytes) Mar 30 15:24:01:521 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:01:521 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (348 bytes) Mar 30 15:24:01:527 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:01:527 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (857 bytes, 0 ms) Mar 30 15:24:01:537 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (368 bytes) Mar 30 15:24:01:537 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:01:537 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (348 bytes) Mar 30 15:24:01:542 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:01:542 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (857 bytes, 0 ms) Mar 30 15:24:01:552 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (368 bytes) Mar 30 15:24:01:552 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:01:552 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (348 bytes) Mar 30 15:24:01:558 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:01:558 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (857 bytes, 0 ms) Mar 30 15:24:01:568 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (368 bytes) Mar 30 15:24:01:568 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:01:568 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (348 bytes) Mar 30 15:24:01:573 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:01:573 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (857 bytes, 0 ms) Mar 30 15:24:01:782 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:02:578 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (383 bytes) Mar 30 15:24:02:578 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:02:578 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (363 bytes) Mar 30 15:24:02:583 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:02:584 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (555 bytes, 0 ms) Mar 30 15:24:02:594 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (383 bytes) Mar 30 15:24:02:594 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:02:594 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (363 bytes) Mar 30 15:24:02:599 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:02:599 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (555 bytes, 0 ms) Mar 30 15:24:03:585 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:03:606 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (380 bytes) Mar 30 15:24:03:606 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:03:606 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (360 bytes) Mar 30 15:24:03:612 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:03:612 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (462 bytes, 0 ms) Mar 30 15:24:03:622 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (380 bytes) Mar 30 15:24:03:622 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:03:622 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (360 bytes) Mar 30 15:24:03:627 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:03:627 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (462 bytes, 0 ms) Mar 30 15:24:04:634 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (377 bytes) Mar 30 15:24:04:634 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:04:634 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (357 bytes) Mar 30 15:24:04:639 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:04:639 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (617 bytes, 0 ms) Mar 30 15:24:04:650 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (377 bytes) Mar 30 15:24:04:650 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:04:650 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (357 bytes) Mar 30 15:24:04:655 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:04:655 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (617 bytes, 0 ms) Mar 30 15:24:04:665 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (377 bytes) Mar 30 15:24:04:665 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:04:665 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (357 bytes) Mar 30 15:24:04:670 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:04:670 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (617 bytes, 0 ms) Mar 30 15:24:05:213 [140605547460672] 3 hub: check_session - session 0x20f9050 (127.0.0.1/48002) is a server Mar 30 15:24:05:213 [140605547460672] 2 hub: Profile - clayk 38 bytes (OK) Mar 30 15:24:05:387 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:05:677 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (381 bytes) Mar 30 15:24:05:677 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:05:677 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (361 bytes) Mar 30 15:24:05:682 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:05:682 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (482 bytes, 0 ms) Mar 30 15:24:05:693 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (381 bytes) Mar 30 15:24:05:693 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:05:693 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (361 bytes) Mar 30 15:24:05:698 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:05:698 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (482 bytes, 0 ms) Mar 30 15:24:06:288 [140603203516160] 5 hub: TSESS-A-6-4 [Secondary] suspended Mar 30 15:24:06:321 [140603195123456] 5 hub: TSESS-A-3-4 [SecondaryHA] suspended Mar 30 15:24:06:702 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (382 bytes) Mar 30 15:24:06:702 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:06:702 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (362 bytes) Mar 30 15:24:06:707 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:06:707 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (489 bytes, 0 ms) Mar 30 15:24:06:717 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (382 bytes) Mar 30 15:24:06:717 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:06:717 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (362 bytes) Mar 30 15:24:06:722 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:06:723 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (489 bytes, 0 ms) Mar 30 15:24:06:953 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:24:06:985 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:24:07:177 [140602666645248] 5 hub: TSESS-P-14-1 [Secondary] suspended Mar 30 15:24:07:187 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:24:07:187 [140603731961600] 2 hub: CTRL SecondaryHA request 'hubup' passed to TSESS-P-4 [140603161552640] Mar 30 15:24:07:187 [140603161552640] 3 hub: TSESS-P-4-5 [SecondaryHA] restarted (for remote) Mar 30 15:24:07:187 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:34:07 Mar 30 15:24:07:187 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:24:07:188 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:24:07:188 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:24:07:188 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:24:07:188 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:24:07:188 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:24:07:188 [140603161552640] 4 hub: CONNECT: 0x7fe09c000c60(41) 127.0.0.1/53378->127.0.0.1/48002 Mar 30 15:24:07:188 [140603161552640] 3 hub: TSESS-P-4-5 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:24:07:188 [140603161552640] 1 hub: TSESS-P-4-5 [SecondaryHA] received 100 'hubup' (687 bytes) Mar 30 15:24:07:188 [140605547460672] 4 hub: medium timeout Mar 30 15:24:07:188 [140605547460672] 3 hub: next_check [1585596396] now [1585596247] Mar 30 15:24:07:188 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53378 (s:392) Mar 30 15:24:07:188 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/53378] Mar 30 15:24:07:188 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:24:07:188 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:24:07:188 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/53378 Mar 30 15:24:07:188 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/53378 same proximity=0 status=0 secver=217 Mar 30 15:24:07:190 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:07:193 [140603161552640] 5 hub: TSESS-P-4-5 [SecondaryHA] local 200 'OK' Mar 30 15:24:07:193 [140603161552640] 1 hub: TSESS-P-4-5 [SecondaryHA] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:24:07:203 [140603161552640] 1 hub: TSESS-P-4-5 [SecondaryHA] ssl 100 '_close' (300 bytes) Mar 30 15:24:07:203 [140603161552640] 3 hub: TSESS-P-4-5 [SecondaryHA] is paired with [A-1948-5] Mar 30 15:24:07:203 [140603161552640] 1 hub: TSESS-P-4-5 [SecondaryHA] received 100 '_close' (296 bytes) Mar 30 15:24:07:208 [140603161552640] 1 hub: TSESS-P-4-5 [SecondaryHA] remote session closed Mar 30 15:24:07:208 [140603161552640] 3 hub: TSESS-P-4-5 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:24:07:208 [140603161552640] 5 hub: TSESS-P-4-5 [SecondaryHA] suspended Mar 30 15:24:07:308 [140603740354304] 5 hub: TSESS-P-14 not internal command - thread will handle. mtype=100; sfd:29 Mar 30 15:24:07:308 [140603740354304] 2 hub: CTRL Secondary request 'hubup' passed to TSESS-P-14 [140602666645248] Mar 30 15:24:07:308 [140602666645248] 3 hub: TSESS-P-14-2 [Secondary] restarted (for remote) Mar 30 15:24:07:309 [140602666645248] 2 hub: TSESS-P-14 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:34:07 Mar 30 15:24:07:309 [140602666645248] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:24:07:309 [140602666645248] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:24:07:309 [140602666645248] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:24:07:309 [140602666645248] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:24:07:309 [140602666645248] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:24:07:309 [140602666645248] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:24:07:309 [140602666645248] 4 hub: CONNECT: 0x7fe094000bb0(41) 127.0.0.1/53380->127.0.0.1/48002 Mar 30 15:24:07:309 [140602666645248] 3 hub: TSESS-P-14-2 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:24:07:309 [140602666645248] 1 hub: TSESS-P-14-2 [Secondary] received 100 'hubup' (683 bytes) Mar 30 15:24:07:309 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53380 (s:388) Mar 30 15:24:07:309 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/53380] Mar 30 15:24:07:309 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:24:07:309 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:24:07:309 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/53380 Mar 30 15:24:07:309 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/53380 same proximity=0 status=0 secver=217 Mar 30 15:24:07:314 [140602666645248] 5 hub: TSESS-P-14-2 [Secondary] local 200 'OK' Mar 30 15:24:07:314 [140602666645248] 1 hub: TSESS-P-14-2 [Secondary] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:24:07:324 [140602666645248] 1 hub: TSESS-P-14-2 [Secondary] ssl 100 '_close' (296 bytes) Mar 30 15:24:07:324 [140602666645248] 3 hub: TSESS-P-14-2 [Secondary] is paired with [A-2123-2] Mar 30 15:24:07:324 [140602666645248] 1 hub: TSESS-P-14-2 [Secondary] received 100 '_close' (296 bytes) Mar 30 15:24:07:329 [140602666645248] 1 hub: TSESS-P-14-2 [Secondary] remote session closed Mar 30 15:24:07:329 [140602666645248] 3 hub: TSESS-P-14-2 [Secondary] exiting msg loop (remote close) Mar 30 15:24:07:329 [140602666645248] 5 hub: TSESS-P-14-2 [Secondary] suspended Mar 30 15:24:07:728 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (372 bytes) Mar 30 15:24:07:729 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:07:729 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (352 bytes) Mar 30 15:24:07:734 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:07:734 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (653 bytes, 0 ms) Mar 30 15:24:07:744 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (372 bytes) Mar 30 15:24:07:744 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:07:744 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (352 bytes) Mar 30 15:24:07:749 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:07:749 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (653 bytes, 0 ms) Mar 30 15:24:07:759 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (372 bytes) Mar 30 15:24:07:759 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:07:759 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (352 bytes) Mar 30 15:24:07:765 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:07:765 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (653 bytes, 0 ms) Mar 30 15:24:07:933 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:24:07:957 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:24:08:696 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:24:08:771 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (368 bytes) Mar 30 15:24:08:771 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:08:771 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (348 bytes) Mar 30 15:24:08:776 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:08:776 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (445 bytes, 0 ms) Mar 30 15:24:08:782 [140603169945344] 5 hub: TSESS-P-7-4 [TunnelB] suspended Mar 30 15:24:08:786 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (368 bytes) Mar 30 15:24:08:786 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:08:786 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (348 bytes) Mar 30 15:24:08:792 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:08:792 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (445 bytes, 0 ms) Mar 30 15:24:08:841 [140603723568896] 5 hub: TSESS-P-7 not internal command - thread will handle. mtype=100; sfd:22 Mar 30 15:24:08:841 [140603723568896] 2 hub: CTRL TunnelB request 'hubup' passed to TSESS-P-7 [140603169945344] Mar 30 15:24:08:841 [140603169945344] 3 hub: TSESS-P-7-5 [TunnelB] restarted (for remote) Mar 30 15:24:08:841 [140603169945344] 2 hub: TSESS-P-7 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:34:08 Mar 30 15:24:08:841 [140603169945344] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:24:08:841 [140603169945344] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:24:08:841 [140603169945344] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:24:08:841 [140603169945344] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:24:08:841 [140603169945344] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:24:08:841 [140603169945344] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:24:08:841 [140603169945344] 4 hub: CONNECT: 0x7fe0a8000d20(41) 127.0.0.1/53382->127.0.0.1/48002 Mar 30 15:24:08:841 [140603169945344] 3 hub: TSESS-P-7-5 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:24:08:841 [140603169945344] 1 hub: TSESS-P-7-5 [TunnelB] received 100 'hubup' (672 bytes) Mar 30 15:24:08:842 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53382 (s:378) Mar 30 15:24:08:842 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/53382] Mar 30 15:24:08:842 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:24:08:842 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:24:08:842 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/53382 Mar 30 15:24:08:842 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/53382 same proximity=0 status=0 secver=217 Mar 30 15:24:08:847 [140603169945344] 5 hub: TSESS-P-7-5 [TunnelB] local 200 'OK' Mar 30 15:24:08:847 [140603169945344] 1 hub: TSESS-P-7-5 [TunnelB] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:24:08:857 [140603169945344] 1 hub: TSESS-P-7-5 [TunnelB] ssl 100 '_close' (283 bytes) Mar 30 15:24:08:857 [140603169945344] 3 hub: TSESS-P-7-5 [TunnelB] is paired with [A-299-5] Mar 30 15:24:08:857 [140603169945344] 1 hub: TSESS-P-7-5 [TunnelB] received 100 '_close' (295 bytes) Mar 30 15:24:08:862 [140603169945344] 1 hub: TSESS-P-7-5 [TunnelB] remote session closed Mar 30 15:24:08:862 [140603169945344] 3 hub: TSESS-P-7-5 [TunnelB] exiting msg loop (remote close) Mar 30 15:24:08:862 [140603169945344] 5 hub: TSESS-P-7-5 [TunnelB] suspended Mar 30 15:24:09:801 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (396 bytes) Mar 30 15:24:09:801 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:09:801 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (376 bytes) Mar 30 15:24:09:806 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:09:806 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (633 bytes, 0 ms) Mar 30 15:24:09:816 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (396 bytes) Mar 30 15:24:09:816 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:09:816 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (376 bytes) Mar 30 15:24:09:821 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:09:821 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (633 bytes, 0 ms) Mar 30 15:24:09:892 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:10:828 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (390 bytes) Mar 30 15:24:10:828 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:10:828 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (370 bytes) Mar 30 15:24:10:833 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:10:833 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (527 bytes, 0 ms) Mar 30 15:24:10:843 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (390 bytes) Mar 30 15:24:10:843 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:10:843 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (370 bytes) Mar 30 15:24:10:848 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:10:848 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (527 bytes, 0 ms) Mar 30 15:24:11:252 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:24:11:252 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:24:11:252 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60910 Mar 30 15:24:11:252 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60910' delegated to TSESS-A-5 [140603715176192] Mar 30 15:24:11:252 [140603715176192] 3 hub: TSESS-A-5-22 [TunnelB] restarted (for local) Mar 30 15:24:11:257 [140603715176192] 5 hub: TSESS-A-5-22 [TunnelB] local 100 'get_info' Mar 30 15:24:11:257 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:11:257 [140603715176192] 1 hub: TSESS-A-5-22 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:24:11:268 [140603715176192] 1 hub: TSESS-A-5-22 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:24:11:268 [140603715176192] 3 hub: TSESS-A-5-22 [TunnelB] is paired with [P-298-22] Mar 30 15:24:11:268 [140603715176192] 1 hub: TSESS-A-5-22 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:24:11:268 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60912 Mar 30 15:24:11:268 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60912' delegated to TSESS-A-10 [140603706783488] Mar 30 15:24:11:268 [140603706783488] 3 hub: TSESS-A-10-16 [TunnelB] restarted (for local) Mar 30 15:24:11:273 [140603715176192] 5 hub: TSESS-A-5-22 [TunnelB] local 100 '_close' Mar 30 15:24:11:273 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:11:273 [140603715176192] 1 hub: TSESS-A-5-22 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:24:11:273 [140603715176192] 3 hub: TSESS-A-5-22 [TunnelB] local session closed, suspending Mar 30 15:24:11:273 [140603706783488] 5 hub: TSESS-A-10-16 [TunnelB] local 100 '_status' Mar 30 15:24:11:273 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:11:273 [140603706783488] 1 hub: TSESS-A-10-16 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:24:11:283 [140603706783488] 1 hub: TSESS-A-10-16 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:24:11:283 [140603706783488] 3 hub: TSESS-A-10-16 [TunnelB] is paired with [P-300-16] Mar 30 15:24:11:283 [140603706783488] 1 hub: TSESS-A-10-16 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:24:11:288 [140603706783488] 5 hub: TSESS-A-10-16 [TunnelB] local 100 '_close' Mar 30 15:24:11:288 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:11:288 [140603706783488] 1 hub: TSESS-A-10-16 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:24:11:288 [140603706783488] 3 hub: TSESS-A-10-16 [TunnelB] local session closed, suspending Mar 30 15:24:11:473 [140603715176192] 5 hub: TSESS-A-5-22 [TunnelB] suspended Mar 30 15:24:11:489 [140603706783488] 5 hub: TSESS-A-10-16 [TunnelB] suspended Mar 30 15:24:11:695 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:11:856 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (368 bytes) Mar 30 15:24:11:856 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:11:856 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (348 bytes) Mar 30 15:24:11:861 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:11:861 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (688 bytes, 0 ms) Mar 30 15:24:11:871 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (368 bytes) Mar 30 15:24:11:871 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:11:871 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (348 bytes) Mar 30 15:24:11:876 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:11:876 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (688 bytes, 0 ms) Mar 30 15:24:11:888 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (368 bytes) Mar 30 15:24:11:888 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:11:888 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (348 bytes) Mar 30 15:24:11:893 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:11:893 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (688 bytes, 0 ms) Mar 30 15:24:12:901 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (382 bytes) Mar 30 15:24:12:901 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:12:901 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (362 bytes) Mar 30 15:24:12:906 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:12:906 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (492 bytes, 0 ms) Mar 30 15:24:12:917 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (382 bytes) Mar 30 15:24:12:917 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:12:917 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (362 bytes) Mar 30 15:24:12:922 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:12:922 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (492 bytes, 0 ms) Mar 30 15:24:13:498 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:13:934 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (384 bytes) Mar 30 15:24:13:934 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:13:934 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (364 bytes) Mar 30 15:24:13:939 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:13:939 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (522 bytes, 0 ms) Mar 30 15:24:13:949 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (384 bytes) Mar 30 15:24:13:949 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:13:950 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (364 bytes) Mar 30 15:24:13:955 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:13:955 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (522 bytes, 0 ms) Mar 30 15:24:14:962 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (386 bytes) Mar 30 15:24:14:962 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:14:962 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (366 bytes) Mar 30 15:24:14:968 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:14:968 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (530 bytes, 0 ms) Mar 30 15:24:14:978 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (386 bytes) Mar 30 15:24:14:978 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:14:978 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (366 bytes) Mar 30 15:24:14:983 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:14:983 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (530 bytes, 0 ms) Mar 30 15:24:15:300 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:15:992 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (382 bytes) Mar 30 15:24:15:992 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:15:992 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (362 bytes) Mar 30 15:24:15:997 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:15:997 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (534 bytes, 0 ms) Mar 30 15:24:16:007 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (382 bytes) Mar 30 15:24:16:007 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:16:007 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (362 bytes) Mar 30 15:24:16:012 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:16:012 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (534 bytes, 0 ms) Mar 30 15:24:16:149 [140605283890944] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:24:16:149 [140605283890944] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:24:16:149 [140605283890944] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:24:16:149 [140605283890944] 1 hub: PERF: hubup_check_interval: 0 sec hubhash_size: 18 Mar 30 15:24:16:149 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:149 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:149 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:149 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:149 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:149 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:149 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:24:16:149 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:149 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:149 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:149 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:149 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:149 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:149 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:149 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:149 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:24:16:149 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:24:16:149 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:24:16:150 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(40) 127.0.0.1/47054->127.0.0.1/48007 Mar 30 15:24:16:150 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/47054 Mar 30 15:24:16:150 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/47054' delegated to TSESS-A-6 [140603203516160] Mar 30 15:24:16:150 [140603203516160] 3 hub: TSESS-A-6-5 [Secondary] restarted (for local) Mar 30 15:24:16:155 [140603203516160] 5 hub: TSESS-A-6-5 [Secondary] local 100 'hubup' Mar 30 15:24:16:155 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:16:155 [140603203516160] 1 hub: TSESS-A-6-5 [Secondary] sent 100 'hubup' (686 bytes, 0 ms) Mar 30 15:24:16:165 [140603203516160] 1 hub: TSESS-A-6-5 [Secondary] ssl 200 '-' (453 bytes) Mar 30 15:24:16:165 [140603203516160] 3 hub: TSESS-A-6-5 [Secondary] is paired with [P-2120-5] Mar 30 15:24:16:165 [140603203516160] 1 hub: TSESS-A-6-5 [Secondary] received 200 '-' (461 bytes) Mar 30 15:24:16:165 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:24:16:165 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:24:16:165 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:24:16:165 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:24:16:165 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary same proximity=0 status=0 secver=217 Mar 30 15:24:16:165 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:165 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:165 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:165 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:165 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:165 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:165 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:24:16:165 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:165 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:165 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:165 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:165 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:165 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:165 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:165 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:165 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:24:16:165 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:24:16:165 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:24:16:165 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(40) 127.0.0.1/60918->127.0.0.1/48009 Mar 30 15:24:16:165 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60918 Mar 30 15:24:16:166 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60918' delegated to TSESS-A-5 [140603715176192] Mar 30 15:24:16:166 [140603715176192] 3 hub: TSESS-A-5-23 [TunnelB] restarted (for local) Mar 30 15:24:16:170 [140603203516160] 5 hub: TSESS-A-6-5 [Secondary] local 100 '_close' Mar 30 15:24:16:170 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:16:170 [140603203516160] 1 hub: TSESS-A-6-5 [Secondary] sent 100 '_close' (293 bytes, 0 ms) Mar 30 15:24:16:170 [140603203516160] 3 hub: TSESS-A-6-5 [Secondary] local session closed, suspending Mar 30 15:24:16:171 [140603715176192] 5 hub: TSESS-A-5-23 [TunnelB] local 100 'hubup' Mar 30 15:24:16:171 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:16:171 [140603715176192] 1 hub: TSESS-A-5-23 [TunnelB] sent 100 'hubup' (685 bytes, 0 ms) Mar 30 15:24:16:181 [140603715176192] 1 hub: TSESS-A-5-23 [TunnelB] ssl 200 '-' (443 bytes) Mar 30 15:24:16:181 [140603715176192] 3 hub: TSESS-A-5-23 [TunnelB] is paired with [P-298-23] Mar 30 15:24:16:181 [140603715176192] 1 hub: TSESS-A-5-23 [TunnelB] received 200 '-' (451 bytes) Mar 30 15:24:16:181 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:24:16:181 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:24:16:181 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:24:16:181 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:24:16:181 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB same proximity=0 status=0 secver=217 Mar 30 15:24:16:181 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:181 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:181 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:181 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:181 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:181 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:181 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:24:16:181 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:181 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:181 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:181 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:181 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:181 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:181 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:181 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:16:181 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:24:16:181 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:24:16:181 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:24:16:181 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(40) 127.0.0.1/40106->127.0.0.1/48008 Mar 30 15:24:16:182 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/40106 Mar 30 15:24:16:182 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/40106' delegated to TSESS-A-3 [140603195123456] Mar 30 15:24:16:182 [140603195123456] 3 hub: TSESS-A-3-5 [SecondaryHA] restarted (for local) Mar 30 15:24:16:186 [140603715176192] 5 hub: TSESS-A-5-23 [TunnelB] local 100 '_close' Mar 30 15:24:16:186 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:16:186 [140603715176192] 1 hub: TSESS-A-5-23 [TunnelB] sent 100 '_close' (292 bytes, 0 ms) Mar 30 15:24:16:186 [140603715176192] 3 hub: TSESS-A-5-23 [TunnelB] local session closed, suspending Mar 30 15:24:16:187 [140603195123456] 5 hub: TSESS-A-3-5 [SecondaryHA] local 100 'hubup' Mar 30 15:24:16:187 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:16:187 [140603195123456] 1 hub: TSESS-A-3-5 [SecondaryHA] sent 100 'hubup' (688 bytes, 0 ms) Mar 30 15:24:16:197 [140603195123456] 1 hub: TSESS-A-3-5 [SecondaryHA] ssl 200 '-' (457 bytes) Mar 30 15:24:16:197 [140603195123456] 3 hub: TSESS-A-3-5 [SecondaryHA] is paired with [P-1947-5] Mar 30 15:24:16:197 [140603195123456] 1 hub: TSESS-A-3-5 [SecondaryHA] received 200 '-' (465 bytes) Mar 30 15:24:16:197 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:24:16:197 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:24:16:197 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:24:16:197 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:24:16:197 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA same proximity=0 status=0 secver=217 Mar 30 15:24:16:197 [140605283890944] 5 hub: mark_hubup_list_updated Mar 30 15:24:16:197 [140605283890944] 5 hub: hublist changed: /DOMAIN/Primary 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:16:197 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:16:197 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:16:197 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:16:197 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:16:197 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:16:197 [140605283890944] 5 hub: hublist changed: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:24:16:197 [140605283890944] 5 hub: hublist changed: /DOMAIN/PrimaryHA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:16:197 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:16:197 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:16:197 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:16:197 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:16:197 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:16:197 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:16:197 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:16:197 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:24:16:197 [140605283890944] 5 hub: hublist changed: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:24:16:197 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelBHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:24:16:197 [140605283890944] 5 hub: mark_hubup_list_updated end Mar 30 15:24:16:197 [140605283890944] 1 hub: PERF: hubup_check_for_updates: 0 sec hubhash_size: 18 Mar 30 15:24:16:197 [140605283890944] 1 hub: PERF: hubi_run_command: 0 sec hubhash_size: 18 Mar 30 15:24:16:197 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:24:16:198 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:24:16:198 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:24:16:199 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:24:16:199 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:24:16:199 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:24:16:199 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:24:16:200 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:24:16:200 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:24:16:201 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:24:16:201 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:24:16:201 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:24:16:201 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:24:16:202 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:24:16:203 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:24:16:203 [140603195123456] 5 hub: TSESS-A-3-5 [SecondaryHA] local 100 '_close' Mar 30 15:24:16:203 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:16:203 [140603195123456] 1 hub: TSESS-A-3-5 [SecondaryHA] sent 100 '_close' (295 bytes, 0 ms) Mar 30 15:24:16:203 [140603195123456] 3 hub: TSESS-A-3-5 [SecondaryHA] local session closed, suspending Mar 30 15:24:16:203 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:24:16:204 [140605283890944] 1 hub: PERF: hubi_check_security: 0 sec hubhash_size: 18 Mar 30 15:24:16:371 [140603203516160] 5 hub: TSESS-A-6-5 [Secondary] suspended Mar 30 15:24:16:387 [140603715176192] 5 hub: TSESS-A-5-23 [TunnelB] suspended Mar 30 15:24:16:403 [140603195123456] 5 hub: TSESS-A-3-5 [SecondaryHA] suspended Mar 30 15:24:16:953 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:24:16:985 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:24:17:021 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (385 bytes) Mar 30 15:24:17:021 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:17:021 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (365 bytes) Mar 30 15:24:17:026 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:17:026 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (552 bytes, 0 ms) Mar 30 15:24:17:037 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (385 bytes) Mar 30 15:24:17:037 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:17:037 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (365 bytes) Mar 30 15:24:17:042 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:17:042 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (552 bytes, 0 ms) Mar 30 15:24:17:102 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:17:208 [140603161552640] 5 hub: TSESS-P-4-5 [SecondaryHA] suspended Mar 30 15:24:17:257 [140605547460672] 4 hub: medium timeout Mar 30 15:24:17:257 [140605547460672] 3 hub: next_check [1585596396] now [1585596257] Mar 30 15:24:17:330 [140602666645248] 5 hub: TSESS-P-14-2 [Secondary] suspended Mar 30 15:24:17:933 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:24:17:957 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:24:18:052 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (388 bytes) Mar 30 15:24:18:052 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:18:052 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (368 bytes) Mar 30 15:24:18:057 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:18:057 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (562 bytes, 0 ms) Mar 30 15:24:18:067 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (388 bytes) Mar 30 15:24:18:067 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:18:067 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (368 bytes) Mar 30 15:24:18:073 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:18:073 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (562 bytes, 0 ms) Mar 30 15:24:18:697 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:24:18:862 [140603169945344] 5 hub: TSESS-P-7-5 [TunnelB] suspended Mar 30 15:24:19:080 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (390 bytes) Mar 30 15:24:19:080 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:19:080 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (370 bytes) Mar 30 15:24:19:085 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:19:085 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (574 bytes, 0 ms) Mar 30 15:24:19:095 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (390 bytes) Mar 30 15:24:19:095 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:19:095 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (370 bytes) Mar 30 15:24:19:100 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:19:100 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (574 bytes, 0 ms) Mar 30 15:24:19:805 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:20:107 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (390 bytes) Mar 30 15:24:20:107 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:20:107 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (370 bytes) Mar 30 15:24:20:112 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:20:112 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (572 bytes, 0 ms) Mar 30 15:24:20:122 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (390 bytes) Mar 30 15:24:20:122 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:20:123 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (370 bytes) Mar 30 15:24:20:128 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:20:128 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (572 bytes, 0 ms) Mar 30 15:24:21:135 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (389 bytes) Mar 30 15:24:21:135 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:21:135 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (369 bytes) Mar 30 15:24:21:140 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:21:140 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (580 bytes, 0 ms) Mar 30 15:24:21:151 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (389 bytes) Mar 30 15:24:21:151 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:21:151 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (369 bytes) Mar 30 15:24:21:156 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:21:156 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (580 bytes, 0 ms) Mar 30 15:24:21:295 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:24:21:295 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:24:21:295 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60926 Mar 30 15:24:21:295 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60926' delegated to TSESS-A-5 [140603715176192] Mar 30 15:24:21:295 [140603715176192] 3 hub: TSESS-A-5-24 [TunnelB] restarted (for local) Mar 30 15:24:21:300 [140603715176192] 5 hub: TSESS-A-5-24 [TunnelB] local 100 'get_info' Mar 30 15:24:21:300 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:21:300 [140603715176192] 1 hub: TSESS-A-5-24 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:24:21:311 [140603715176192] 1 hub: TSESS-A-5-24 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:24:21:311 [140603715176192] 3 hub: TSESS-A-5-24 [TunnelB] is paired with [P-298-24] Mar 30 15:24:21:311 [140603715176192] 1 hub: TSESS-A-5-24 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:24:21:311 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60928 Mar 30 15:24:21:311 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60928' delegated to TSESS-A-10 [140603706783488] Mar 30 15:24:21:311 [140603706783488] 3 hub: TSESS-A-10-17 [TunnelB] restarted (for local) Mar 30 15:24:21:316 [140603715176192] 5 hub: TSESS-A-5-24 [TunnelB] local 100 '_close' Mar 30 15:24:21:316 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:21:316 [140603715176192] 1 hub: TSESS-A-5-24 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:24:21:316 [140603715176192] 3 hub: TSESS-A-5-24 [TunnelB] local session closed, suspending Mar 30 15:24:21:316 [140603706783488] 5 hub: TSESS-A-10-17 [TunnelB] local 100 '_status' Mar 30 15:24:21:316 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:21:316 [140603706783488] 1 hub: TSESS-A-10-17 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:24:21:326 [140603706783488] 1 hub: TSESS-A-10-17 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:24:21:326 [140603706783488] 3 hub: TSESS-A-10-17 [TunnelB] is paired with [P-300-17] Mar 30 15:24:21:327 [140603706783488] 1 hub: TSESS-A-10-17 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:24:21:332 [140603706783488] 5 hub: TSESS-A-10-17 [TunnelB] local 100 '_close' Mar 30 15:24:21:332 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:21:332 [140603706783488] 1 hub: TSESS-A-10-17 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:24:21:332 [140603706783488] 3 hub: TSESS-A-10-17 [TunnelB] local session closed, suspending Mar 30 15:24:21:516 [140603715176192] 5 hub: TSESS-A-5-24 [TunnelB] suspended Mar 30 15:24:21:532 [140603706783488] 5 hub: TSESS-A-10-17 [TunnelB] suspended Mar 30 15:24:21:608 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:22:164 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (392 bytes) Mar 30 15:24:22:164 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:22:164 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (372 bytes) Mar 30 15:24:22:169 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:22:169 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (572 bytes, 0 ms) Mar 30 15:24:22:179 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (392 bytes) Mar 30 15:24:22:179 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:22:179 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (372 bytes) Mar 30 15:24:22:184 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:22:185 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (572 bytes, 0 ms) Mar 30 15:24:23:186 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (373 bytes) Mar 30 15:24:23:186 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:23:186 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (353 bytes) Mar 30 15:24:23:191 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:23:191 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (693 bytes, 0 ms) Mar 30 15:24:23:201 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (373 bytes) Mar 30 15:24:23:201 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:23:201 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (353 bytes) Mar 30 15:24:23:207 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:23:207 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (693 bytes, 0 ms) Mar 30 15:24:23:217 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (373 bytes) Mar 30 15:24:23:217 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:23:217 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (353 bytes) Mar 30 15:24:23:222 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:23:222 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (693 bytes, 0 ms) Mar 30 15:24:23:410 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:24:230 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (377 bytes) Mar 30 15:24:24:230 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:24:230 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (357 bytes) Mar 30 15:24:24:235 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:24:235 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (448 bytes, 0 ms) Mar 30 15:24:24:245 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (377 bytes) Mar 30 15:24:24:245 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:24:245 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (357 bytes) Mar 30 15:24:24:250 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:24:250 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (448 bytes, 0 ms) Mar 30 15:24:25:212 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:25:257 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (382 bytes) Mar 30 15:24:25:257 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:25:257 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (362 bytes) Mar 30 15:24:25:262 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:25:262 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (699 bytes, 0 ms) Mar 30 15:24:25:273 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (382 bytes) Mar 30 15:24:25:273 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:25:273 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (362 bytes) Mar 30 15:24:25:278 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:25:278 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (699 bytes, 0 ms) Mar 30 15:24:25:288 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (382 bytes) Mar 30 15:24:25:288 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:25:288 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (362 bytes) Mar 30 15:24:25:293 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:25:293 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (699 bytes, 0 ms) Mar 30 15:24:26:300 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (388 bytes) Mar 30 15:24:26:300 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:26:300 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (368 bytes) Mar 30 15:24:26:305 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:26:306 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (809 bytes, 0 ms) Mar 30 15:24:26:316 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (388 bytes) Mar 30 15:24:26:316 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:26:316 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (368 bytes) Mar 30 15:24:26:321 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:26:321 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (809 bytes, 0 ms) Mar 30 15:24:26:331 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (388 bytes) Mar 30 15:24:26:331 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:26:331 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (368 bytes) Mar 30 15:24:26:336 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:26:336 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (809 bytes, 0 ms) Mar 30 15:24:26:371 [140603203516160] 5 hub: TSESS-A-6-5 [Secondary] suspended Mar 30 15:24:26:403 [140603195123456] 5 hub: TSESS-A-3-5 [SecondaryHA] suspended Mar 30 15:24:26:953 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:24:26:985 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:24:27:015 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:27:209 [140603161552640] 5 hub: TSESS-P-4-5 [SecondaryHA] suspended Mar 30 15:24:27:301 [140605547460672] 4 hub: medium timeout Mar 30 15:24:27:301 [140605547460672] 3 hub: next_check [1585596396] now [1585596267] Mar 30 15:24:27:330 [140602666645248] 5 hub: TSESS-P-14-2 [Secondary] suspended Mar 30 15:24:27:340 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (394 bytes) Mar 30 15:24:27:340 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:27:340 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (374 bytes) Mar 30 15:24:27:345 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:27:345 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (559 bytes, 0 ms) Mar 30 15:24:27:356 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (394 bytes) Mar 30 15:24:27:356 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:27:356 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (374 bytes) Mar 30 15:24:27:361 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:27:361 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (559 bytes, 0 ms) Mar 30 15:24:27:933 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:24:27:957 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:24:28:367 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (387 bytes) Mar 30 15:24:28:367 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:28:367 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (367 bytes) Mar 30 15:24:28:372 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:28:372 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (818 bytes, 0 ms) Mar 30 15:24:28:383 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (387 bytes) Mar 30 15:24:28:383 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:28:383 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (367 bytes) Mar 30 15:24:28:388 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:28:388 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (818 bytes, 0 ms) Mar 30 15:24:28:398 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (387 bytes) Mar 30 15:24:28:398 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:28:398 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (367 bytes) Mar 30 15:24:28:403 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:28:403 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (818 bytes, 0 ms) Mar 30 15:24:28:697 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:24:28:862 [140603169945344] 5 hub: TSESS-P-7-5 [TunnelB] suspended Mar 30 15:24:29:411 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (384 bytes) Mar 30 15:24:29:411 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:29:411 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (364 bytes) Mar 30 15:24:29:416 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:29:416 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (743 bytes, 0 ms) Mar 30 15:24:29:426 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (384 bytes) Mar 30 15:24:29:426 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:29:426 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (364 bytes) Mar 30 15:24:29:431 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:29:431 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (743 bytes, 0 ms) Mar 30 15:24:29:441 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (384 bytes) Mar 30 15:24:29:441 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:29:441 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (364 bytes) Mar 30 15:24:29:447 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:29:447 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (743 bytes, 0 ms) Mar 30 15:24:29:718 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:30:456 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (385 bytes) Mar 30 15:24:30:456 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:30:456 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (365 bytes) Mar 30 15:24:30:461 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:30:461 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (567 bytes, 0 ms) Mar 30 15:24:30:471 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (385 bytes) Mar 30 15:24:30:471 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:30:471 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (365 bytes) Mar 30 15:24:30:476 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:30:476 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (567 bytes, 0 ms) Mar 30 15:24:31:338 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:24:31:338 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:24:31:338 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60934 Mar 30 15:24:31:338 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60934' delegated to TSESS-A-5 [140603715176192] Mar 30 15:24:31:338 [140603715176192] 3 hub: TSESS-A-5-25 [TunnelB] restarted (for local) Mar 30 15:24:31:343 [140603715176192] 5 hub: TSESS-A-5-25 [TunnelB] local 100 'get_info' Mar 30 15:24:31:343 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:31:343 [140603715176192] 1 hub: TSESS-A-5-25 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:24:31:354 [140603715176192] 1 hub: TSESS-A-5-25 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:24:31:354 [140603715176192] 3 hub: TSESS-A-5-25 [TunnelB] is paired with [P-298-25] Mar 30 15:24:31:354 [140603715176192] 1 hub: TSESS-A-5-25 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:24:31:354 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60936 Mar 30 15:24:31:354 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60936' delegated to TSESS-A-10 [140603706783488] Mar 30 15:24:31:354 [140603706783488] 3 hub: TSESS-A-10-18 [TunnelB] restarted (for local) Mar 30 15:24:31:359 [140603715176192] 5 hub: TSESS-A-5-25 [TunnelB] local 100 '_close' Mar 30 15:24:31:359 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:31:359 [140603715176192] 1 hub: TSESS-A-5-25 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:24:31:359 [140603715176192] 3 hub: TSESS-A-5-25 [TunnelB] local session closed, suspending Mar 30 15:24:31:359 [140603706783488] 5 hub: TSESS-A-10-18 [TunnelB] local 100 '_status' Mar 30 15:24:31:359 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:31:359 [140603706783488] 1 hub: TSESS-A-10-18 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:24:31:369 [140603706783488] 1 hub: TSESS-A-10-18 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:24:31:369 [140603706783488] 3 hub: TSESS-A-10-18 [TunnelB] is paired with [P-300-18] Mar 30 15:24:31:370 [140603706783488] 1 hub: TSESS-A-10-18 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:24:31:375 [140603706783488] 5 hub: TSESS-A-10-18 [TunnelB] local 100 '_close' Mar 30 15:24:31:375 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:31:375 [140603706783488] 1 hub: TSESS-A-10-18 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:24:31:375 [140603706783488] 3 hub: TSESS-A-10-18 [TunnelB] local session closed, suspending Mar 30 15:24:31:483 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (368 bytes) Mar 30 15:24:31:483 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:31:483 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (348 bytes) Mar 30 15:24:31:488 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:31:488 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (445 bytes, 0 ms) Mar 30 15:24:31:498 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (368 bytes) Mar 30 15:24:31:498 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:31:498 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (348 bytes) Mar 30 15:24:31:504 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:31:504 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (445 bytes, 0 ms) Mar 30 15:24:31:521 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:31:559 [140603715176192] 5 hub: TSESS-A-5-25 [TunnelB] suspended Mar 30 15:24:31:575 [140603706783488] 5 hub: TSESS-A-10-18 [TunnelB] suspended Mar 30 15:24:32:511 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (377 bytes) Mar 30 15:24:32:511 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:32:511 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (357 bytes) Mar 30 15:24:32:516 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:32:516 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (503 bytes, 0 ms) Mar 30 15:24:32:527 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (377 bytes) Mar 30 15:24:32:527 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:32:527 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (357 bytes) Mar 30 15:24:32:532 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:32:532 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (503 bytes, 0 ms) Mar 30 15:24:33:323 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:33:539 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (391 bytes) Mar 30 15:24:33:539 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:33:539 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (371 bytes) Mar 30 15:24:33:544 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:33:544 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (603 bytes, 0 ms) Mar 30 15:24:33:554 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (391 bytes) Mar 30 15:24:33:554 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:33:554 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (371 bytes) Mar 30 15:24:33:560 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:33:560 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (603 bytes, 0 ms) Mar 30 15:24:34:566 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (386 bytes) Mar 30 15:24:34:566 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:34:567 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (366 bytes) Mar 30 15:24:34:572 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:34:572 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (573 bytes, 0 ms) Mar 30 15:24:34:582 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (386 bytes) Mar 30 15:24:34:582 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:34:582 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (366 bytes) Mar 30 15:24:34:587 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:34:587 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (573 bytes, 0 ms) Mar 30 15:24:34:861 [140604042360576] 5 hub: There are 0 objects waiting on 8 threads Mar 30 15:24:35:126 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:35:342 [140605547460672] 3 hub: check_session - session 0x20f9050 (127.0.0.1/48002) is a server Mar 30 15:24:35:342 [140605547460672] 2 hub: Profile - clayk 38 bytes (OK) Mar 30 15:24:35:596 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (368 bytes) Mar 30 15:24:35:596 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:35:596 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (348 bytes) Mar 30 15:24:35:601 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:35:601 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (427 bytes, 0 ms) Mar 30 15:24:35:611 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (368 bytes) Mar 30 15:24:35:611 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:35:611 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (348 bytes) Mar 30 15:24:35:616 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:35:616 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (427 bytes, 0 ms) Mar 30 15:24:36:371 [140603203516160] 5 hub: TSESS-A-6-5 [Secondary] suspended Mar 30 15:24:36:404 [140603195123456] 5 hub: TSESS-A-3-5 [SecondaryHA] suspended Mar 30 15:24:36:624 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (377 bytes) Mar 30 15:24:36:624 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:36:624 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (357 bytes) Mar 30 15:24:36:629 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:36:629 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (279 bytes, 0 ms) Mar 30 15:24:36:953 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:24:36:986 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:24:37:209 [140603161552640] 5 hub: TSESS-P-4-5 [SecondaryHA] suspended Mar 30 15:24:37:330 [140602666645248] 5 hub: TSESS-P-14-2 [Secondary] suspended Mar 30 15:24:37:340 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:24:37:340 [140603731961600] 2 hub: CTRL SecondaryHA request 'hubup' passed to TSESS-P-4 [140603161552640] Mar 30 15:24:37:340 [140603161552640] 3 hub: TSESS-P-4-6 [SecondaryHA] restarted (for remote) Mar 30 15:24:37:340 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:34:37 Mar 30 15:24:37:340 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:24:37:340 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:24:37:340 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:24:37:340 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:24:37:340 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:24:37:340 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:24:37:340 [140603161552640] 4 hub: CONNECT: 0x7fe09c000c60(41) 127.0.0.1/53418->127.0.0.1/48002 Mar 30 15:24:37:340 [140603161552640] 3 hub: TSESS-P-4-6 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:24:37:340 [140605547460672] 4 hub: medium timeout Mar 30 15:24:37:340 [140605547460672] 3 hub: next_check [1585596396] now [1585596277] Mar 30 15:24:37:340 [140603161552640] 1 hub: TSESS-P-4-6 [SecondaryHA] received 100 'hubup' (687 bytes) Mar 30 15:24:37:341 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53418 (s:392) Mar 30 15:24:37:341 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/53418] Mar 30 15:24:37:341 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:24:37:341 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:24:37:341 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/53418 Mar 30 15:24:37:341 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/53418 same proximity=0 status=0 secver=217 Mar 30 15:24:37:346 [140603161552640] 5 hub: TSESS-P-4-6 [SecondaryHA] local 200 'OK' Mar 30 15:24:37:346 [140603161552640] 1 hub: TSESS-P-4-6 [SecondaryHA] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:24:37:356 [140603161552640] 1 hub: TSESS-P-4-6 [SecondaryHA] ssl 100 '_close' (300 bytes) Mar 30 15:24:37:356 [140603161552640] 3 hub: TSESS-P-4-6 [SecondaryHA] is paired with [A-1948-6] Mar 30 15:24:37:356 [140603161552640] 1 hub: TSESS-P-4-6 [SecondaryHA] received 100 '_close' (296 bytes) Mar 30 15:24:37:361 [140603161552640] 1 hub: TSESS-P-4-6 [SecondaryHA] remote session closed Mar 30 15:24:37:361 [140603161552640] 3 hub: TSESS-P-4-6 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:24:37:361 [140603161552640] 5 hub: TSESS-P-4-6 [SecondaryHA] suspended Mar 30 15:24:37:456 [140603740354304] 5 hub: TSESS-P-14 not internal command - thread will handle. mtype=100; sfd:29 Mar 30 15:24:37:456 [140603740354304] 2 hub: CTRL Secondary request 'hubup' passed to TSESS-P-14 [140602666645248] Mar 30 15:24:37:456 [140602666645248] 3 hub: TSESS-P-14-3 [Secondary] restarted (for remote) Mar 30 15:24:37:456 [140602666645248] 2 hub: TSESS-P-14 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:34:37 Mar 30 15:24:37:456 [140602666645248] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:24:37:456 [140602666645248] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:24:37:456 [140602666645248] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:24:37:456 [140602666645248] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:24:37:456 [140602666645248] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:24:37:456 [140602666645248] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:24:37:456 [140602666645248] 4 hub: CONNECT: 0x7fe094000c00(40) 127.0.0.1/53420->127.0.0.1/48002 Mar 30 15:24:37:456 [140602666645248] 3 hub: TSESS-P-14-3 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:24:37:456 [140602666645248] 1 hub: TSESS-P-14-3 [Secondary] received 100 'hubup' (683 bytes) Mar 30 15:24:37:456 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53420 (s:388) Mar 30 15:24:37:456 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/53420] Mar 30 15:24:37:456 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:24:37:457 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:24:37:457 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/53420 Mar 30 15:24:37:457 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/53420 same proximity=0 status=0 secver=217 Mar 30 15:24:37:462 [140602666645248] 5 hub: TSESS-P-14-3 [Secondary] local 200 'OK' Mar 30 15:24:37:462 [140602666645248] 1 hub: TSESS-P-14-3 [Secondary] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:24:37:473 [140602666645248] 1 hub: TSESS-P-14-3 [Secondary] ssl 100 '_close' (296 bytes) Mar 30 15:24:37:473 [140602666645248] 3 hub: TSESS-P-14-3 [Secondary] is paired with [A-2123-3] Mar 30 15:24:37:473 [140602666645248] 1 hub: TSESS-P-14-3 [Secondary] received 100 '_close' (296 bytes) Mar 30 15:24:37:478 [140602666645248] 1 hub: TSESS-P-14-3 [Secondary] remote session closed Mar 30 15:24:37:478 [140602666645248] 3 hub: TSESS-P-14-3 [Secondary] exiting msg loop (remote close) Mar 30 15:24:37:478 [140602666645248] 5 hub: TSESS-P-14-3 [Secondary] suspended Mar 30 15:24:37:630 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (383 bytes) Mar 30 15:24:37:630 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:37:630 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (363 bytes) Mar 30 15:24:37:635 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:37:635 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (489 bytes, 0 ms) Mar 30 15:24:37:645 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (383 bytes) Mar 30 15:24:37:645 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:37:645 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (363 bytes) Mar 30 15:24:37:651 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:37:651 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (489 bytes, 0 ms) Mar 30 15:24:37:829 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:37:933 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:24:37:957 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:24:38:659 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (367 bytes) Mar 30 15:24:38:659 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:38:659 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (347 bytes) Mar 30 15:24:38:664 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:38:665 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (446 bytes, 0 ms) Mar 30 15:24:38:675 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (367 bytes) Mar 30 15:24:38:675 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:38:675 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (347 bytes) Mar 30 15:24:38:680 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:38:680 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (446 bytes, 0 ms) Mar 30 15:24:38:697 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:24:38:862 [140603169945344] 5 hub: TSESS-P-7-5 [TunnelB] suspended Mar 30 15:24:38:921 [140603723568896] 5 hub: TSESS-P-7 not internal command - thread will handle. mtype=100; sfd:22 Mar 30 15:24:38:921 [140603723568896] 2 hub: CTRL TunnelB request 'hubup' passed to TSESS-P-7 [140603169945344] Mar 30 15:24:38:921 [140603169945344] 3 hub: TSESS-P-7-6 [TunnelB] restarted (for remote) Mar 30 15:24:38:921 [140603169945344] 2 hub: TSESS-P-7 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:34:38 Mar 30 15:24:38:921 [140603169945344] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:24:38:921 [140603169945344] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:24:38:921 [140603169945344] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:24:38:921 [140603169945344] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:24:38:921 [140603169945344] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:24:38:921 [140603169945344] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:24:38:921 [140603169945344] 4 hub: CONNECT: 0x7fe0a8000d20(41) 127.0.0.1/53424->127.0.0.1/48002 Mar 30 15:24:38:921 [140603169945344] 3 hub: TSESS-P-7-6 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:24:38:921 [140603169945344] 1 hub: TSESS-P-7-6 [TunnelB] received 100 'hubup' (672 bytes) Mar 30 15:24:38:921 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53424 (s:378) Mar 30 15:24:38:921 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/53424] Mar 30 15:24:38:921 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:24:38:921 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:24:38:921 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/53424 Mar 30 15:24:38:921 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/53424 same proximity=0 status=0 secver=217 Mar 30 15:24:38:927 [140603169945344] 5 hub: TSESS-P-7-6 [TunnelB] local 200 'OK' Mar 30 15:24:38:927 [140603169945344] 1 hub: TSESS-P-7-6 [TunnelB] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:24:38:937 [140603169945344] 1 hub: TSESS-P-7-6 [TunnelB] ssl 100 '_close' (283 bytes) Mar 30 15:24:38:937 [140603169945344] 3 hub: TSESS-P-7-6 [TunnelB] is paired with [A-299-6] Mar 30 15:24:38:937 [140603169945344] 1 hub: TSESS-P-7-6 [TunnelB] received 100 '_close' (295 bytes) Mar 30 15:24:38:942 [140603169945344] 1 hub: TSESS-P-7-6 [TunnelB] remote session closed Mar 30 15:24:38:942 [140603169945344] 3 hub: TSESS-P-7-6 [TunnelB] exiting msg loop (remote close) Mar 30 15:24:38:942 [140603169945344] 5 hub: TSESS-P-7-6 [TunnelB] suspended Mar 30 15:24:39:632 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:39:686 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (379 bytes) Mar 30 15:24:39:686 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:39:686 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (359 bytes) Mar 30 15:24:39:691 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:39:692 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (455 bytes, 0 ms) Mar 30 15:24:39:702 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (379 bytes) Mar 30 15:24:39:702 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:39:702 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (359 bytes) Mar 30 15:24:39:707 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:39:707 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (455 bytes, 0 ms) Mar 30 15:24:40:714 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (387 bytes) Mar 30 15:24:40:714 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:40:714 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (367 bytes) Mar 30 15:24:40:719 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:40:719 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (555 bytes, 0 ms) Mar 30 15:24:40:730 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (387 bytes) Mar 30 15:24:40:730 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:40:730 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (367 bytes) Mar 30 15:24:40:735 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:40:735 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (555 bytes, 0 ms) Mar 30 15:24:41:381 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:24:41:381 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:24:41:381 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60952 Mar 30 15:24:41:381 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60952' delegated to TSESS-A-5 [140603715176192] Mar 30 15:24:41:381 [140603715176192] 3 hub: TSESS-A-5-26 [TunnelB] restarted (for local) Mar 30 15:24:41:386 [140603715176192] 5 hub: TSESS-A-5-26 [TunnelB] local 100 'get_info' Mar 30 15:24:41:386 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:41:387 [140603715176192] 1 hub: TSESS-A-5-26 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:24:41:397 [140603715176192] 1 hub: TSESS-A-5-26 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:24:41:397 [140603715176192] 3 hub: TSESS-A-5-26 [TunnelB] is paired with [P-298-26] Mar 30 15:24:41:397 [140603715176192] 1 hub: TSESS-A-5-26 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:24:41:397 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60954 Mar 30 15:24:41:397 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60954' delegated to TSESS-A-10 [140603706783488] Mar 30 15:24:41:397 [140603706783488] 3 hub: TSESS-A-10-19 [TunnelB] restarted (for local) Mar 30 15:24:41:402 [140603715176192] 5 hub: TSESS-A-5-26 [TunnelB] local 100 '_close' Mar 30 15:24:41:402 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:41:402 [140603715176192] 1 hub: TSESS-A-5-26 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:24:41:402 [140603715176192] 3 hub: TSESS-A-5-26 [TunnelB] local session closed, suspending Mar 30 15:24:41:402 [140603706783488] 5 hub: TSESS-A-10-19 [TunnelB] local 100 '_status' Mar 30 15:24:41:402 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:41:402 [140603706783488] 1 hub: TSESS-A-10-19 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:24:41:412 [140603706783488] 1 hub: TSESS-A-10-19 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:24:41:412 [140603706783488] 3 hub: TSESS-A-10-19 [TunnelB] is paired with [P-300-19] Mar 30 15:24:41:413 [140603706783488] 1 hub: TSESS-A-10-19 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:24:41:418 [140603706783488] 5 hub: TSESS-A-10-19 [TunnelB] local 100 '_close' Mar 30 15:24:41:418 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:41:418 [140603706783488] 1 hub: TSESS-A-10-19 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:24:41:418 [140603706783488] 3 hub: TSESS-A-10-19 [TunnelB] local session closed, suspending Mar 30 15:24:41:434 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:41:602 [140603715176192] 5 hub: TSESS-A-5-26 [TunnelB] suspended Mar 30 15:24:41:618 [140603706783488] 5 hub: TSESS-A-10-19 [TunnelB] suspended Mar 30 15:24:41:745 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (385 bytes) Mar 30 15:24:41:745 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:41:745 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (365 bytes) Mar 30 15:24:41:750 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:41:750 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (573 bytes, 0 ms) Mar 30 15:24:41:760 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (385 bytes) Mar 30 15:24:41:760 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:41:760 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (365 bytes) Mar 30 15:24:41:766 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:41:766 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (573 bytes, 0 ms) Mar 30 15:24:42:772 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (385 bytes) Mar 30 15:24:42:772 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:42:772 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (365 bytes) Mar 30 15:24:42:778 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:42:778 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (573 bytes, 0 ms) Mar 30 15:24:42:788 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (385 bytes) Mar 30 15:24:42:788 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:42:788 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (365 bytes) Mar 30 15:24:42:793 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:42:793 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (573 bytes, 0 ms) Mar 30 15:24:43:237 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:43:795 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (367 bytes) Mar 30 15:24:43:796 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:43:796 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (347 bytes) Mar 30 15:24:43:801 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:43:801 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (260 bytes, 0 ms) Mar 30 15:24:44:809 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (372 bytes) Mar 30 15:24:44:809 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:44:809 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (352 bytes) Mar 30 15:24:44:814 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:44:814 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (465 bytes, 0 ms) Mar 30 15:24:44:824 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (372 bytes) Mar 30 15:24:44:824 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:44:824 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (352 bytes) Mar 30 15:24:44:829 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:44:829 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (465 bytes, 0 ms) Mar 30 15:24:45:039 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:45:836 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (375 bytes) Mar 30 15:24:45:836 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:45:836 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (355 bytes) Mar 30 15:24:45:841 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:45:842 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (442 bytes, 0 ms) Mar 30 15:24:45:852 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (375 bytes) Mar 30 15:24:45:852 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:45:852 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (355 bytes) Mar 30 15:24:45:857 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:45:857 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (442 bytes, 0 ms) Mar 30 15:24:46:234 [140605283890944] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:24:46:234 [140605283890944] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:24:46:234 [140605283890944] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:24:46:234 [140605283890944] 1 hub: PERF: hubup_check_interval: 0 sec hubhash_size: 18 Mar 30 15:24:46:234 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:234 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:234 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:234 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:234 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:234 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:234 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:24:46:235 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:235 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:235 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:235 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:235 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:235 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:235 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:235 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:235 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:24:46:235 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:24:46:235 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:24:46:235 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(40) 127.0.0.1/47096->127.0.0.1/48007 Mar 30 15:24:46:235 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/47096 Mar 30 15:24:46:235 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/47096' delegated to TSESS-A-6 [140603203516160] Mar 30 15:24:46:235 [140603203516160] 3 hub: TSESS-A-6-6 [Secondary] restarted (for local) Mar 30 15:24:46:240 [140603203516160] 5 hub: TSESS-A-6-6 [Secondary] local 100 'hubup' Mar 30 15:24:46:240 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:46:240 [140603203516160] 1 hub: TSESS-A-6-6 [Secondary] sent 100 'hubup' (686 bytes, 0 ms) Mar 30 15:24:46:250 [140603203516160] 1 hub: TSESS-A-6-6 [Secondary] ssl 200 '-' (453 bytes) Mar 30 15:24:46:250 [140603203516160] 3 hub: TSESS-A-6-6 [Secondary] is paired with [P-2120-6] Mar 30 15:24:46:250 [140603203516160] 1 hub: TSESS-A-6-6 [Secondary] received 200 '-' (461 bytes) Mar 30 15:24:46:250 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:24:46:250 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:24:46:250 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:24:46:250 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:24:46:251 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary same proximity=0 status=0 secver=217 Mar 30 15:24:46:251 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:251 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:251 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:251 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:251 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:251 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:251 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:24:46:251 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:251 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:251 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:251 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:251 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:251 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:251 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:251 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:251 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:24:46:251 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:24:46:251 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:24:46:251 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(40) 127.0.0.1/60960->127.0.0.1/48009 Mar 30 15:24:46:251 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60960 Mar 30 15:24:46:251 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60960' delegated to TSESS-A-5 [140603715176192] Mar 30 15:24:46:251 [140603715176192] 3 hub: TSESS-A-5-27 [TunnelB] restarted (for local) Mar 30 15:24:46:256 [140603203516160] 5 hub: TSESS-A-6-6 [Secondary] local 100 '_close' Mar 30 15:24:46:256 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:46:256 [140603203516160] 1 hub: TSESS-A-6-6 [Secondary] sent 100 '_close' (293 bytes, 0 ms) Mar 30 15:24:46:256 [140603203516160] 3 hub: TSESS-A-6-6 [Secondary] local session closed, suspending Mar 30 15:24:46:257 [140603715176192] 5 hub: TSESS-A-5-27 [TunnelB] local 100 'hubup' Mar 30 15:24:46:257 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:46:257 [140603715176192] 1 hub: TSESS-A-5-27 [TunnelB] sent 100 'hubup' (685 bytes, 0 ms) Mar 30 15:24:46:267 [140603715176192] 1 hub: TSESS-A-5-27 [TunnelB] ssl 200 '-' (443 bytes) Mar 30 15:24:46:267 [140603715176192] 3 hub: TSESS-A-5-27 [TunnelB] is paired with [P-298-27] Mar 30 15:24:46:267 [140603715176192] 1 hub: TSESS-A-5-27 [TunnelB] received 200 '-' (451 bytes) Mar 30 15:24:46:267 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:24:46:267 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:24:46:267 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:24:46:267 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:24:46:267 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB same proximity=0 status=0 secver=217 Mar 30 15:24:46:267 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:267 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:267 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:267 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:267 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:267 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:267 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:24:46:267 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:267 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:267 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:267 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:267 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:267 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:267 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:267 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:24:46:267 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:24:46:267 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:24:46:267 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:24:46:267 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(40) 127.0.0.1/40148->127.0.0.1/48008 Mar 30 15:24:46:267 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/40148 Mar 30 15:24:46:267 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/40148' delegated to TSESS-A-3 [140603195123456] Mar 30 15:24:46:267 [140603195123456] 3 hub: TSESS-A-3-6 [SecondaryHA] restarted (for local) Mar 30 15:24:46:272 [140603715176192] 5 hub: TSESS-A-5-27 [TunnelB] local 100 '_close' Mar 30 15:24:46:272 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:46:272 [140603715176192] 1 hub: TSESS-A-5-27 [TunnelB] sent 100 '_close' (292 bytes, 0 ms) Mar 30 15:24:46:272 [140603715176192] 3 hub: TSESS-A-5-27 [TunnelB] local session closed, suspending Mar 30 15:24:46:273 [140603195123456] 5 hub: TSESS-A-3-6 [SecondaryHA] local 100 'hubup' Mar 30 15:24:46:273 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:46:273 [140603195123456] 1 hub: TSESS-A-3-6 [SecondaryHA] sent 100 'hubup' (688 bytes, 0 ms) Mar 30 15:24:46:284 [140603195123456] 1 hub: TSESS-A-3-6 [SecondaryHA] ssl 200 '-' (457 bytes) Mar 30 15:24:46:284 [140603195123456] 3 hub: TSESS-A-3-6 [SecondaryHA] is paired with [P-1947-6] Mar 30 15:24:46:284 [140603195123456] 1 hub: TSESS-A-3-6 [SecondaryHA] received 200 '-' (465 bytes) Mar 30 15:24:46:284 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:24:46:284 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:24:46:284 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:24:46:284 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:24:46:284 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA same proximity=0 status=0 secver=217 Mar 30 15:24:46:284 [140605283890944] 5 hub: mark_hubup_list_updated Mar 30 15:24:46:284 [140605283890944] 5 hub: hublist changed: /DOMAIN/Primary 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:46:284 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:46:284 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:46:284 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:46:284 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:46:284 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:46:284 [140605283890944] 5 hub: hublist changed: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:24:46:284 [140605283890944] 5 hub: hublist changed: /DOMAIN/PrimaryHA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:46:284 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:46:284 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:46:284 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:46:284 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:46:284 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:46:284 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:46:284 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:24:46:284 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:24:46:284 [140605283890944] 5 hub: hublist changed: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:24:46:284 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelBHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:24:46:284 [140605283890944] 5 hub: mark_hubup_list_updated end Mar 30 15:24:46:284 [140605283890944] 1 hub: PERF: hubup_check_for_updates: 0 sec hubhash_size: 18 Mar 30 15:24:46:284 [140605283890944] 1 hub: PERF: hubi_run_command: 0 sec hubhash_size: 18 Mar 30 15:24:46:284 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:24:46:285 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:24:46:285 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:24:46:286 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:24:46:286 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:24:46:286 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:24:46:286 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:24:46:287 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:24:46:287 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:24:46:288 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:24:46:288 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:24:46:288 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:24:46:288 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:24:46:289 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:24:46:290 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:24:46:290 [140603195123456] 5 hub: TSESS-A-3-6 [SecondaryHA] local 100 '_close' Mar 30 15:24:46:290 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:46:290 [140603195123456] 1 hub: TSESS-A-3-6 [SecondaryHA] sent 100 '_close' (295 bytes, 0 ms) Mar 30 15:24:46:290 [140603195123456] 3 hub: TSESS-A-3-6 [SecondaryHA] local session closed, suspending Mar 30 15:24:46:290 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:24:46:291 [140605283890944] 1 hub: PERF: hubi_check_security: 0 sec hubhash_size: 18 Mar 30 15:24:46:456 [140603203516160] 5 hub: TSESS-A-6-6 [Secondary] suspended Mar 30 15:24:46:472 [140603715176192] 5 hub: TSESS-A-5-27 [TunnelB] suspended Mar 30 15:24:46:490 [140603195123456] 5 hub: TSESS-A-3-6 [SecondaryHA] suspended Mar 30 15:24:46:864 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (375 bytes) Mar 30 15:24:46:864 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:46:864 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (355 bytes) Mar 30 15:24:46:869 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:46:869 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (464 bytes, 0 ms) Mar 30 15:24:46:880 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (375 bytes) Mar 30 15:24:46:880 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:46:880 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (355 bytes) Mar 30 15:24:46:885 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:46:885 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (464 bytes, 0 ms) Mar 30 15:24:46:953 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:24:46:979 [140605547460672] 2 hub: verify_login from ctrl 127.0.0.1/53458 Mar 30 15:24:46:979 [140605547460672] 1 hub: verify - [target] prid=controller cmd=probe_list ip=10.x.x.x perm=1 Mar 30 15:24:46:979 [140605547460672] 1 hub: verify - [source] id=ppm ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:46:986 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:24:47:361 [140603161552640] 5 hub: TSESS-P-4-6 [SecondaryHA] suspended Mar 30 15:24:47:479 [140602666645248] 5 hub: TSESS-P-14-3 [Secondary] suspended Mar 30 15:24:47:743 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:47:892 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (375 bytes) Mar 30 15:24:47:892 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:47:892 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (355 bytes) Mar 30 15:24:47:897 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:47:897 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (460 bytes, 0 ms) Mar 30 15:24:47:908 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (375 bytes) Mar 30 15:24:47:908 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:47:908 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (355 bytes) Mar 30 15:24:47:913 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:47:913 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (460 bytes, 0 ms) Mar 30 15:24:47:933 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:24:47:957 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:24:47:981 [140605547460672] 4 hub: medium timeout Mar 30 15:24:47:981 [140605547460672] 3 hub: next_check [1585596396] now [1585596287] Mar 30 15:24:48:697 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:24:48:921 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (376 bytes) Mar 30 15:24:48:921 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:48:921 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (356 bytes) Mar 30 15:24:48:926 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:48:926 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (450 bytes, 0 ms) Mar 30 15:24:48:936 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (376 bytes) Mar 30 15:24:48:936 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:48:937 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (356 bytes) Mar 30 15:24:48:942 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:48:942 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (450 bytes, 0 ms) Mar 30 15:24:48:942 [140603169945344] 5 hub: TSESS-P-7-6 [TunnelB] suspended Mar 30 15:24:49:544 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:49:946 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (375 bytes) Mar 30 15:24:49:946 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:49:946 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (355 bytes) Mar 30 15:24:49:951 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:49:952 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (522 bytes, 0 ms) Mar 30 15:24:49:965 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (375 bytes) Mar 30 15:24:49:965 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:49:965 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (355 bytes) Mar 30 15:24:49:970 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:49:970 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (522 bytes, 0 ms) Mar 30 15:24:50:977 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (375 bytes) Mar 30 15:24:50:977 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:50:977 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (355 bytes) Mar 30 15:24:50:982 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:50:982 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (452 bytes, 0 ms) Mar 30 15:24:50:993 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (375 bytes) Mar 30 15:24:50:993 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:50:993 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (355 bytes) Mar 30 15:24:50:998 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:50:998 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (452 bytes, 0 ms) Mar 30 15:24:51:347 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:51:424 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:24:51:424 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:24:51:424 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60986 Mar 30 15:24:51:424 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60986' delegated to TSESS-A-5 [140603715176192] Mar 30 15:24:51:424 [140603715176192] 3 hub: TSESS-A-5-28 [TunnelB] restarted (for local) Mar 30 15:24:51:429 [140603715176192] 5 hub: TSESS-A-5-28 [TunnelB] local 100 'get_info' Mar 30 15:24:51:429 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:51:430 [140603715176192] 1 hub: TSESS-A-5-28 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:24:51:440 [140603715176192] 1 hub: TSESS-A-5-28 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:24:51:440 [140603715176192] 3 hub: TSESS-A-5-28 [TunnelB] is paired with [P-298-28] Mar 30 15:24:51:440 [140603715176192] 1 hub: TSESS-A-5-28 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:24:51:440 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60988 Mar 30 15:24:51:440 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60988' delegated to TSESS-A-10 [140603706783488] Mar 30 15:24:51:440 [140603706783488] 3 hub: TSESS-A-10-20 [TunnelB] restarted (for local) Mar 30 15:24:51:445 [140603715176192] 5 hub: TSESS-A-5-28 [TunnelB] local 100 '_close' Mar 30 15:24:51:445 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:51:445 [140603715176192] 1 hub: TSESS-A-5-28 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:24:51:445 [140603715176192] 3 hub: TSESS-A-5-28 [TunnelB] local session closed, suspending Mar 30 15:24:51:445 [140603706783488] 5 hub: TSESS-A-10-20 [TunnelB] local 100 '_status' Mar 30 15:24:51:445 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:51:445 [140603706783488] 1 hub: TSESS-A-10-20 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:24:51:456 [140603706783488] 1 hub: TSESS-A-10-20 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:24:51:456 [140603706783488] 3 hub: TSESS-A-10-20 [TunnelB] is paired with [P-300-20] Mar 30 15:24:51:456 [140603706783488] 1 hub: TSESS-A-10-20 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:24:51:461 [140603706783488] 5 hub: TSESS-A-10-20 [TunnelB] local 100 '_close' Mar 30 15:24:51:461 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:24:51:461 [140603706783488] 1 hub: TSESS-A-10-20 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:24:51:461 [140603706783488] 3 hub: TSESS-A-10-20 [TunnelB] local session closed, suspending Mar 30 15:24:51:645 [140603715176192] 5 hub: TSESS-A-5-28 [TunnelB] suspended Mar 30 15:24:51:661 [140603706783488] 5 hub: TSESS-A-10-20 [TunnelB] suspended Mar 30 15:24:52:006 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (378 bytes) Mar 30 15:24:52:007 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:52:007 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (358 bytes) Mar 30 15:24:52:012 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:52:012 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (982 bytes, 0 ms) Mar 30 15:24:52:022 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (378 bytes) Mar 30 15:24:52:022 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:52:022 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (358 bytes) Mar 30 15:24:52:027 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:52:027 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (982 bytes, 0 ms) Mar 30 15:24:52:037 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (378 bytes) Mar 30 15:24:52:037 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:52:037 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (358 bytes) Mar 30 15:24:52:043 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:52:043 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (982 bytes, 0 ms) Mar 30 15:24:52:053 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (378 bytes) Mar 30 15:24:52:053 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:52:053 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (358 bytes) Mar 30 15:24:52:058 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:52:058 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (982 bytes, 0 ms) Mar 30 15:24:53:061 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (368 bytes) Mar 30 15:24:53:061 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:53:061 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (348 bytes) Mar 30 15:24:53:066 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:53:066 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (423 bytes, 0 ms) Mar 30 15:24:53:076 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (368 bytes) Mar 30 15:24:53:076 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:53:077 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (348 bytes) Mar 30 15:24:53:082 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:53:082 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (423 bytes, 0 ms) Mar 30 15:24:53:149 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:54:089 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (383 bytes) Mar 30 15:24:54:089 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:54:089 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (363 bytes) Mar 30 15:24:54:095 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:54:095 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (555 bytes, 0 ms) Mar 30 15:24:54:105 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (383 bytes) Mar 30 15:24:54:105 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:54:105 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (363 bytes) Mar 30 15:24:54:110 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:54:110 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (555 bytes, 0 ms) Mar 30 15:24:55:117 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (390 bytes) Mar 30 15:24:55:117 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:55:117 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (370 bytes) Mar 30 15:24:55:122 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:55:123 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (597 bytes, 0 ms) Mar 30 15:24:55:133 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (390 bytes) Mar 30 15:24:55:133 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:55:133 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (370 bytes) Mar 30 15:24:55:138 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:55:138 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (597 bytes, 0 ms) Mar 30 15:24:55:853 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:56:144 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (376 bytes) Mar 30 15:24:56:144 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:56:144 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (356 bytes) Mar 30 15:24:56:149 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:56:149 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (485 bytes, 0 ms) Mar 30 15:24:56:160 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (376 bytes) Mar 30 15:24:56:160 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:56:160 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (356 bytes) Mar 30 15:24:56:165 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:56:165 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (485 bytes, 0 ms) Mar 30 15:24:56:456 [140603203516160] 5 hub: TSESS-A-6-6 [Secondary] suspended Mar 30 15:24:56:491 [140603195123456] 5 hub: TSESS-A-3-6 [SecondaryHA] suspended Mar 30 15:24:56:953 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:24:56:986 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:24:57:173 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (389 bytes) Mar 30 15:24:57:173 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:57:173 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (369 bytes) Mar 30 15:24:57:179 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:57:179 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (591 bytes, 0 ms) Mar 30 15:24:57:189 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (389 bytes) Mar 30 15:24:57:189 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:57:189 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (369 bytes) Mar 30 15:24:57:194 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:57:195 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (591 bytes, 0 ms) Mar 30 15:24:57:361 [140603161552640] 5 hub: TSESS-P-4-6 [SecondaryHA] suspended Mar 30 15:24:57:430 [140605547460672] 4 hub: medium timeout Mar 30 15:24:57:430 [140605547460672] 3 hub: next_check [1585596396] now [1585596297] Mar 30 15:24:57:479 [140602666645248] 5 hub: TSESS-P-14-3 [Secondary] suspended Mar 30 15:24:57:655 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:24:57:933 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:24:57:957 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:24:58:195 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (375 bytes) Mar 30 15:24:58:195 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:58:196 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (355 bytes) Mar 30 15:24:58:201 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:58:201 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (521 bytes, 0 ms) Mar 30 15:24:58:211 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (375 bytes) Mar 30 15:24:58:211 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:58:211 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (355 bytes) Mar 30 15:24:58:216 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:58:216 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (521 bytes, 0 ms) Mar 30 15:24:58:697 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:24:58:942 [140603169945344] 5 hub: TSESS-P-7-6 [TunnelB] suspended Mar 30 15:24:59:223 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (380 bytes) Mar 30 15:24:59:223 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:59:223 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (360 bytes) Mar 30 15:24:59:229 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:59:229 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (689 bytes, 0 ms) Mar 30 15:24:59:239 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (380 bytes) Mar 30 15:24:59:239 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:59:239 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (360 bytes) Mar 30 15:24:59:244 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:59:244 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (689 bytes, 0 ms) Mar 30 15:24:59:254 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (380 bytes) Mar 30 15:24:59:254 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:24:59:254 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (360 bytes) Mar 30 15:24:59:259 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:24:59:260 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (689 bytes, 0 ms) Mar 30 15:24:59:458 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:00:266 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (392 bytes) Mar 30 15:25:00:266 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:00:266 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (372 bytes) Mar 30 15:25:00:272 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:00:272 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (802 bytes, 0 ms) Mar 30 15:25:00:282 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (392 bytes) Mar 30 15:25:00:282 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:00:282 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (372 bytes) Mar 30 15:25:00:287 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:00:287 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (802 bytes, 0 ms) Mar 30 15:25:00:297 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (392 bytes) Mar 30 15:25:00:297 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:00:298 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (372 bytes) Mar 30 15:25:00:303 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:00:303 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (802 bytes, 0 ms) Mar 30 15:25:01:260 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:01:312 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (372 bytes) Mar 30 15:25:01:312 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:01:312 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (352 bytes) Mar 30 15:25:01:317 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:01:317 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (433 bytes, 0 ms) Mar 30 15:25:01:328 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (372 bytes) Mar 30 15:25:01:328 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:01:328 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (352 bytes) Mar 30 15:25:01:333 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:01:333 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (433 bytes, 0 ms) Mar 30 15:25:01:467 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:25:01:467 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:25:01:468 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60996 Mar 30 15:25:01:468 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60996' delegated to TSESS-A-5 [140603715176192] Mar 30 15:25:01:468 [140603715176192] 3 hub: TSESS-A-5-29 [TunnelB] restarted (for local) Mar 30 15:25:01:473 [140603715176192] 5 hub: TSESS-A-5-29 [TunnelB] local 100 'get_info' Mar 30 15:25:01:473 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:01:473 [140603715176192] 1 hub: TSESS-A-5-29 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:25:01:483 [140603715176192] 1 hub: TSESS-A-5-29 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:25:01:483 [140603715176192] 3 hub: TSESS-A-5-29 [TunnelB] is paired with [P-298-29] Mar 30 15:25:01:483 [140603715176192] 1 hub: TSESS-A-5-29 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:25:01:484 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/60998 Mar 30 15:25:01:484 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/60998' delegated to TSESS-A-10 [140603706783488] Mar 30 15:25:01:484 [140603706783488] 3 hub: TSESS-A-10-21 [TunnelB] restarted (for local) Mar 30 15:25:01:489 [140603715176192] 5 hub: TSESS-A-5-29 [TunnelB] local 100 '_close' Mar 30 15:25:01:489 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:01:489 [140603715176192] 1 hub: TSESS-A-5-29 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:25:01:489 [140603715176192] 3 hub: TSESS-A-5-29 [TunnelB] local session closed, suspending Mar 30 15:25:01:489 [140603706783488] 5 hub: TSESS-A-10-21 [TunnelB] local 100 '_status' Mar 30 15:25:01:489 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:01:489 [140603706783488] 1 hub: TSESS-A-10-21 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:25:01:500 [140603706783488] 1 hub: TSESS-A-10-21 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:25:01:500 [140603706783488] 3 hub: TSESS-A-10-21 [TunnelB] is paired with [P-300-21] Mar 30 15:25:01:500 [140603706783488] 1 hub: TSESS-A-10-21 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:25:01:505 [140603706783488] 5 hub: TSESS-A-10-21 [TunnelB] local 100 '_close' Mar 30 15:25:01:505 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:01:505 [140603706783488] 1 hub: TSESS-A-10-21 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:25:01:505 [140603706783488] 3 hub: TSESS-A-10-21 [TunnelB] local session closed, suspending Mar 30 15:25:01:690 [140603715176192] 5 hub: TSESS-A-5-29 [TunnelB] suspended Mar 30 15:25:01:706 [140603706783488] 5 hub: TSESS-A-10-21 [TunnelB] suspended Mar 30 15:25:02:339 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (377 bytes) Mar 30 15:25:02:339 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:02:339 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (357 bytes) Mar 30 15:25:02:345 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:02:345 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (428 bytes, 0 ms) Mar 30 15:25:02:355 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (377 bytes) Mar 30 15:25:02:355 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:02:355 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (357 bytes) Mar 30 15:25:02:360 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:02:360 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (428 bytes, 0 ms) Mar 30 15:25:03:063 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:03:369 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (371 bytes) Mar 30 15:25:03:369 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:03:369 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (351 bytes) Mar 30 15:25:03:375 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:03:375 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (403 bytes, 0 ms) Mar 30 15:25:03:385 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (371 bytes) Mar 30 15:25:03:385 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:03:385 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (351 bytes) Mar 30 15:25:03:390 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:03:390 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (403 bytes, 0 ms) Mar 30 15:25:04:392 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (380 bytes) Mar 30 15:25:04:392 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:04:393 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (360 bytes) Mar 30 15:25:04:398 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:04:398 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (310 bytes, 0 ms) Mar 30 15:25:05:405 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (393 bytes) Mar 30 15:25:05:405 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:05:405 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (373 bytes) Mar 30 15:25:05:410 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:05:410 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (555 bytes, 0 ms) Mar 30 15:25:05:421 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (393 bytes) Mar 30 15:25:05:421 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:05:421 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (373 bytes) Mar 30 15:25:05:426 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:05:426 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (555 bytes, 0 ms) Mar 30 15:25:05:472 [140605547460672] 3 hub: check_session - session 0x20f9050 (127.0.0.1/48002) is a server Mar 30 15:25:05:472 [140605547460672] 2 hub: Profile - clayk 38 bytes (OK) Mar 30 15:25:05:766 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:06:433 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (394 bytes) Mar 30 15:25:06:433 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:06:433 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (374 bytes) Mar 30 15:25:06:438 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:06:439 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (317 bytes, 0 ms) Mar 30 15:25:06:456 [140603203516160] 5 hub: TSESS-A-6-6 [Secondary] suspended Mar 30 15:25:06:491 [140603195123456] 5 hub: TSESS-A-3-6 [SecondaryHA] suspended Mar 30 15:25:06:953 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:25:06:986 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:25:07:361 [140603161552640] 5 hub: TSESS-P-4-6 [SecondaryHA] suspended Mar 30 15:25:07:446 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (390 bytes) Mar 30 15:25:07:446 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:07:446 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (370 bytes) Mar 30 15:25:07:452 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:07:452 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (565 bytes, 0 ms) Mar 30 15:25:07:462 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (390 bytes) Mar 30 15:25:07:462 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:07:462 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (370 bytes) Mar 30 15:25:07:467 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:07:467 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (565 bytes, 0 ms) Mar 30 15:25:07:474 [140605547460672] 4 hub: medium timeout Mar 30 15:25:07:474 [140605547460672] 3 hub: next_check [1585596396] now [1585596307] Mar 30 15:25:07:479 [140602666645248] 5 hub: TSESS-P-14-3 [Secondary] suspended Mar 30 15:25:07:489 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:25:07:489 [140603731961600] 2 hub: CTRL SecondaryHA request 'hubup' passed to TSESS-P-4 [140603161552640] Mar 30 15:25:07:489 [140603161552640] 3 hub: TSESS-P-4-7 [SecondaryHA] restarted (for remote) Mar 30 15:25:07:489 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:35:07 Mar 30 15:25:07:489 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:25:07:489 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:25:07:489 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:25:07:489 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:25:07:489 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:25:07:489 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:25:07:489 [140603161552640] 4 hub: CONNECT: 0x7fe09c000de0(40) 127.0.0.1/53478->127.0.0.1/48002 Mar 30 15:25:07:489 [140603161552640] 3 hub: TSESS-P-4-7 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:25:07:489 [140603161552640] 1 hub: TSESS-P-4-7 [SecondaryHA] received 100 'hubup' (687 bytes) Mar 30 15:25:07:489 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53478 (s:392) Mar 30 15:25:07:489 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/53478] Mar 30 15:25:07:489 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:25:07:490 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:25:07:490 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/53478 Mar 30 15:25:07:490 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/53478 same proximity=0 status=0 secver=217 Mar 30 15:25:07:495 [140603161552640] 5 hub: TSESS-P-4-7 [SecondaryHA] local 200 'OK' Mar 30 15:25:07:495 [140603161552640] 1 hub: TSESS-P-4-7 [SecondaryHA] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:25:07:505 [140603161552640] 1 hub: TSESS-P-4-7 [SecondaryHA] ssl 100 '_close' (300 bytes) Mar 30 15:25:07:505 [140603161552640] 3 hub: TSESS-P-4-7 [SecondaryHA] is paired with [A-1948-7] Mar 30 15:25:07:505 [140603161552640] 1 hub: TSESS-P-4-7 [SecondaryHA] received 100 '_close' (296 bytes) Mar 30 15:25:07:510 [140603161552640] 1 hub: TSESS-P-4-7 [SecondaryHA] remote session closed Mar 30 15:25:07:510 [140603161552640] 3 hub: TSESS-P-4-7 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:25:07:510 [140603161552640] 5 hub: TSESS-P-4-7 [SecondaryHA] suspended Mar 30 15:25:07:568 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:07:582 [140603740354304] 5 hub: TSESS-P-14 not internal command - thread will handle. mtype=100; sfd:29 Mar 30 15:25:07:582 [140603740354304] 2 hub: CTRL Secondary request 'hubup' passed to TSESS-P-14 [140602666645248] Mar 30 15:25:07:582 [140602666645248] 3 hub: TSESS-P-14-4 [Secondary] restarted (for remote) Mar 30 15:25:07:582 [140602666645248] 2 hub: TSESS-P-14 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:35:07 Mar 30 15:25:07:582 [140602666645248] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:25:07:582 [140602666645248] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:25:07:582 [140602666645248] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:25:07:582 [140602666645248] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:25:07:582 [140602666645248] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:25:07:582 [140602666645248] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:25:07:582 [140602666645248] 4 hub: CONNECT: 0x7fe094000b50(40) 127.0.0.1/53480->127.0.0.1/48002 Mar 30 15:25:07:582 [140602666645248] 3 hub: TSESS-P-14-4 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:25:07:582 [140602666645248] 1 hub: TSESS-P-14-4 [Secondary] received 100 'hubup' (683 bytes) Mar 30 15:25:07:582 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53480 (s:388) Mar 30 15:25:07:582 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/53480] Mar 30 15:25:07:582 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:25:07:582 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:25:07:582 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/53480 Mar 30 15:25:07:582 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/53480 same proximity=0 status=0 secver=217 Mar 30 15:25:07:587 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] local 200 'OK' Mar 30 15:25:07:587 [140602666645248] 1 hub: TSESS-P-14-4 [Secondary] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:25:07:598 [140602666645248] 1 hub: TSESS-P-14-4 [Secondary] ssl 100 '_close' (296 bytes) Mar 30 15:25:07:598 [140602666645248] 3 hub: TSESS-P-14-4 [Secondary] is paired with [A-2123-4] Mar 30 15:25:07:598 [140602666645248] 1 hub: TSESS-P-14-4 [Secondary] received 100 '_close' (296 bytes) Mar 30 15:25:07:603 [140602666645248] 1 hub: TSESS-P-14-4 [Secondary] remote session closed Mar 30 15:25:07:603 [140602666645248] 3 hub: TSESS-P-14-4 [Secondary] exiting msg loop (remote close) Mar 30 15:25:07:603 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:25:07:934 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:25:07:957 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:25:08:474 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (391 bytes) Mar 30 15:25:08:474 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:08:474 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (371 bytes) Mar 30 15:25:08:479 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:08:479 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (563 bytes, 0 ms) Mar 30 15:25:08:489 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (391 bytes) Mar 30 15:25:08:489 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:08:489 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (371 bytes) Mar 30 15:25:08:494 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:08:494 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (563 bytes, 0 ms) Mar 30 15:25:08:697 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:25:08:942 [140603169945344] 5 hub: TSESS-P-7-6 [TunnelB] suspended Mar 30 15:25:09:003 [140603723568896] 5 hub: TSESS-P-7 not internal command - thread will handle. mtype=100; sfd:22 Mar 30 15:25:09:003 [140603723568896] 2 hub: CTRL TunnelB request 'hubup' passed to TSESS-P-7 [140603169945344] Mar 30 15:25:09:003 [140603169945344] 3 hub: TSESS-P-7-7 [TunnelB] restarted (for remote) Mar 30 15:25:09:003 [140603169945344] 2 hub: TSESS-P-7 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:35:09 Mar 30 15:25:09:003 [140603169945344] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:25:09:003 [140603169945344] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:25:09:003 [140603169945344] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:25:09:003 [140603169945344] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:25:09:003 [140603169945344] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:25:09:004 [140603169945344] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:25:09:004 [140603169945344] 4 hub: CONNECT: 0x7fe0a8000d20(40) 127.0.0.1/53482->127.0.0.1/48002 Mar 30 15:25:09:004 [140603169945344] 3 hub: TSESS-P-7-7 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:25:09:004 [140603169945344] 1 hub: TSESS-P-7-7 [TunnelB] received 100 'hubup' (672 bytes) Mar 30 15:25:09:004 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53482 (s:378) Mar 30 15:25:09:004 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/53482] Mar 30 15:25:09:004 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:25:09:004 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:25:09:004 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/53482 Mar 30 15:25:09:004 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/53482 same proximity=0 status=0 secver=217 Mar 30 15:25:09:009 [140603169945344] 5 hub: TSESS-P-7-7 [TunnelB] local 200 'OK' Mar 30 15:25:09:009 [140603169945344] 1 hub: TSESS-P-7-7 [TunnelB] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:25:09:019 [140603169945344] 1 hub: TSESS-P-7-7 [TunnelB] ssl 100 '_close' (283 bytes) Mar 30 15:25:09:019 [140603169945344] 3 hub: TSESS-P-7-7 [TunnelB] is paired with [A-299-7] Mar 30 15:25:09:019 [140603169945344] 1 hub: TSESS-P-7-7 [TunnelB] received 100 '_close' (295 bytes) Mar 30 15:25:09:024 [140603169945344] 1 hub: TSESS-P-7-7 [TunnelB] remote session closed Mar 30 15:25:09:024 [140603169945344] 3 hub: TSESS-P-7-7 [TunnelB] exiting msg loop (remote close) Mar 30 15:25:09:024 [140603169945344] 5 hub: TSESS-P-7-7 [TunnelB] suspended Mar 30 15:25:09:370 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:09:502 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (388 bytes) Mar 30 15:25:09:502 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:09:502 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (368 bytes) Mar 30 15:25:09:507 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:09:507 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (539 bytes, 0 ms) Mar 30 15:25:09:517 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (388 bytes) Mar 30 15:25:09:517 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:09:517 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (368 bytes) Mar 30 15:25:09:522 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:09:522 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (539 bytes, 0 ms) Mar 30 15:25:10:529 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (380 bytes) Mar 30 15:25:10:529 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:10:529 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (360 bytes) Mar 30 15:25:10:534 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:10:534 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (523 bytes, 0 ms) Mar 30 15:25:10:545 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (380 bytes) Mar 30 15:25:10:545 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:10:545 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (360 bytes) Mar 30 15:25:10:550 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:10:550 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (523 bytes, 0 ms) Mar 30 15:25:11:173 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:11:511 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:25:11:511 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:25:11:511 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32778 Mar 30 15:25:11:511 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32778' delegated to TSESS-A-5 [140603715176192] Mar 30 15:25:11:511 [140603715176192] 3 hub: TSESS-A-5-30 [TunnelB] restarted (for local) Mar 30 15:25:11:517 [140603715176192] 5 hub: TSESS-A-5-30 [TunnelB] local 100 'get_info' Mar 30 15:25:11:517 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:11:517 [140603715176192] 1 hub: TSESS-A-5-30 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:25:11:527 [140603715176192] 1 hub: TSESS-A-5-30 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:25:11:527 [140603715176192] 3 hub: TSESS-A-5-30 [TunnelB] is paired with [P-298-30] Mar 30 15:25:11:527 [140603715176192] 1 hub: TSESS-A-5-30 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:25:11:527 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32780 Mar 30 15:25:11:527 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32780' delegated to TSESS-A-10 [140603706783488] Mar 30 15:25:11:527 [140603706783488] 3 hub: TSESS-A-10-22 [TunnelB] restarted (for local) Mar 30 15:25:11:532 [140603715176192] 5 hub: TSESS-A-5-30 [TunnelB] local 100 '_close' Mar 30 15:25:11:532 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:11:532 [140603715176192] 1 hub: TSESS-A-5-30 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:25:11:532 [140603715176192] 3 hub: TSESS-A-5-30 [TunnelB] local session closed, suspending Mar 30 15:25:11:532 [140603706783488] 5 hub: TSESS-A-10-22 [TunnelB] local 100 '_status' Mar 30 15:25:11:532 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:11:532 [140603706783488] 1 hub: TSESS-A-10-22 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:25:11:543 [140603706783488] 1 hub: TSESS-A-10-22 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:25:11:543 [140603706783488] 3 hub: TSESS-A-10-22 [TunnelB] is paired with [P-300-22] Mar 30 15:25:11:543 [140603706783488] 1 hub: TSESS-A-10-22 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:25:11:548 [140603706783488] 5 hub: TSESS-A-10-22 [TunnelB] local 100 '_close' Mar 30 15:25:11:548 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:11:548 [140603706783488] 1 hub: TSESS-A-10-22 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:25:11:548 [140603706783488] 3 hub: TSESS-A-10-22 [TunnelB] local session closed, suspending Mar 30 15:25:11:556 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (369 bytes) Mar 30 15:25:11:556 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:11:556 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (349 bytes) Mar 30 15:25:11:562 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:11:562 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:25:11:572 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (369 bytes) Mar 30 15:25:11:572 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:11:572 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (349 bytes) Mar 30 15:25:11:577 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:11:577 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:25:11:732 [140603715176192] 5 hub: TSESS-A-5-30 [TunnelB] suspended Mar 30 15:25:11:748 [140603706783488] 5 hub: TSESS-A-10-22 [TunnelB] suspended Mar 30 15:25:12:583 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (382 bytes) Mar 30 15:25:12:583 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:12:584 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (362 bytes) Mar 30 15:25:12:589 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:12:589 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (463 bytes, 0 ms) Mar 30 15:25:12:599 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 'archive_list' (382 bytes) Mar 30 15:25:12:599 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:12:599 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 'archive_list' (362 bytes) Mar 30 15:25:12:604 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] local 200 'OK' Mar 30 15:25:12:604 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] sent 200 'OK' (463 bytes, 0 ms) Mar 30 15:25:12:614 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] ssl 100 '_close' (329 bytes) Mar 30 15:25:12:614 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] is paired with [A-2121-4] Mar 30 15:25:12:614 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] received 100 '_close' (309 bytes) Mar 30 15:25:12:619 [140603153159936] 1 hub: TSESS-P-8-4 [Secondary] remote session closed Mar 30 15:25:12:619 [140603153159936] 3 hub: TSESS-P-8-4 [Secondary] exiting msg loop (remote close) Mar 30 15:25:12:620 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] suspended Mar 30 15:25:13:876 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:15:679 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:16:324 [140605283890944] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:25:16:324 [140605283890944] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:25:16:324 [140605283890944] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:25:16:324 [140605283890944] 1 hub: PERF: hubup_check_interval: 0 sec hubhash_size: 18 Mar 30 15:25:16:324 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:324 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:324 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:324 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:324 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:324 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:324 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:25:16:324 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:324 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:324 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:324 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:324 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:324 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:325 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:325 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:325 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:25:16:325 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:25:16:325 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:25:16:325 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/47154->127.0.0.1/48007 Mar 30 15:25:16:325 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/47154 Mar 30 15:25:16:325 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/47154' delegated to TSESS-A-6 [140603203516160] Mar 30 15:25:16:325 [140603203516160] 3 hub: TSESS-A-6-7 [Secondary] restarted (for local) Mar 30 15:25:16:330 [140603203516160] 5 hub: TSESS-A-6-7 [Secondary] local 100 'hubup' Mar 30 15:25:16:330 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:16:330 [140603203516160] 1 hub: TSESS-A-6-7 [Secondary] sent 100 'hubup' (686 bytes, 0 ms) Mar 30 15:25:16:340 [140603203516160] 1 hub: TSESS-A-6-7 [Secondary] ssl 200 '-' (453 bytes) Mar 30 15:25:16:340 [140603203516160] 3 hub: TSESS-A-6-7 [Secondary] is paired with [P-2120-7] Mar 30 15:25:16:340 [140603203516160] 1 hub: TSESS-A-6-7 [Secondary] received 200 '-' (461 bytes) Mar 30 15:25:16:340 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:25:16:340 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:25:16:340 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:25:16:340 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:25:16:340 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary same proximity=0 status=0 secver=217 Mar 30 15:25:16:340 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:340 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:340 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:340 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:340 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:340 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:340 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:25:16:340 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:340 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:340 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:340 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:340 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:340 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:340 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:340 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:340 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:25:16:340 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:25:16:340 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:25:16:341 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/32786->127.0.0.1/48009 Mar 30 15:25:16:341 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32786 Mar 30 15:25:16:341 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32786' delegated to TSESS-A-5 [140603715176192] Mar 30 15:25:16:341 [140603715176192] 3 hub: TSESS-A-5-31 [TunnelB] restarted (for local) Mar 30 15:25:16:345 [140603203516160] 5 hub: TSESS-A-6-7 [Secondary] local 100 '_close' Mar 30 15:25:16:345 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:16:345 [140603203516160] 1 hub: TSESS-A-6-7 [Secondary] sent 100 '_close' (293 bytes, 0 ms) Mar 30 15:25:16:345 [140603203516160] 3 hub: TSESS-A-6-7 [Secondary] local session closed, suspending Mar 30 15:25:16:346 [140603715176192] 5 hub: TSESS-A-5-31 [TunnelB] local 100 'hubup' Mar 30 15:25:16:346 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:16:346 [140603715176192] 1 hub: TSESS-A-5-31 [TunnelB] sent 100 'hubup' (685 bytes, 0 ms) Mar 30 15:25:16:356 [140603715176192] 1 hub: TSESS-A-5-31 [TunnelB] ssl 200 '-' (443 bytes) Mar 30 15:25:16:356 [140603715176192] 3 hub: TSESS-A-5-31 [TunnelB] is paired with [P-298-31] Mar 30 15:25:16:356 [140603715176192] 1 hub: TSESS-A-5-31 [TunnelB] received 200 '-' (451 bytes) Mar 30 15:25:16:356 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:25:16:356 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:25:16:356 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:25:16:356 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:25:16:356 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB same proximity=0 status=0 secver=217 Mar 30 15:25:16:356 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:356 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:356 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:356 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:356 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:356 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:356 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:25:16:356 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:356 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:356 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:356 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:356 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:356 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:356 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:356 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:16:356 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:25:16:356 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:25:16:356 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:25:16:357 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/40206->127.0.0.1/48008 Mar 30 15:25:16:357 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/40206 Mar 30 15:25:16:357 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/40206' delegated to TSESS-A-3 [140603195123456] Mar 30 15:25:16:357 [140603195123456] 3 hub: TSESS-A-3-7 [SecondaryHA] restarted (for local) Mar 30 15:25:16:361 [140603715176192] 5 hub: TSESS-A-5-31 [TunnelB] local 100 '_close' Mar 30 15:25:16:361 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:16:361 [140603715176192] 1 hub: TSESS-A-5-31 [TunnelB] sent 100 '_close' (292 bytes, 0 ms) Mar 30 15:25:16:361 [140603715176192] 3 hub: TSESS-A-5-31 [TunnelB] local session closed, suspending Mar 30 15:25:16:362 [140603195123456] 5 hub: TSESS-A-3-7 [SecondaryHA] local 100 'hubup' Mar 30 15:25:16:362 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:16:362 [140603195123456] 1 hub: TSESS-A-3-7 [SecondaryHA] sent 100 'hubup' (688 bytes, 0 ms) Mar 30 15:25:16:372 [140603195123456] 1 hub: TSESS-A-3-7 [SecondaryHA] ssl 200 '-' (457 bytes) Mar 30 15:25:16:372 [140603195123456] 3 hub: TSESS-A-3-7 [SecondaryHA] is paired with [P-1947-7] Mar 30 15:25:16:372 [140603195123456] 1 hub: TSESS-A-3-7 [SecondaryHA] received 200 '-' (465 bytes) Mar 30 15:25:16:372 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:25:16:372 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:25:16:372 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:25:16:372 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:25:16:372 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA same proximity=0 status=0 secver=217 Mar 30 15:25:16:372 [140605283890944] 5 hub: mark_hubup_list_updated Mar 30 15:25:16:372 [140605283890944] 5 hub: hublist changed: /DOMAIN/Primary 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:16:372 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:16:372 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:16:372 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:16:372 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:16:372 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:16:372 [140605283890944] 5 hub: hublist changed: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:25:16:372 [140605283890944] 5 hub: hublist changed: /DOMAIN/PrimaryHA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:16:372 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:16:372 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:16:372 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:16:372 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:16:372 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:16:372 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:16:372 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:16:372 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:25:16:372 [140605283890944] 5 hub: hublist changed: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:25:16:372 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelBHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:25:16:372 [140605283890944] 5 hub: mark_hubup_list_updated end Mar 30 15:25:16:372 [140605283890944] 1 hub: PERF: hubup_check_for_updates: 0 sec hubhash_size: 18 Mar 30 15:25:16:372 [140605283890944] 1 hub: PERF: hubi_run_command: 0 sec hubhash_size: 18 Mar 30 15:25:16:372 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:25:16:373 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:25:16:374 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:25:16:374 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:25:16:375 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:25:16:375 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:25:16:375 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:25:16:376 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:25:16:376 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:25:16:376 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:25:16:377 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:25:16:377 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:25:16:377 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:25:16:377 [140603195123456] 5 hub: TSESS-A-3-7 [SecondaryHA] local 100 '_close' Mar 30 15:25:16:377 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:16:377 [140603195123456] 1 hub: TSESS-A-3-7 [SecondaryHA] sent 100 '_close' (295 bytes, 0 ms) Mar 30 15:25:16:377 [140603195123456] 3 hub: TSESS-A-3-7 [SecondaryHA] local session closed, suspending Mar 30 15:25:16:378 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:25:16:378 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:25:16:379 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:25:16:379 [140605283890944] 1 hub: PERF: hubi_check_security: 0 sec hubhash_size: 18 Mar 30 15:25:16:546 [140603203516160] 5 hub: TSESS-A-6-7 [Secondary] suspended Mar 30 15:25:16:562 [140603715176192] 5 hub: TSESS-A-5-31 [TunnelB] suspended Mar 30 15:25:16:578 [140603195123456] 5 hub: TSESS-A-3-7 [SecondaryHA] suspended Mar 30 15:25:16:954 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:25:16:986 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:25:17:481 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:17:510 [140603161552640] 5 hub: TSESS-P-4-7 [SecondaryHA] suspended Mar 30 15:25:17:518 [140605547460672] 4 hub: medium timeout Mar 30 15:25:17:518 [140605547460672] 3 hub: next_check [1585596396] now [1585596317] Mar 30 15:25:17:603 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:25:17:934 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:25:17:957 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:25:18:697 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:25:19:024 [140603169945344] 5 hub: TSESS-P-7-7 [TunnelB] suspended Mar 30 15:25:19:284 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:21:086 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:21:553 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:25:21:553 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:25:21:554 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32794 Mar 30 15:25:21:554 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32794' delegated to TSESS-A-5 [140603715176192] Mar 30 15:25:21:554 [140603715176192] 3 hub: TSESS-A-5-32 [TunnelB] restarted (for local) Mar 30 15:25:21:559 [140603715176192] 5 hub: TSESS-A-5-32 [TunnelB] local 100 'get_info' Mar 30 15:25:21:559 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:21:559 [140603715176192] 1 hub: TSESS-A-5-32 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:25:21:569 [140603715176192] 1 hub: TSESS-A-5-32 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:25:21:569 [140603715176192] 3 hub: TSESS-A-5-32 [TunnelB] is paired with [P-298-32] Mar 30 15:25:21:569 [140603715176192] 1 hub: TSESS-A-5-32 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:25:21:570 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32796 Mar 30 15:25:21:570 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32796' delegated to TSESS-A-10 [140603706783488] Mar 30 15:25:21:570 [140603706783488] 3 hub: TSESS-A-10-23 [TunnelB] restarted (for local) Mar 30 15:25:21:574 [140603715176192] 5 hub: TSESS-A-5-32 [TunnelB] local 100 '_close' Mar 30 15:25:21:574 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:21:574 [140603715176192] 1 hub: TSESS-A-5-32 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:25:21:575 [140603715176192] 3 hub: TSESS-A-5-32 [TunnelB] local session closed, suspending Mar 30 15:25:21:575 [140603706783488] 5 hub: TSESS-A-10-23 [TunnelB] local 100 '_status' Mar 30 15:25:21:575 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:21:575 [140603706783488] 1 hub: TSESS-A-10-23 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:25:21:585 [140603706783488] 1 hub: TSESS-A-10-23 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:25:21:585 [140603706783488] 3 hub: TSESS-A-10-23 [TunnelB] is paired with [P-300-23] Mar 30 15:25:21:585 [140603706783488] 1 hub: TSESS-A-10-23 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:25:21:590 [140603706783488] 5 hub: TSESS-A-10-23 [TunnelB] local 100 '_close' Mar 30 15:25:21:590 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:21:590 [140603706783488] 1 hub: TSESS-A-10-23 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:25:21:590 [140603706783488] 3 hub: TSESS-A-10-23 [TunnelB] local session closed, suspending Mar 30 15:25:21:775 [140603715176192] 5 hub: TSESS-A-5-32 [TunnelB] suspended Mar 30 15:25:21:791 [140603706783488] 5 hub: TSESS-A-10-23 [TunnelB] suspended Mar 30 15:25:22:620 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] suspended Mar 30 15:25:23:790 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:25:592 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:26:546 [140603203516160] 5 hub: TSESS-A-6-7 [Secondary] suspended Mar 30 15:25:26:578 [140603195123456] 5 hub: TSESS-A-3-7 [SecondaryHA] suspended Mar 30 15:25:26:958 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:25:26:986 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:25:27:395 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:27:510 [140603161552640] 5 hub: TSESS-P-4-7 [SecondaryHA] suspended Mar 30 15:25:27:559 [140605547460672] 4 hub: medium timeout Mar 30 15:25:27:559 [140605547460672] 3 hub: next_check [1585596396] now [1585596327] Mar 30 15:25:27:603 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:25:27:934 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:25:27:958 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:25:28:698 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:25:29:025 [140603169945344] 5 hub: TSESS-P-7-7 [TunnelB] suspended Mar 30 15:25:29:198 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:31:596 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:25:31:596 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:25:31:597 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32802 Mar 30 15:25:31:597 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32802' delegated to TSESS-A-5 [140603715176192] Mar 30 15:25:31:597 [140603715176192] 3 hub: TSESS-A-5-33 [TunnelB] restarted (for local) Mar 30 15:25:31:602 [140603715176192] 5 hub: TSESS-A-5-33 [TunnelB] local 100 'get_info' Mar 30 15:25:31:602 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:31:602 [140603715176192] 1 hub: TSESS-A-5-33 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:25:31:612 [140603715176192] 1 hub: TSESS-A-5-33 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:25:31:612 [140603715176192] 3 hub: TSESS-A-5-33 [TunnelB] is paired with [P-298-33] Mar 30 15:25:31:612 [140603715176192] 1 hub: TSESS-A-5-33 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:25:31:613 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32804 Mar 30 15:25:31:613 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32804' delegated to TSESS-A-10 [140603706783488] Mar 30 15:25:31:613 [140603706783488] 3 hub: TSESS-A-10-24 [TunnelB] restarted (for local) Mar 30 15:25:31:618 [140603715176192] 5 hub: TSESS-A-5-33 [TunnelB] local 100 '_close' Mar 30 15:25:31:618 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:31:618 [140603715176192] 1 hub: TSESS-A-5-33 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:25:31:618 [140603715176192] 3 hub: TSESS-A-5-33 [TunnelB] local session closed, suspending Mar 30 15:25:31:618 [140603706783488] 5 hub: TSESS-A-10-24 [TunnelB] local 100 '_status' Mar 30 15:25:31:618 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:31:618 [140603706783488] 1 hub: TSESS-A-10-24 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:25:31:628 [140603706783488] 1 hub: TSESS-A-10-24 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:25:31:628 [140603706783488] 3 hub: TSESS-A-10-24 [TunnelB] is paired with [P-300-24] Mar 30 15:25:31:628 [140603706783488] 1 hub: TSESS-A-10-24 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:25:31:633 [140603706783488] 5 hub: TSESS-A-10-24 [TunnelB] local 100 '_close' Mar 30 15:25:31:633 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:31:633 [140603706783488] 1 hub: TSESS-A-10-24 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:25:31:633 [140603706783488] 3 hub: TSESS-A-10-24 [TunnelB] local session closed, suspending Mar 30 15:25:31:818 [140603715176192] 5 hub: TSESS-A-5-33 [TunnelB] suspended Mar 30 15:25:31:834 [140603706783488] 5 hub: TSESS-A-10-24 [TunnelB] suspended Mar 30 15:25:31:900 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:32:620 [140603153159936] 5 hub: TSESS-P-8-4 [Secondary] suspended Mar 30 15:25:33:703 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:34:923 [140604042360576] 5 hub: There are 0 objects waiting on 8 threads Mar 30 15:25:35:506 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:35:600 [140605547460672] 3 hub: check_session - session 0x20f9050 (127.0.0.1/48002) is a server Mar 30 15:25:35:600 [140605547460672] 2 hub: Profile - clayk 38 bytes (OK) Mar 30 15:25:36:546 [140603203516160] 5 hub: TSESS-A-6-7 [Secondary] suspended Mar 30 15:25:36:578 [140603195123456] 5 hub: TSESS-A-3-7 [SecondaryHA] suspended Mar 30 15:25:36:958 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:25:36:986 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:25:37:308 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:37:510 [140603161552640] 5 hub: TSESS-P-4-7 [SecondaryHA] suspended Mar 30 15:25:37:603 [140605547460672] 4 hub: medium timeout Mar 30 15:25:37:603 [140605547460672] 3 hub: next_check [1585596396] now [1585596337] Mar 30 15:25:37:603 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:25:37:623 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:25:37:623 [140603731961600] 2 hub: CTRL SecondaryHA request 'hubup' passed to TSESS-P-4 [140603161552640] Mar 30 15:25:37:623 [140603161552640] 3 hub: TSESS-P-4-8 [SecondaryHA] restarted (for remote) Mar 30 15:25:37:623 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:35:37 Mar 30 15:25:37:623 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:25:37:623 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:25:37:623 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:25:37:623 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:25:37:623 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:25:37:623 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:25:37:623 [140603161552640] 4 hub: CONNECT: 0x7fe09c000de0(39) 127.0.0.1/53516->127.0.0.1/48002 Mar 30 15:25:37:623 [140603161552640] 3 hub: TSESS-P-4-8 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:25:37:623 [140603161552640] 1 hub: TSESS-P-4-8 [SecondaryHA] received 100 'hubup' (687 bytes) Mar 30 15:25:37:623 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53516 (s:392) Mar 30 15:25:37:623 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/53516] Mar 30 15:25:37:623 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:25:37:623 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:25:37:623 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/53516 Mar 30 15:25:37:623 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/53516 same proximity=0 status=0 secver=217 Mar 30 15:25:37:628 [140603161552640] 5 hub: TSESS-P-4-8 [SecondaryHA] local 200 'OK' Mar 30 15:25:37:628 [140603161552640] 1 hub: TSESS-P-4-8 [SecondaryHA] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:25:37:639 [140603161552640] 1 hub: TSESS-P-4-8 [SecondaryHA] ssl 100 '_close' (300 bytes) Mar 30 15:25:37:639 [140603161552640] 3 hub: TSESS-P-4-8 [SecondaryHA] is paired with [A-1948-8] Mar 30 15:25:37:639 [140603161552640] 1 hub: TSESS-P-4-8 [SecondaryHA] received 100 '_close' (296 bytes) Mar 30 15:25:37:644 [140603161552640] 1 hub: TSESS-P-4-8 [SecondaryHA] remote session closed Mar 30 15:25:37:644 [140603161552640] 3 hub: TSESS-P-4-8 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:25:37:644 [140603161552640] 5 hub: TSESS-P-4-8 [SecondaryHA] suspended Mar 30 15:25:37:732 [140603740354304] 5 hub: TSESS-P-8 not internal command - thread will handle. mtype=100; sfd:23 Mar 30 15:25:37:732 [140603740354304] 2 hub: CTRL Secondary request 'hubup' passed to TSESS-P-8 [140603153159936] Mar 30 15:25:37:732 [140603153159936] 3 hub: TSESS-P-8-5 [Secondary] restarted (for remote) Mar 30 15:25:37:732 [140603153159936] 2 hub: TSESS-P-8 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:35:37 Mar 30 15:25:37:732 [140603153159936] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:25:37:733 [140603153159936] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:25:37:733 [140603153159936] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:25:37:733 [140603153159936] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:25:37:733 [140603153159936] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:25:37:733 [140603153159936] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:25:37:733 [140603153159936] 4 hub: CONNECT: 0x7fe0a0000d90(39) 127.0.0.1/53518->127.0.0.1/48002 Mar 30 15:25:37:733 [140603153159936] 3 hub: TSESS-P-8-5 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:25:37:733 [140603153159936] 1 hub: TSESS-P-8-5 [Secondary] received 100 'hubup' (683 bytes) Mar 30 15:25:37:733 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53518 (s:388) Mar 30 15:25:37:733 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/53518] Mar 30 15:25:37:733 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:25:37:733 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:25:37:733 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/53518 Mar 30 15:25:37:733 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/53518 same proximity=0 status=0 secver=217 Mar 30 15:25:37:738 [140603153159936] 5 hub: TSESS-P-8-5 [Secondary] local 200 'OK' Mar 30 15:25:37:738 [140603153159936] 1 hub: TSESS-P-8-5 [Secondary] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:25:37:748 [140603153159936] 1 hub: TSESS-P-8-5 [Secondary] ssl 100 '_close' (296 bytes) Mar 30 15:25:37:748 [140603153159936] 3 hub: TSESS-P-8-5 [Secondary] is paired with [A-2121-5] Mar 30 15:25:37:748 [140603153159936] 1 hub: TSESS-P-8-5 [Secondary] received 100 '_close' (296 bytes) Mar 30 15:25:37:753 [140603153159936] 1 hub: TSESS-P-8-5 [Secondary] remote session closed Mar 30 15:25:37:753 [140603153159936] 3 hub: TSESS-P-8-5 [Secondary] exiting msg loop (remote close) Mar 30 15:25:37:753 [140603153159936] 5 hub: TSESS-P-8-5 [Secondary] suspended Mar 30 15:25:37:934 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:25:37:958 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:25:38:087 [140603723568896] 5 hub: TSESS-P-7 not internal command - thread will handle. mtype=100; sfd:22 Mar 30 15:25:38:087 [140603723568896] 2 hub: CTRL TunnelB request 'hubup' passed to TSESS-P-7 [140603169945344] Mar 30 15:25:38:087 [140603169945344] 3 hub: TSESS-P-7-8 [TunnelB] restarted (for remote) Mar 30 15:25:38:087 [140603169945344] 2 hub: TSESS-P-7 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:35:38 Mar 30 15:25:38:087 [140603169945344] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:25:38:088 [140603169945344] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:25:38:088 [140603169945344] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:25:38:088 [140603169945344] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:25:38:088 [140603169945344] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:25:38:088 [140603169945344] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:25:38:088 [140603169945344] 4 hub: CONNECT: 0x7fe0a8000d20(39) 127.0.0.1/53520->127.0.0.1/48002 Mar 30 15:25:38:088 [140603169945344] 3 hub: TSESS-P-7-8 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:25:38:088 [140603169945344] 1 hub: TSESS-P-7-8 [TunnelB] received 100 'hubup' (672 bytes) Mar 30 15:25:38:088 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53520 (s:378) Mar 30 15:25:38:088 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/53520] Mar 30 15:25:38:088 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:25:38:088 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:25:38:088 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/53520 Mar 30 15:25:38:088 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/53520 same proximity=0 status=0 secver=217 Mar 30 15:25:38:093 [140603169945344] 5 hub: TSESS-P-7-8 [TunnelB] local 200 'OK' Mar 30 15:25:38:093 [140603169945344] 1 hub: TSESS-P-7-8 [TunnelB] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:25:38:103 [140603169945344] 1 hub: TSESS-P-7-8 [TunnelB] ssl 100 '_close' (283 bytes) Mar 30 15:25:38:103 [140603169945344] 3 hub: TSESS-P-7-8 [TunnelB] is paired with [A-299-8] Mar 30 15:25:38:103 [140603169945344] 1 hub: TSESS-P-7-8 [TunnelB] received 100 '_close' (295 bytes) Mar 30 15:25:38:108 [140603169945344] 1 hub: TSESS-P-7-8 [TunnelB] remote session closed Mar 30 15:25:38:108 [140603169945344] 3 hub: TSESS-P-7-8 [TunnelB] exiting msg loop (remote close) Mar 30 15:25:38:108 [140603169945344] 5 hub: TSESS-P-7-8 [TunnelB] suspended Mar 30 15:25:38:698 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:25:39:110 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:41:638 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:25:41:638 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:25:41:639 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32818 Mar 30 15:25:41:639 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32818' delegated to TSESS-A-5 [140603715176192] Mar 30 15:25:41:639 [140603715176192] 3 hub: TSESS-A-5-34 [TunnelB] restarted (for local) Mar 30 15:25:41:644 [140603715176192] 5 hub: TSESS-A-5-34 [TunnelB] local 100 'get_info' Mar 30 15:25:41:644 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:41:644 [140603715176192] 1 hub: TSESS-A-5-34 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:25:41:654 [140603715176192] 1 hub: TSESS-A-5-34 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:25:41:654 [140603715176192] 3 hub: TSESS-A-5-34 [TunnelB] is paired with [P-298-34] Mar 30 15:25:41:654 [140603715176192] 1 hub: TSESS-A-5-34 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:25:41:655 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32820 Mar 30 15:25:41:655 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32820' delegated to TSESS-A-10 [140603706783488] Mar 30 15:25:41:655 [140603706783488] 3 hub: TSESS-A-10-25 [TunnelB] restarted (for local) Mar 30 15:25:41:659 [140603715176192] 5 hub: TSESS-A-5-34 [TunnelB] local 100 '_close' Mar 30 15:25:41:660 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:41:660 [140603715176192] 1 hub: TSESS-A-5-34 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:25:41:660 [140603715176192] 3 hub: TSESS-A-5-34 [TunnelB] local session closed, suspending Mar 30 15:25:41:660 [140603706783488] 5 hub: TSESS-A-10-25 [TunnelB] local 100 '_status' Mar 30 15:25:41:660 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:41:660 [140603706783488] 1 hub: TSESS-A-10-25 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:25:41:670 [140603706783488] 1 hub: TSESS-A-10-25 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:25:41:670 [140603706783488] 3 hub: TSESS-A-10-25 [TunnelB] is paired with [P-300-25] Mar 30 15:25:41:670 [140603706783488] 1 hub: TSESS-A-10-25 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:25:41:675 [140603706783488] 5 hub: TSESS-A-10-25 [TunnelB] local 100 '_close' Mar 30 15:25:41:675 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:41:675 [140603706783488] 1 hub: TSESS-A-10-25 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:25:41:675 [140603706783488] 3 hub: TSESS-A-10-25 [TunnelB] local session closed, suspending Mar 30 15:25:41:814 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:41:860 [140603715176192] 5 hub: TSESS-A-5-34 [TunnelB] suspended Mar 30 15:25:41:876 [140603706783488] 5 hub: TSESS-A-10-25 [TunnelB] suspended Mar 30 15:25:43:617 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:45:420 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:46:411 [140605283890944] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:25:46:412 [140605283890944] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:25:46:412 [140605283890944] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:25:46:412 [140605283890944] 1 hub: PERF: hubup_check_interval: 0 sec hubhash_size: 18 Mar 30 15:25:46:412 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:412 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:412 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:412 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:412 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:412 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:412 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:25:46:412 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:412 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:412 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:412 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:412 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:412 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:412 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:412 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:412 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:25:46:412 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:25:46:412 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:25:46:412 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/47196->127.0.0.1/48007 Mar 30 15:25:46:412 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/47196 Mar 30 15:25:46:412 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/47196' delegated to TSESS-A-6 [140603203516160] Mar 30 15:25:46:412 [140603203516160] 3 hub: TSESS-A-6-8 [Secondary] restarted (for local) Mar 30 15:25:46:417 [140603203516160] 5 hub: TSESS-A-6-8 [Secondary] local 100 'hubup' Mar 30 15:25:46:417 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:46:417 [140603203516160] 1 hub: TSESS-A-6-8 [Secondary] sent 100 'hubup' (686 bytes, 0 ms) Mar 30 15:25:46:428 [140603203516160] 1 hub: TSESS-A-6-8 [Secondary] ssl 200 '-' (453 bytes) Mar 30 15:25:46:428 [140603203516160] 3 hub: TSESS-A-6-8 [Secondary] is paired with [P-2120-8] Mar 30 15:25:46:428 [140603203516160] 1 hub: TSESS-A-6-8 [Secondary] received 200 '-' (461 bytes) Mar 30 15:25:46:428 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:25:46:428 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:25:46:428 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:25:46:428 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:25:46:428 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary same proximity=0 status=0 secver=217 Mar 30 15:25:46:428 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:428 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:428 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:428 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:428 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:428 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:428 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:25:46:428 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:428 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:428 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:428 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:428 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:428 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:428 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:428 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:428 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:25:46:428 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:25:46:428 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:25:46:428 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/32828->127.0.0.1/48009 Mar 30 15:25:46:428 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32828 Mar 30 15:25:46:428 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32828' delegated to TSESS-A-5 [140603715176192] Mar 30 15:25:46:428 [140603715176192] 3 hub: TSESS-A-5-35 [TunnelB] restarted (for local) Mar 30 15:25:46:433 [140603203516160] 5 hub: TSESS-A-6-8 [Secondary] local 100 '_close' Mar 30 15:25:46:433 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:46:433 [140603203516160] 1 hub: TSESS-A-6-8 [Secondary] sent 100 '_close' (293 bytes, 0 ms) Mar 30 15:25:46:433 [140603203516160] 3 hub: TSESS-A-6-8 [Secondary] local session closed, suspending Mar 30 15:25:46:433 [140603715176192] 5 hub: TSESS-A-5-35 [TunnelB] local 100 'hubup' Mar 30 15:25:46:433 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:46:433 [140603715176192] 1 hub: TSESS-A-5-35 [TunnelB] sent 100 'hubup' (685 bytes, 0 ms) Mar 30 15:25:46:444 [140603715176192] 1 hub: TSESS-A-5-35 [TunnelB] ssl 200 '-' (443 bytes) Mar 30 15:25:46:444 [140603715176192] 3 hub: TSESS-A-5-35 [TunnelB] is paired with [P-298-35] Mar 30 15:25:46:444 [140603715176192] 1 hub: TSESS-A-5-35 [TunnelB] received 200 '-' (451 bytes) Mar 30 15:25:46:444 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:25:46:444 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:25:46:444 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:25:46:444 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:25:46:444 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB same proximity=0 status=0 secver=217 Mar 30 15:25:46:444 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:444 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:444 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:444 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:444 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:444 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:444 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:25:46:444 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:444 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:444 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:444 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:444 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:444 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:444 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:444 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:25:46:444 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:25:46:444 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:25:46:444 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:25:46:444 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/40248->127.0.0.1/48008 Mar 30 15:25:46:444 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/40248 Mar 30 15:25:46:444 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/40248' delegated to TSESS-A-3 [140603195123456] Mar 30 15:25:46:444 [140603195123456] 3 hub: TSESS-A-3-8 [SecondaryHA] restarted (for local) Mar 30 15:25:46:449 [140603715176192] 5 hub: TSESS-A-5-35 [TunnelB] local 100 '_close' Mar 30 15:25:46:449 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:46:449 [140603715176192] 1 hub: TSESS-A-5-35 [TunnelB] sent 100 '_close' (292 bytes, 0 ms) Mar 30 15:25:46:449 [140603715176192] 3 hub: TSESS-A-5-35 [TunnelB] local session closed, suspending Mar 30 15:25:46:449 [140603195123456] 5 hub: TSESS-A-3-8 [SecondaryHA] local 100 'hubup' Mar 30 15:25:46:449 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:46:449 [140603195123456] 1 hub: TSESS-A-3-8 [SecondaryHA] sent 100 'hubup' (688 bytes, 0 ms) Mar 30 15:25:46:459 [140603195123456] 1 hub: TSESS-A-3-8 [SecondaryHA] ssl 200 '-' (457 bytes) Mar 30 15:25:46:459 [140603195123456] 3 hub: TSESS-A-3-8 [SecondaryHA] is paired with [P-1947-8] Mar 30 15:25:46:459 [140603195123456] 1 hub: TSESS-A-3-8 [SecondaryHA] received 200 '-' (465 bytes) Mar 30 15:25:46:460 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:25:46:460 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:25:46:460 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:25:46:460 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:25:46:460 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA same proximity=0 status=0 secver=217 Mar 30 15:25:46:460 [140605283890944] 5 hub: mark_hubup_list_updated Mar 30 15:25:46:460 [140605283890944] 5 hub: hublist changed: /DOMAIN/Primary 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:46:460 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:46:460 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:46:460 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:46:460 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:46:460 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:46:460 [140605283890944] 5 hub: hublist changed: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:25:46:460 [140605283890944] 5 hub: hublist changed: /DOMAIN/PrimaryHA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:46:460 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:46:460 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:46:460 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:46:460 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:46:460 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:46:460 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:46:460 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:25:46:460 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:25:46:460 [140605283890944] 5 hub: hublist changed: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:25:46:460 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelBHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:25:46:460 [140605283890944] 5 hub: mark_hubup_list_updated end Mar 30 15:25:46:460 [140605283890944] 1 hub: PERF: hubup_check_for_updates: 0 sec hubhash_size: 18 Mar 30 15:25:46:460 [140605283890944] 1 hub: PERF: hubi_run_command: 0 sec hubhash_size: 18 Mar 30 15:25:46:460 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:25:46:460 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:25:46:461 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:25:46:462 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:25:46:462 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:25:46:462 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:25:46:462 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:25:46:463 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:25:46:463 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:25:46:464 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:25:46:464 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:25:46:464 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:25:46:464 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:25:46:465 [140603195123456] 5 hub: TSESS-A-3-8 [SecondaryHA] local 100 '_close' Mar 30 15:25:46:465 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:46:465 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:25:46:465 [140603195123456] 1 hub: TSESS-A-3-8 [SecondaryHA] sent 100 '_close' (295 bytes, 0 ms) Mar 30 15:25:46:465 [140603195123456] 3 hub: TSESS-A-3-8 [SecondaryHA] local session closed, suspending Mar 30 15:25:46:466 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:25:46:466 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:25:46:467 [140605283890944] 1 hub: PERF: hubi_check_security: 0 sec hubhash_size: 18 Mar 30 15:25:46:633 [140603203516160] 5 hub: TSESS-A-6-8 [Secondary] suspended Mar 30 15:25:46:649 [140603715176192] 5 hub: TSESS-A-5-35 [TunnelB] suspended Mar 30 15:25:46:665 [140603195123456] 5 hub: TSESS-A-3-8 [SecondaryHA] suspended Mar 30 15:25:46:958 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:25:46:987 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:25:46:987 [140605547460672] 2 hub: verify_login from ctrl 127.0.0.1/53558 Mar 30 15:25:46:987 [140605547460672] 1 hub: verify - [target] prid=controller cmd=probe_list ip=10.x.x.x perm=1 Mar 30 15:25:46:987 [140605547460672] 1 hub: verify - [source] id=ppm ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:47:223 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:47:603 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:25:47:644 [140603161552640] 5 hub: TSESS-P-4-8 [SecondaryHA] suspended Mar 30 15:25:47:754 [140603153159936] 5 hub: TSESS-P-8-5 [Secondary] suspended Mar 30 15:25:47:934 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:25:47:958 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:25:47:988 [140605547460672] 4 hub: medium timeout Mar 30 15:25:47:988 [140605547460672] 3 hub: next_check [1585596396] now [1585596347] Mar 30 15:25:48:108 [140603169945344] 5 hub: TSESS-P-7-8 [TunnelB] suspended Mar 30 15:25:48:698 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:25:49:026 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:51:682 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:25:51:682 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:25:51:682 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32854 Mar 30 15:25:51:682 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32854' delegated to TSESS-A-5 [140603715176192] Mar 30 15:25:51:682 [140603715176192] 3 hub: TSESS-A-5-36 [TunnelB] restarted (for local) Mar 30 15:25:51:687 [140603715176192] 5 hub: TSESS-A-5-36 [TunnelB] local 100 'get_info' Mar 30 15:25:51:687 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:51:687 [140603715176192] 1 hub: TSESS-A-5-36 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:25:51:698 [140603715176192] 1 hub: TSESS-A-5-36 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:25:51:698 [140603715176192] 3 hub: TSESS-A-5-36 [TunnelB] is paired with [P-298-36] Mar 30 15:25:51:698 [140603715176192] 1 hub: TSESS-A-5-36 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:25:51:698 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32856 Mar 30 15:25:51:698 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32856' delegated to TSESS-A-10 [140603706783488] Mar 30 15:25:51:698 [140603706783488] 3 hub: TSESS-A-10-26 [TunnelB] restarted (for local) Mar 30 15:25:51:703 [140603715176192] 5 hub: TSESS-A-5-36 [TunnelB] local 100 '_close' Mar 30 15:25:51:703 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:51:703 [140603715176192] 1 hub: TSESS-A-5-36 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:25:51:703 [140603715176192] 3 hub: TSESS-A-5-36 [TunnelB] local session closed, suspending Mar 30 15:25:51:703 [140603706783488] 5 hub: TSESS-A-10-26 [TunnelB] local 100 '_status' Mar 30 15:25:51:703 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:51:703 [140603706783488] 1 hub: TSESS-A-10-26 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:25:51:714 [140603706783488] 1 hub: TSESS-A-10-26 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:25:51:714 [140603706783488] 3 hub: TSESS-A-10-26 [TunnelB] is paired with [P-300-26] Mar 30 15:25:51:714 [140603706783488] 1 hub: TSESS-A-10-26 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:25:51:719 [140603706783488] 5 hub: TSESS-A-10-26 [TunnelB] local 100 '_close' Mar 30 15:25:51:719 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:25:51:719 [140603706783488] 1 hub: TSESS-A-10-26 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:25:51:719 [140603706783488] 3 hub: TSESS-A-10-26 [TunnelB] local session closed, suspending Mar 30 15:25:51:729 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:51:903 [140603715176192] 5 hub: TSESS-A-5-36 [TunnelB] suspended Mar 30 15:25:51:919 [140603706783488] 5 hub: TSESS-A-10-26 [TunnelB] suspended Mar 30 15:25:53:532 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:55:334 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:56:633 [140603203516160] 5 hub: TSESS-A-6-8 [Secondary] suspended Mar 30 15:25:56:665 [140603195123456] 5 hub: TSESS-A-3-8 [SecondaryHA] suspended Mar 30 15:25:56:958 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:25:56:987 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:25:57:137 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:25:57:603 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:25:57:644 [140603161552640] 5 hub: TSESS-P-4-8 [SecondaryHA] suspended Mar 30 15:25:57:689 [140605547460672] 4 hub: medium timeout Mar 30 15:25:57:689 [140605547460672] 3 hub: next_check [1585596396] now [1585596357] Mar 30 15:25:57:754 [140603153159936] 5 hub: TSESS-P-8-5 [Secondary] suspended Mar 30 15:25:57:934 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:25:57:958 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:25:58:109 [140603169945344] 5 hub: TSESS-P-7-8 [TunnelB] suspended Mar 30 15:25:58:698 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:25:59:840 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:01:643 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:01:725 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:26:01:725 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:26:01:726 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32864 Mar 30 15:26:01:726 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32864' delegated to TSESS-A-5 [140603715176192] Mar 30 15:26:01:726 [140603715176192] 3 hub: TSESS-A-5-37 [TunnelB] restarted (for local) Mar 30 15:26:01:731 [140603715176192] 5 hub: TSESS-A-5-37 [TunnelB] local 100 'get_info' Mar 30 15:26:01:731 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:01:731 [140603715176192] 1 hub: TSESS-A-5-37 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:26:01:741 [140603715176192] 1 hub: TSESS-A-5-37 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:26:01:741 [140603715176192] 3 hub: TSESS-A-5-37 [TunnelB] is paired with [P-298-37] Mar 30 15:26:01:741 [140603715176192] 1 hub: TSESS-A-5-37 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:26:01:741 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32866 Mar 30 15:26:01:742 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32866' delegated to TSESS-A-10 [140603706783488] Mar 30 15:26:01:742 [140603706783488] 3 hub: TSESS-A-10-27 [TunnelB] restarted (for local) Mar 30 15:26:01:746 [140603715176192] 5 hub: TSESS-A-5-37 [TunnelB] local 100 '_close' Mar 30 15:26:01:746 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:01:746 [140603715176192] 1 hub: TSESS-A-5-37 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:26:01:746 [140603715176192] 3 hub: TSESS-A-5-37 [TunnelB] local session closed, suspending Mar 30 15:26:01:747 [140603706783488] 5 hub: TSESS-A-10-27 [TunnelB] local 100 '_status' Mar 30 15:26:01:747 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:01:747 [140603706783488] 1 hub: TSESS-A-10-27 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:26:01:757 [140603706783488] 1 hub: TSESS-A-10-27 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:26:01:757 [140603706783488] 3 hub: TSESS-A-10-27 [TunnelB] is paired with [P-300-27] Mar 30 15:26:01:757 [140603706783488] 1 hub: TSESS-A-10-27 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:26:01:762 [140603706783488] 5 hub: TSESS-A-10-27 [TunnelB] local 100 '_close' Mar 30 15:26:01:762 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:01:762 [140603706783488] 1 hub: TSESS-A-10-27 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:26:01:762 [140603706783488] 3 hub: TSESS-A-10-27 [TunnelB] local session closed, suspending Mar 30 15:26:01:947 [140603715176192] 5 hub: TSESS-A-5-37 [TunnelB] suspended Mar 30 15:26:01:963 [140603706783488] 5 hub: TSESS-A-10-27 [TunnelB] suspended Mar 30 15:26:03:445 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:05:247 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:05:730 [140605547460672] 3 hub: check_session - session 0x20f9050 (127.0.0.1/48002) is a server Mar 30 15:26:05:730 [140605547460672] 2 hub: Profile - clayk 38 bytes (OK) Mar 30 15:26:06:634 [140603203516160] 5 hub: TSESS-A-6-8 [Secondary] suspended Mar 30 15:26:06:666 [140603195123456] 5 hub: TSESS-A-3-8 [SecondaryHA] suspended Mar 30 15:26:06:958 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:26:06:987 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:26:07:050 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:07:604 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:26:07:644 [140603161552640] 5 hub: TSESS-P-4-8 [SecondaryHA] suspended Mar 30 15:26:07:732 [140605547460672] 4 hub: medium timeout Mar 30 15:26:07:732 [140605547460672] 3 hub: next_check [1585596396] now [1585596367] Mar 30 15:26:07:754 [140603153159936] 5 hub: TSESS-P-8-5 [Secondary] suspended Mar 30 15:26:07:765 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:26:07:765 [140603731961600] 2 hub: CTRL SecondaryHA request 'hubup' passed to TSESS-P-4 [140603161552640] Mar 30 15:26:07:765 [140603161552640] 3 hub: TSESS-P-4-9 [SecondaryHA] restarted (for remote) Mar 30 15:26:07:765 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:36:07 Mar 30 15:26:07:765 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:26:07:766 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:26:07:766 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:26:07:766 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:26:07:766 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:26:07:766 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:26:07:766 [140603161552640] 4 hub: CONNECT: 0x7fe09c000d70(39) 127.0.0.1/53578->127.0.0.1/48002 Mar 30 15:26:07:766 [140603161552640] 3 hub: TSESS-P-4-9 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:26:07:766 [140603161552640] 1 hub: TSESS-P-4-9 [SecondaryHA] received 100 'hubup' (687 bytes) Mar 30 15:26:07:766 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53578 (s:392) Mar 30 15:26:07:766 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/53578] Mar 30 15:26:07:766 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:26:07:766 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:26:07:766 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/53578 Mar 30 15:26:07:766 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/53578 same proximity=0 status=0 secver=217 Mar 30 15:26:07:771 [140603161552640] 5 hub: TSESS-P-4-9 [SecondaryHA] local 200 'OK' Mar 30 15:26:07:771 [140603161552640] 1 hub: TSESS-P-4-9 [SecondaryHA] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:26:07:781 [140603161552640] 1 hub: TSESS-P-4-9 [SecondaryHA] ssl 100 '_close' (300 bytes) Mar 30 15:26:07:781 [140603161552640] 3 hub: TSESS-P-4-9 [SecondaryHA] is paired with [A-1948-9] Mar 30 15:26:07:781 [140603161552640] 1 hub: TSESS-P-4-9 [SecondaryHA] received 100 '_close' (296 bytes) Mar 30 15:26:07:786 [140603161552640] 1 hub: TSESS-P-4-9 [SecondaryHA] remote session closed Mar 30 15:26:07:786 [140603161552640] 3 hub: TSESS-P-4-9 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:26:07:786 [140603161552640] 5 hub: TSESS-P-4-9 [SecondaryHA] suspended Mar 30 15:26:07:880 [140603740354304] 5 hub: TSESS-P-8 not internal command - thread will handle. mtype=100; sfd:23 Mar 30 15:26:07:880 [140603740354304] 2 hub: CTRL Secondary request 'hubup' passed to TSESS-P-8 [140603153159936] Mar 30 15:26:07:880 [140603153159936] 3 hub: TSESS-P-8-6 [Secondary] restarted (for remote) Mar 30 15:26:07:880 [140603153159936] 2 hub: TSESS-P-8 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:36:07 Mar 30 15:26:07:880 [140603153159936] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:26:07:880 [140603153159936] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:26:07:880 [140603153159936] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:26:07:880 [140603153159936] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:26:07:880 [140603153159936] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:26:07:880 [140603153159936] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:26:07:880 [140603153159936] 4 hub: CONNECT: 0x7fe0a0000d90(39) 127.0.0.1/53580->127.0.0.1/48002 Mar 30 15:26:07:880 [140603153159936] 3 hub: TSESS-P-8-6 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:26:07:880 [140603153159936] 1 hub: TSESS-P-8-6 [Secondary] received 100 'hubup' (683 bytes) Mar 30 15:26:07:880 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53580 (s:388) Mar 30 15:26:07:880 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/53580] Mar 30 15:26:07:880 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:26:07:880 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:26:07:880 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/53580 Mar 30 15:26:07:880 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/53580 same proximity=0 status=0 secver=217 Mar 30 15:26:07:885 [140603153159936] 5 hub: TSESS-P-8-6 [Secondary] local 200 'OK' Mar 30 15:26:07:885 [140603153159936] 1 hub: TSESS-P-8-6 [Secondary] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:26:07:898 [140603153159936] 1 hub: TSESS-P-8-6 [Secondary] ssl 100 '_close' (296 bytes) Mar 30 15:26:07:898 [140603153159936] 3 hub: TSESS-P-8-6 [Secondary] is paired with [A-2121-6] Mar 30 15:26:07:898 [140603153159936] 1 hub: TSESS-P-8-6 [Secondary] received 100 '_close' (296 bytes) Mar 30 15:26:07:903 [140603153159936] 1 hub: TSESS-P-8-6 [Secondary] remote session closed Mar 30 15:26:07:904 [140603153159936] 3 hub: TSESS-P-8-6 [Secondary] exiting msg loop (remote close) Mar 30 15:26:07:904 [140603153159936] 5 hub: TSESS-P-8-6 [Secondary] suspended Mar 30 15:26:07:934 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:26:07:958 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:26:08:109 [140603169945344] 5 hub: TSESS-P-7-8 [TunnelB] suspended Mar 30 15:26:08:172 [140603723568896] 5 hub: TSESS-P-7 not internal command - thread will handle. mtype=100; sfd:22 Mar 30 15:26:08:172 [140603723568896] 2 hub: CTRL TunnelB request 'hubup' passed to TSESS-P-7 [140603169945344] Mar 30 15:26:08:172 [140603169945344] 3 hub: TSESS-P-7-9 [TunnelB] restarted (for remote) Mar 30 15:26:08:172 [140603169945344] 2 hub: TSESS-P-7 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:36:08 Mar 30 15:26:08:172 [140603169945344] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:26:08:172 [140603169945344] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:26:08:172 [140603169945344] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:26:08:172 [140603169945344] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:26:08:172 [140603169945344] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:26:08:172 [140603169945344] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:26:08:173 [140603169945344] 4 hub: CONNECT: 0x7fe0a8000d20(39) 127.0.0.1/53582->127.0.0.1/48002 Mar 30 15:26:08:173 [140603169945344] 3 hub: TSESS-P-7-9 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:26:08:173 [140603169945344] 1 hub: TSESS-P-7-9 [TunnelB] received 100 'hubup' (672 bytes) Mar 30 15:26:08:173 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53582 (s:378) Mar 30 15:26:08:173 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/53582] Mar 30 15:26:08:173 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:26:08:173 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:26:08:173 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/53582 Mar 30 15:26:08:173 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/53582 same proximity=0 status=0 secver=217 Mar 30 15:26:08:178 [140603169945344] 5 hub: TSESS-P-7-9 [TunnelB] local 200 'OK' Mar 30 15:26:08:178 [140603169945344] 1 hub: TSESS-P-7-9 [TunnelB] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:26:08:188 [140603169945344] 1 hub: TSESS-P-7-9 [TunnelB] ssl 100 '_close' (283 bytes) Mar 30 15:26:08:188 [140603169945344] 3 hub: TSESS-P-7-9 [TunnelB] is paired with [A-299-9] Mar 30 15:26:08:188 [140603169945344] 1 hub: TSESS-P-7-9 [TunnelB] received 100 '_close' (295 bytes) Mar 30 15:26:08:193 [140603169945344] 1 hub: TSESS-P-7-9 [TunnelB] remote session closed Mar 30 15:26:08:193 [140603169945344] 3 hub: TSESS-P-7-9 [TunnelB] exiting msg loop (remote close) Mar 30 15:26:08:193 [140603169945344] 5 hub: TSESS-P-7-9 [TunnelB] suspended Mar 30 15:26:08:698 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:26:09:753 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:11:556 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:11:768 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:26:11:768 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:26:11:769 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32878 Mar 30 15:26:11:769 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32878' delegated to TSESS-A-5 [140603715176192] Mar 30 15:26:11:769 [140603715176192] 3 hub: TSESS-A-5-38 [TunnelB] restarted (for local) Mar 30 15:26:11:774 [140603715176192] 5 hub: TSESS-A-5-38 [TunnelB] local 100 'get_info' Mar 30 15:26:11:774 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:11:774 [140603715176192] 1 hub: TSESS-A-5-38 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:26:11:784 [140603715176192] 1 hub: TSESS-A-5-38 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:26:11:784 [140603715176192] 3 hub: TSESS-A-5-38 [TunnelB] is paired with [P-298-38] Mar 30 15:26:11:784 [140603715176192] 1 hub: TSESS-A-5-38 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:26:11:785 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32880 Mar 30 15:26:11:785 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32880' delegated to TSESS-A-10 [140603706783488] Mar 30 15:26:11:785 [140603706783488] 3 hub: TSESS-A-10-28 [TunnelB] restarted (for local) Mar 30 15:26:11:790 [140603715176192] 5 hub: TSESS-A-5-38 [TunnelB] local 100 '_close' Mar 30 15:26:11:790 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:11:790 [140603715176192] 1 hub: TSESS-A-5-38 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:26:11:790 [140603715176192] 3 hub: TSESS-A-5-38 [TunnelB] local session closed, suspending Mar 30 15:26:11:790 [140603706783488] 5 hub: TSESS-A-10-28 [TunnelB] local 100 '_status' Mar 30 15:26:11:790 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:11:790 [140603706783488] 1 hub: TSESS-A-10-28 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:26:11:800 [140603706783488] 1 hub: TSESS-A-10-28 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:26:11:800 [140603706783488] 3 hub: TSESS-A-10-28 [TunnelB] is paired with [P-300-28] Mar 30 15:26:11:800 [140603706783488] 1 hub: TSESS-A-10-28 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:26:11:805 [140603706783488] 5 hub: TSESS-A-10-28 [TunnelB] local 100 '_close' Mar 30 15:26:11:805 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:11:805 [140603706783488] 1 hub: TSESS-A-10-28 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:26:11:805 [140603706783488] 3 hub: TSESS-A-10-28 [TunnelB] local session closed, suspending Mar 30 15:26:11:990 [140603715176192] 5 hub: TSESS-A-5-38 [TunnelB] suspended Mar 30 15:26:12:006 [140603706783488] 5 hub: TSESS-A-10-28 [TunnelB] suspended Mar 30 15:26:13:358 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:15:161 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:16:497 [140605283890944] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:26:16:498 [140605283890944] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:26:16:498 [140605283890944] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:26:16:498 [140605283890944] 1 hub: PERF: hubup_check_interval: 0 sec hubhash_size: 18 Mar 30 15:26:16:498 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:498 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:498 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:498 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:498 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:498 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:498 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:26:16:498 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:498 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:498 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:498 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:498 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:498 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:498 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:498 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:498 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:26:16:498 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:26:16:498 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:26:16:498 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/47256->127.0.0.1/48007 Mar 30 15:26:16:498 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/47256 Mar 30 15:26:16:498 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/47256' delegated to TSESS-A-6 [140603203516160] Mar 30 15:26:16:498 [140603203516160] 3 hub: TSESS-A-6-9 [Secondary] restarted (for local) Mar 30 15:26:16:503 [140603203516160] 5 hub: TSESS-A-6-9 [Secondary] local 100 'hubup' Mar 30 15:26:16:503 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:16:504 [140603203516160] 1 hub: TSESS-A-6-9 [Secondary] sent 100 'hubup' (686 bytes, 0 ms) Mar 30 15:26:16:514 [140603203516160] 1 hub: TSESS-A-6-9 [Secondary] ssl 200 '-' (453 bytes) Mar 30 15:26:16:514 [140603203516160] 3 hub: TSESS-A-6-9 [Secondary] is paired with [P-2120-9] Mar 30 15:26:16:514 [140603203516160] 1 hub: TSESS-A-6-9 [Secondary] received 200 '-' (461 bytes) Mar 30 15:26:16:514 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:26:16:514 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:26:16:514 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:26:16:514 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:26:16:514 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary same proximity=0 status=0 secver=217 Mar 30 15:26:16:514 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:514 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:514 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:514 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:514 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:514 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:514 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:26:16:514 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:514 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:514 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:514 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:514 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:514 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:514 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:514 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:514 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:26:16:514 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:26:16:514 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:26:16:514 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/32888->127.0.0.1/48009 Mar 30 15:26:16:514 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32888 Mar 30 15:26:16:514 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32888' delegated to TSESS-A-5 [140603715176192] Mar 30 15:26:16:514 [140603715176192] 3 hub: TSESS-A-5-39 [TunnelB] restarted (for local) Mar 30 15:26:16:519 [140603203516160] 5 hub: TSESS-A-6-9 [Secondary] local 100 '_close' Mar 30 15:26:16:519 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:16:519 [140603203516160] 1 hub: TSESS-A-6-9 [Secondary] sent 100 '_close' (293 bytes, 0 ms) Mar 30 15:26:16:519 [140603203516160] 3 hub: TSESS-A-6-9 [Secondary] local session closed, suspending Mar 30 15:26:16:520 [140603715176192] 5 hub: TSESS-A-5-39 [TunnelB] local 100 'hubup' Mar 30 15:26:16:520 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:16:520 [140603715176192] 1 hub: TSESS-A-5-39 [TunnelB] sent 100 'hubup' (685 bytes, 0 ms) Mar 30 15:26:16:530 [140603715176192] 1 hub: TSESS-A-5-39 [TunnelB] ssl 200 '-' (443 bytes) Mar 30 15:26:16:530 [140603715176192] 3 hub: TSESS-A-5-39 [TunnelB] is paired with [P-298-39] Mar 30 15:26:16:530 [140603715176192] 1 hub: TSESS-A-5-39 [TunnelB] received 200 '-' (451 bytes) Mar 30 15:26:16:530 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:26:16:530 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:26:16:530 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:26:16:530 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:26:16:530 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB same proximity=0 status=0 secver=217 Mar 30 15:26:16:530 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:530 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:530 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:530 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:530 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:530 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:530 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:26:16:530 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:530 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:530 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:530 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:530 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:530 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:530 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:530 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:16:530 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:26:16:530 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:26:16:530 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:26:16:530 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/40308->127.0.0.1/48008 Mar 30 15:26:16:530 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/40308 Mar 30 15:26:16:530 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/40308' delegated to TSESS-A-3 [140603195123456] Mar 30 15:26:16:530 [140603195123456] 3 hub: TSESS-A-3-9 [SecondaryHA] restarted (for local) Mar 30 15:26:16:535 [140603715176192] 5 hub: TSESS-A-5-39 [TunnelB] local 100 '_close' Mar 30 15:26:16:535 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:16:535 [140603715176192] 1 hub: TSESS-A-5-39 [TunnelB] sent 100 '_close' (292 bytes, 0 ms) Mar 30 15:26:16:535 [140603715176192] 3 hub: TSESS-A-5-39 [TunnelB] local session closed, suspending Mar 30 15:26:16:536 [140603195123456] 5 hub: TSESS-A-3-9 [SecondaryHA] local 100 'hubup' Mar 30 15:26:16:536 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:16:536 [140603195123456] 1 hub: TSESS-A-3-9 [SecondaryHA] sent 100 'hubup' (688 bytes, 0 ms) Mar 30 15:26:16:546 [140603195123456] 1 hub: TSESS-A-3-9 [SecondaryHA] ssl 200 '-' (457 bytes) Mar 30 15:26:16:546 [140603195123456] 3 hub: TSESS-A-3-9 [SecondaryHA] is paired with [P-1947-9] Mar 30 15:26:16:546 [140603195123456] 1 hub: TSESS-A-3-9 [SecondaryHA] received 200 '-' (465 bytes) Mar 30 15:26:16:546 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:26:16:546 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:26:16:546 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:26:16:546 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:26:16:546 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA same proximity=0 status=0 secver=217 Mar 30 15:26:16:546 [140605283890944] 5 hub: mark_hubup_list_updated Mar 30 15:26:16:546 [140605283890944] 5 hub: hublist changed: /DOMAIN/Primary 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:16:546 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:16:546 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:16:546 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:16:546 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:16:546 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:16:546 [140605283890944] 5 hub: hublist changed: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:26:16:546 [140605283890944] 5 hub: hublist changed: /DOMAIN/PrimaryHA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:16:546 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:16:546 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:16:546 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:16:546 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:16:546 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:16:546 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:16:546 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:16:546 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:26:16:546 [140605283890944] 5 hub: hublist changed: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:26:16:546 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelBHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:26:16:546 [140605283890944] 5 hub: mark_hubup_list_updated end Mar 30 15:26:16:546 [140605283890944] 1 hub: PERF: hubup_check_for_updates: 0 sec hubhash_size: 18 Mar 30 15:26:16:546 [140605283890944] 1 hub: PERF: hubi_run_command: 0 sec hubhash_size: 18 Mar 30 15:26:16:546 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:26:16:547 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:26:16:547 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:26:16:548 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:26:16:548 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:26:16:548 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:26:16:548 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:26:16:549 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:26:16:549 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:26:16:550 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:26:16:550 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:26:16:550 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:26:16:550 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:26:16:551 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:26:16:551 [140603195123456] 5 hub: TSESS-A-3-9 [SecondaryHA] local 100 '_close' Mar 30 15:26:16:551 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:16:551 [140603195123456] 1 hub: TSESS-A-3-9 [SecondaryHA] sent 100 '_close' (295 bytes, 0 ms) Mar 30 15:26:16:551 [140603195123456] 3 hub: TSESS-A-3-9 [SecondaryHA] local session closed, suspending Mar 30 15:26:16:552 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:26:16:552 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:26:16:553 [140605283890944] 1 hub: PERF: hubi_check_security: 0 sec hubhash_size: 18 Mar 30 15:26:16:719 [140603203516160] 5 hub: TSESS-A-6-9 [Secondary] suspended Mar 30 15:26:16:735 [140603715176192] 5 hub: TSESS-A-5-39 [TunnelB] suspended Mar 30 15:26:16:752 [140603195123456] 5 hub: TSESS-A-3-9 [SecondaryHA] suspended Mar 30 15:26:16:959 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:26:16:987 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:26:17:604 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:26:17:775 [140605547460672] 4 hub: medium timeout Mar 30 15:26:17:775 [140605547460672] 3 hub: next_check [1585596396] now [1585596377] Mar 30 15:26:17:787 [140603161552640] 5 hub: TSESS-P-4-9 [SecondaryHA] suspended Mar 30 15:26:17:864 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:17:904 [140603153159936] 5 hub: TSESS-P-8-6 [Secondary] suspended Mar 30 15:26:17:934 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:26:17:958 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:26:18:193 [140603169945344] 5 hub: TSESS-P-7-9 [TunnelB] suspended Mar 30 15:26:18:698 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:26:19:666 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:21:469 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:21:811 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:26:21:812 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:26:21:812 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32896 Mar 30 15:26:21:812 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32896' delegated to TSESS-A-5 [140603715176192] Mar 30 15:26:21:812 [140603715176192] 3 hub: TSESS-A-5-40 [TunnelB] restarted (for local) Mar 30 15:26:21:817 [140603715176192] 5 hub: TSESS-A-5-40 [TunnelB] local 100 'get_info' Mar 30 15:26:21:817 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:21:817 [140603715176192] 1 hub: TSESS-A-5-40 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:26:21:827 [140603715176192] 1 hub: TSESS-A-5-40 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:26:21:827 [140603715176192] 3 hub: TSESS-A-5-40 [TunnelB] is paired with [P-298-40] Mar 30 15:26:21:827 [140603715176192] 1 hub: TSESS-A-5-40 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:26:21:828 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32898 Mar 30 15:26:21:828 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32898' delegated to TSESS-A-10 [140603706783488] Mar 30 15:26:21:828 [140603706783488] 3 hub: TSESS-A-10-29 [TunnelB] restarted (for local) Mar 30 15:26:21:833 [140603715176192] 5 hub: TSESS-A-5-40 [TunnelB] local 100 '_close' Mar 30 15:26:21:833 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:21:833 [140603715176192] 1 hub: TSESS-A-5-40 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:26:21:833 [140603715176192] 3 hub: TSESS-A-5-40 [TunnelB] local session closed, suspending Mar 30 15:26:21:833 [140603706783488] 5 hub: TSESS-A-10-29 [TunnelB] local 100 '_status' Mar 30 15:26:21:833 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:21:833 [140603706783488] 1 hub: TSESS-A-10-29 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:26:21:843 [140603706783488] 1 hub: TSESS-A-10-29 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:26:21:843 [140603706783488] 3 hub: TSESS-A-10-29 [TunnelB] is paired with [P-300-29] Mar 30 15:26:21:843 [140603706783488] 1 hub: TSESS-A-10-29 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:26:21:848 [140603706783488] 5 hub: TSESS-A-10-29 [TunnelB] local 100 '_close' Mar 30 15:26:21:848 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:21:848 [140603706783488] 1 hub: TSESS-A-10-29 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:26:21:848 [140603706783488] 3 hub: TSESS-A-10-29 [TunnelB] local session closed, suspending Mar 30 15:26:22:033 [140603715176192] 5 hub: TSESS-A-5-40 [TunnelB] suspended Mar 30 15:26:22:049 [140603706783488] 5 hub: TSESS-A-10-29 [TunnelB] suspended Mar 30 15:26:23:271 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:25:074 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:26:720 [140603203516160] 5 hub: TSESS-A-6-9 [Secondary] suspended Mar 30 15:26:26:752 [140603195123456] 5 hub: TSESS-A-3-9 [SecondaryHA] suspended Mar 30 15:26:26:959 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:26:26:987 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:26:27:604 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:26:27:777 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:27:787 [140603161552640] 5 hub: TSESS-P-4-9 [SecondaryHA] suspended Mar 30 15:26:27:818 [140605547460672] 4 hub: medium timeout Mar 30 15:26:27:818 [140605547460672] 3 hub: next_check [1585596396] now [1585596387] Mar 30 15:26:27:904 [140603153159936] 5 hub: TSESS-P-8-6 [Secondary] suspended Mar 30 15:26:27:935 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:26:27:958 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:26:28:193 [140603169945344] 5 hub: TSESS-P-7-9 [TunnelB] suspended Mar 30 15:26:28:698 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:26:29:580 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:31:382 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:31:852 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:26:31:852 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:26:31:852 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32904 Mar 30 15:26:31:852 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32904' delegated to TSESS-A-5 [140603715176192] Mar 30 15:26:31:852 [140603715176192] 3 hub: TSESS-A-5-41 [TunnelB] restarted (for local) Mar 30 15:26:31:857 [140603715176192] 5 hub: TSESS-A-5-41 [TunnelB] local 100 'get_info' Mar 30 15:26:31:857 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:31:857 [140603715176192] 1 hub: TSESS-A-5-41 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:26:31:868 [140603715176192] 1 hub: TSESS-A-5-41 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:26:31:868 [140603715176192] 3 hub: TSESS-A-5-41 [TunnelB] is paired with [P-298-41] Mar 30 15:26:31:868 [140603715176192] 1 hub: TSESS-A-5-41 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:26:31:868 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32906 Mar 30 15:26:31:868 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32906' delegated to TSESS-A-10 [140603706783488] Mar 30 15:26:31:868 [140603706783488] 3 hub: TSESS-A-10-30 [TunnelB] restarted (for local) Mar 30 15:26:31:873 [140603715176192] 5 hub: TSESS-A-5-41 [TunnelB] local 100 '_close' Mar 30 15:26:31:873 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:31:873 [140603715176192] 1 hub: TSESS-A-5-41 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:26:31:873 [140603715176192] 3 hub: TSESS-A-5-41 [TunnelB] local session closed, suspending Mar 30 15:26:31:873 [140603706783488] 5 hub: TSESS-A-10-30 [TunnelB] local 100 '_status' Mar 30 15:26:31:873 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:31:873 [140603706783488] 1 hub: TSESS-A-10-30 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:26:31:883 [140603706783488] 1 hub: TSESS-A-10-30 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:26:31:883 [140603706783488] 3 hub: TSESS-A-10-30 [TunnelB] is paired with [P-300-30] Mar 30 15:26:31:883 [140603706783488] 1 hub: TSESS-A-10-30 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:26:31:889 [140603706783488] 5 hub: TSESS-A-10-30 [TunnelB] local 100 '_close' Mar 30 15:26:31:889 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:31:889 [140603706783488] 1 hub: TSESS-A-10-30 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:26:31:889 [140603706783488] 3 hub: TSESS-A-10-30 [TunnelB] local session closed, suspending Mar 30 15:26:32:073 [140603715176192] 5 hub: TSESS-A-5-41 [TunnelB] suspended Mar 30 15:26:32:089 [140603706783488] 5 hub: TSESS-A-10-30 [TunnelB] suspended Mar 30 15:26:33:185 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:34:023 [140603723568896] 3 hub: CTRL TunnelB updating address info for (10.x.x.x:48003) [attempt 1 of 10] Mar 30 15:26:34:024 [140603731961600] 3 hub: CTRL SecondaryHA updating address info for (10.x.x.x:48003) [attempt 1 of 10] Mar 30 15:26:34:035 [140603740354304] 3 hub: CTRL Secondary updating address info for (10.x.x.x:48003) [attempt 1 of 10] Mar 30 15:26:34:985 [140604042360576] 5 hub: There are 0 objects waiting on 8 threads Mar 30 15:26:35:856 [140605547460672] 3 hub: check_session - session 0x20f9050 (127.0.0.1/48002) is a server Mar 30 15:26:35:856 [140605547460672] 2 hub: Profile - clayk 38 bytes (OK) Mar 30 15:26:35:888 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:36:720 [140603203516160] 5 hub: TSESS-A-6-9 [Secondary] suspended Mar 30 15:26:36:752 [140603195123456] 5 hub: TSESS-A-3-9 [SecondaryHA] suspended Mar 30 15:26:36:858 [140605007062784] 5 hub: Reclaiming post session [0x7fe114001fd0] from '127.0.0.1/42140' Mar 30 15:26:36:959 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:26:36:987 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:26:37:081 [140603723568896] 3 hub: CTRL TunnelB heartbeat sent on TSESS-A-17; heartbeat: 300; state[0x24] Mar 30 15:26:37:082 [140603723568896] 3 hub: CTRL TunnelB heartbeat sent on TSESS-P-18; heartbeat: 300; state[0x24] Mar 30 15:26:37:082 [140603731961600] 3 hub: CTRL SecondaryHA heartbeat sent on TSESS-A-15; heartbeat: 300; state[0x24] Mar 30 15:26:37:083 [140603731961600] 3 hub: CTRL SecondaryHA heartbeat sent on TSESS-P-16; heartbeat: 300; state[0x24] Mar 30 15:26:37:094 [140603740354304] 3 hub: CTRL Secondary heartbeat sent on TSESS-A-19; heartbeat: 300; state[0x24] Mar 30 15:26:37:604 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:26:37:691 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:37:787 [140603161552640] 5 hub: TSESS-P-4-9 [SecondaryHA] suspended Mar 30 15:26:37:858 [140605547460672] 4 hub: medium timeout Mar 30 15:26:37:858 [140605547460672] 3 hub: next_check [1585596396] now [1585596397] Mar 30 15:26:37:859 [140605547460672] 4 hub: CONNECT: 0x2140d10(36) 127.0.0.1/42154->127.0.0.1/48001 Mar 30 15:26:37:859 [140604965099264] 5 hub: (Metric Plugin): cached QOS_DEFINITION hub QOS_POWER_STATE (State State) Mar 30 15:26:37:859 [140604965099264] 5 hub: (Metric Plugin): Processing pipeline finished, 1 messages output Mar 30 15:26:37:859 [140604965099264] 3 hub: nimSession:0x7fe114001fd0:11:127.0.0.1/42154:127.0.0.1/42154 - needs cross-thread locking Mar 30 15:26:37:859 [140605547460672] 2 hub: robotname= ROBOT_NAME ip= 10.x.x.x is_passive= 0 Mar 30 15:26:37:859 [140605547460672] 5 hub: ciOpen - cache path: /opt/nimsoft/niscache Mar 30 15:26:37:859 [140605547460672] 1 hub: ciGetDeviceIdentifiers - host=ROBOT_NAME ip=10.x.x.x probe= [D06F8A5F560C51929E2741716FF79808B] Mar 30 15:26:37:859 [140605547460672] 2 hub: ciSave - saving DEV [D06F8A5F560C51929E2741716FF79808B] Mar 30 15:26:37:860 [140605547460672] 2 hub: ciSave - saving CI [CD09FC907406F46129BCA027B1BD0F82D] Mar 30 15:26:37:860 [140605547460672] 2 hub: ciSaveMetric - saving MET [MEA0832F44CF5433788AB853B2E778B4F] 10.2:10 Mar 30 15:26:37:860 [140605547460672] 4 hub: CONNECT: 0x2140d10(36) 127.0.0.1/42156->127.0.0.1/48001 Mar 30 15:26:37:860 [140605547460672] 5 hub: ciSessionPostMessage - adding prid [hub] Mar 30 15:26:37:860 [140604604409600] 5 hub: (Metric Plugin): hub is not a configured probe, skipping Mar 30 15:26:37:860 [140604604409600] 5 hub: (Metric Plugin): Processing pipeline finished, 1 messages output Mar 30 15:26:37:860 [140604604409600] 3 hub: nimSession:0x7fe114001fd0:11:127.0.0.1/42156:127.0.0.1/42156 - needs cross-thread locking Mar 30 15:26:37:861 [140605547460672] 4 hub: ciClose - [CD09FC907406F46129BCA027B1BD0F82D] Mar 30 15:26:37:901 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:26:37:901 [140603731961600] 2 hub: CTRL SecondaryHA request 'hubup' passed to TSESS-P-4 [140603161552640] Mar 30 15:26:37:901 [140603161552640] 3 hub: TSESS-P-4-10 [SecondaryHA] restarted (for remote) Mar 30 15:26:37:901 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:36:37 Mar 30 15:26:37:901 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:26:37:901 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:26:37:901 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:26:37:902 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:26:37:902 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:26:37:902 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:26:37:902 [140603161552640] 4 hub: CONNECT: 0x7fe09c000d70(39) 127.0.0.1/53636->127.0.0.1/48002 Mar 30 15:26:37:902 [140603161552640] 3 hub: TSESS-P-4-10 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:26:37:902 [140603161552640] 1 hub: TSESS-P-4-10 [SecondaryHA] received 100 'hubup' (689 bytes) Mar 30 15:26:37:902 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53636 (s:392) Mar 30 15:26:37:902 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/53636] Mar 30 15:26:37:902 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:26:37:902 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:26:37:902 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/53636 Mar 30 15:26:37:902 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/53636 same proximity=0 status=0 secver=217 Mar 30 15:26:37:904 [140603153159936] 5 hub: TSESS-P-8-6 [Secondary] suspended Mar 30 15:26:37:907 [140603161552640] 5 hub: TSESS-P-4-10 [SecondaryHA] local 200 'OK' Mar 30 15:26:37:907 [140603161552640] 1 hub: TSESS-P-4-10 [SecondaryHA] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:26:37:917 [140603161552640] 1 hub: TSESS-P-4-10 [SecondaryHA] ssl 100 '_close' (302 bytes) Mar 30 15:26:37:917 [140603161552640] 3 hub: TSESS-P-4-10 [SecondaryHA] is paired with [A-1948-10] Mar 30 15:26:37:917 [140603161552640] 1 hub: TSESS-P-4-10 [SecondaryHA] received 100 '_close' (298 bytes) Mar 30 15:26:37:922 [140603161552640] 1 hub: TSESS-P-4-10 [SecondaryHA] remote session closed Mar 30 15:26:37:922 [140603161552640] 3 hub: TSESS-P-4-10 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:26:37:922 [140603161552640] 5 hub: TSESS-P-4-10 [SecondaryHA] suspended Mar 30 15:26:37:935 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:26:37:959 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:26:38:029 [140603740354304] 5 hub: TSESS-P-8 not internal command - thread will handle. mtype=100; sfd:23 Mar 30 15:26:38:029 [140603740354304] 2 hub: CTRL Secondary request 'hubup' passed to TSESS-P-8 [140603153159936] Mar 30 15:26:38:029 [140603153159936] 3 hub: TSESS-P-8-7 [Secondary] restarted (for remote) Mar 30 15:26:38:029 [140603153159936] 2 hub: TSESS-P-8 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:36:37 Mar 30 15:26:38:029 [140603153159936] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:26:38:029 [140603153159936] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:26:38:029 [140603153159936] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:26:38:029 [140603153159936] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:26:38:029 [140603153159936] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:26:38:029 [140603153159936] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:26:38:030 [140603153159936] 4 hub: CONNECT: 0x7fe0a0000d90(39) 127.0.0.1/53638->127.0.0.1/48002 Mar 30 15:26:38:030 [140603153159936] 3 hub: TSESS-P-8-7 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:26:38:030 [140603153159936] 1 hub: TSESS-P-8-7 [Secondary] received 100 'hubup' (683 bytes) Mar 30 15:26:38:030 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53638 (s:388) Mar 30 15:26:38:030 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/53638] Mar 30 15:26:38:030 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:26:38:030 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:26:38:030 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/53638 Mar 30 15:26:38:030 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/53638 same proximity=0 status=0 secver=217 Mar 30 15:26:38:035 [140603153159936] 5 hub: TSESS-P-8-7 [Secondary] local 200 'OK' Mar 30 15:26:38:035 [140603153159936] 1 hub: TSESS-P-8-7 [Secondary] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:26:38:045 [140603153159936] 1 hub: TSESS-P-8-7 [Secondary] ssl 100 '_close' (296 bytes) Mar 30 15:26:38:045 [140603153159936] 3 hub: TSESS-P-8-7 [Secondary] is paired with [A-2121-7] Mar 30 15:26:38:045 [140603153159936] 1 hub: TSESS-P-8-7 [Secondary] received 100 '_close' (296 bytes) Mar 30 15:26:38:050 [140603153159936] 1 hub: TSESS-P-8-7 [Secondary] remote session closed Mar 30 15:26:38:050 [140603153159936] 3 hub: TSESS-P-8-7 [Secondary] exiting msg loop (remote close) Mar 30 15:26:38:050 [140603153159936] 5 hub: TSESS-P-8-7 [Secondary] suspended Mar 30 15:26:38:194 [140603169945344] 5 hub: TSESS-P-7-9 [TunnelB] suspended Mar 30 15:26:38:257 [140603723568896] 5 hub: TSESS-P-7 not internal command - thread will handle. mtype=100; sfd:22 Mar 30 15:26:38:257 [140603723568896] 2 hub: CTRL TunnelB request 'hubup' passed to TSESS-P-7 [140603169945344] Mar 30 15:26:38:257 [140603169945344] 3 hub: TSESS-P-7-10 [TunnelB] restarted (for remote) Mar 30 15:26:38:257 [140603169945344] 2 hub: TSESS-P-7 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:36:38 Mar 30 15:26:38:257 [140603169945344] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:26:38:257 [140603169945344] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:26:38:257 [140603169945344] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:26:38:257 [140603169945344] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:26:38:257 [140603169945344] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:26:38:257 [140603169945344] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:26:38:257 [140603169945344] 4 hub: CONNECT: 0x7fe0a8000d20(39) 127.0.0.1/53640->127.0.0.1/48002 Mar 30 15:26:38:257 [140603169945344] 3 hub: TSESS-P-7-10 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:26:38:257 [140603169945344] 1 hub: TSESS-P-7-10 [TunnelB] received 100 'hubup' (673 bytes) Mar 30 15:26:38:258 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53640 (s:378) Mar 30 15:26:38:258 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/53640] Mar 30 15:26:38:258 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:26:38:258 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:26:38:258 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/53640 Mar 30 15:26:38:258 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/53640 same proximity=0 status=0 secver=217 Mar 30 15:26:38:263 [140603169945344] 5 hub: TSESS-P-7-10 [TunnelB] local 200 'OK' Mar 30 15:26:38:263 [140603169945344] 1 hub: TSESS-P-7-10 [TunnelB] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:26:38:273 [140603169945344] 1 hub: TSESS-P-7-10 [TunnelB] ssl 100 '_close' (284 bytes) Mar 30 15:26:38:273 [140603169945344] 3 hub: TSESS-P-7-10 [TunnelB] is paired with [A-299-10] Mar 30 15:26:38:273 [140603169945344] 1 hub: TSESS-P-7-10 [TunnelB] received 100 '_close' (296 bytes) Mar 30 15:26:38:278 [140603169945344] 1 hub: TSESS-P-7-10 [TunnelB] remote session closed Mar 30 15:26:38:278 [140603169945344] 3 hub: TSESS-P-7-10 [TunnelB] exiting msg loop (remote close) Mar 30 15:26:38:278 [140603169945344] 5 hub: TSESS-P-7-10 [TunnelB] suspended Mar 30 15:26:38:699 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:26:39:493 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:40:864 [140605015455488] 5 hub: (Metric Plugin): cdm is not a configured probe, skipping Mar 30 15:26:40:864 [140605015455488] 5 hub: (Metric Plugin): Processing pipeline finished, 1 messages output Mar 30 15:26:40:864 [140605015455488] 3 hub: nimSession:0x7fe114001fd0:36:127.0.0.1/42164:127.0.0.1/42164 - needs cross-thread locking Mar 30 15:26:40:864 [140604579231488] 5 hub: (Metric Plugin): cdm is not a configured probe, skipping Mar 30 15:26:40:864 [140604579231488] 5 hub: (Metric Plugin): Processing pipeline finished, 1 messages output Mar 30 15:26:40:864 [140604579231488] 3 hub: nimSession:0x7fe114004b10:39:127.0.0.1/42166:127.0.0.1/42166 - needs cross-thread locking Mar 30 15:26:40:865 [140604998670080] 5 hub: (Metric Plugin): cdm is not a configured probe, skipping Mar 30 15:26:40:865 [140604998670080] 5 hub: (Metric Plugin): Processing pipeline finished, 1 messages output Mar 30 15:26:40:865 [140604998670080] 3 hub: nimSession:0x7fe114001fd0:36:127.0.0.1/42168:127.0.0.1/42168 - needs cross-thread locking Mar 30 15:26:41:296 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:41:867 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:26:41:867 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:26:41:868 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32944 Mar 30 15:26:41:868 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32944' delegated to TSESS-A-5 [140603715176192] Mar 30 15:26:41:868 [140603715176192] 3 hub: TSESS-A-5-42 [TunnelB] restarted (for local) Mar 30 15:26:41:873 [140603715176192] 5 hub: TSESS-A-5-42 [TunnelB] local 100 'get_info' Mar 30 15:26:41:873 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:41:873 [140603715176192] 1 hub: TSESS-A-5-42 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:26:41:883 [140603715176192] 1 hub: TSESS-A-5-42 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:26:41:883 [140603715176192] 3 hub: TSESS-A-5-42 [TunnelB] is paired with [P-298-42] Mar 30 15:26:41:883 [140603715176192] 1 hub: TSESS-A-5-42 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:26:41:883 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32946 Mar 30 15:26:41:884 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32946' delegated to TSESS-A-10 [140603706783488] Mar 30 15:26:41:884 [140603706783488] 3 hub: TSESS-A-10-31 [TunnelB] restarted (for local) Mar 30 15:26:41:888 [140603715176192] 5 hub: TSESS-A-5-42 [TunnelB] local 100 '_close' Mar 30 15:26:41:888 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:41:888 [140603715176192] 1 hub: TSESS-A-5-42 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:26:41:888 [140603715176192] 3 hub: TSESS-A-5-42 [TunnelB] local session closed, suspending Mar 30 15:26:41:889 [140603706783488] 5 hub: TSESS-A-10-31 [TunnelB] local 100 '_status' Mar 30 15:26:41:889 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:41:889 [140603706783488] 1 hub: TSESS-A-10-31 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:26:41:899 [140603706783488] 1 hub: TSESS-A-10-31 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:26:41:899 [140603706783488] 3 hub: TSESS-A-10-31 [TunnelB] is paired with [P-300-31] Mar 30 15:26:41:899 [140603706783488] 1 hub: TSESS-A-10-31 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:26:41:904 [140603706783488] 5 hub: TSESS-A-10-31 [TunnelB] local 100 '_close' Mar 30 15:26:41:904 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:41:904 [140603706783488] 1 hub: TSESS-A-10-31 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:26:41:904 [140603706783488] 3 hub: TSESS-A-10-31 [TunnelB] local session closed, suspending Mar 30 15:26:42:089 [140603715176192] 5 hub: TSESS-A-5-42 [TunnelB] suspended Mar 30 15:26:42:104 [140603706783488] 5 hub: TSESS-A-10-31 [TunnelB] suspended Mar 30 15:26:43:098 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:45:802 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:46:584 [140605283890944] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:26:46:584 [140605283890944] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:26:46:584 [140605283890944] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:26:46:584 [140605283890944] 1 hub: PERF: hubup_check_interval: 0 sec hubhash_size: 18 Mar 30 15:26:46:584 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:584 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:584 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:584 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:584 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:584 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:584 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:26:46:584 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:584 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:584 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:584 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:584 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:584 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:584 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:584 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:584 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:26:46:584 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:26:46:584 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:26:46:584 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/47318->127.0.0.1/48007 Mar 30 15:26:46:584 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/47318 Mar 30 15:26:46:584 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/47318' delegated to TSESS-A-6 [140603203516160] Mar 30 15:26:46:584 [140603203516160] 3 hub: TSESS-A-6-10 [Secondary] restarted (for local) Mar 30 15:26:46:590 [140603203516160] 5 hub: TSESS-A-6-10 [Secondary] local 100 'hubup' Mar 30 15:26:46:590 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:46:590 [140603203516160] 1 hub: TSESS-A-6-10 [Secondary] sent 100 'hubup' (687 bytes, 0 ms) Mar 30 15:26:46:600 [140603203516160] 1 hub: TSESS-A-6-10 [Secondary] ssl 200 '-' (455 bytes) Mar 30 15:26:46:600 [140603203516160] 3 hub: TSESS-A-6-10 [Secondary] is paired with [P-2120-10] Mar 30 15:26:46:600 [140603203516160] 1 hub: TSESS-A-6-10 [Secondary] received 200 '-' (463 bytes) Mar 30 15:26:46:600 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:26:46:600 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:26:46:600 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:26:46:600 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:26:46:600 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary same proximity=0 status=0 secver=217 Mar 30 15:26:46:600 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:600 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:600 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:600 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:600 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:600 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:600 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:26:46:600 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:600 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:600 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:600 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:600 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:600 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:600 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:600 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:600 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:26:46:600 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:26:46:600 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:26:46:600 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/32950->127.0.0.1/48009 Mar 30 15:26:46:600 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32950 Mar 30 15:26:46:600 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32950' delegated to TSESS-A-5 [140603715176192] Mar 30 15:26:46:600 [140603715176192] 3 hub: TSESS-A-5-43 [TunnelB] restarted (for local) Mar 30 15:26:46:605 [140603203516160] 5 hub: TSESS-A-6-10 [Secondary] local 100 '_close' Mar 30 15:26:46:605 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:46:605 [140603203516160] 1 hub: TSESS-A-6-10 [Secondary] sent 100 '_close' (294 bytes, 0 ms) Mar 30 15:26:46:605 [140603203516160] 3 hub: TSESS-A-6-10 [Secondary] local session closed, suspending Mar 30 15:26:46:605 [140603715176192] 5 hub: TSESS-A-5-43 [TunnelB] local 100 'hubup' Mar 30 15:26:46:606 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:46:606 [140603715176192] 1 hub: TSESS-A-5-43 [TunnelB] sent 100 'hubup' (685 bytes, 0 ms) Mar 30 15:26:46:616 [140603715176192] 1 hub: TSESS-A-5-43 [TunnelB] ssl 200 '-' (443 bytes) Mar 30 15:26:46:616 [140603715176192] 3 hub: TSESS-A-5-43 [TunnelB] is paired with [P-298-43] Mar 30 15:26:46:616 [140603715176192] 1 hub: TSESS-A-5-43 [TunnelB] received 200 '-' (451 bytes) Mar 30 15:26:46:616 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:26:46:616 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:26:46:616 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:26:46:616 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:26:46:616 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB same proximity=0 status=0 secver=217 Mar 30 15:26:46:616 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:616 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:616 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:616 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:616 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:616 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:616 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:26:46:616 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:616 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:616 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:616 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:616 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:616 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:616 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:616 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:26:46:616 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:26:46:616 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:26:46:616 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:26:46:616 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/40370->127.0.0.1/48008 Mar 30 15:26:46:616 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/40370 Mar 30 15:26:46:616 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/40370' delegated to TSESS-A-3 [140603195123456] Mar 30 15:26:46:616 [140603195123456] 3 hub: TSESS-A-3-10 [SecondaryHA] restarted (for local) Mar 30 15:26:46:621 [140603715176192] 5 hub: TSESS-A-5-43 [TunnelB] local 100 '_close' Mar 30 15:26:46:621 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:46:621 [140603715176192] 1 hub: TSESS-A-5-43 [TunnelB] sent 100 '_close' (292 bytes, 0 ms) Mar 30 15:26:46:621 [140603715176192] 3 hub: TSESS-A-5-43 [TunnelB] local session closed, suspending Mar 30 15:26:46:621 [140603195123456] 5 hub: TSESS-A-3-10 [SecondaryHA] local 100 'hubup' Mar 30 15:26:46:621 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:46:621 [140603195123456] 1 hub: TSESS-A-3-10 [SecondaryHA] sent 100 'hubup' (689 bytes, 0 ms) Mar 30 15:26:46:632 [140603195123456] 1 hub: TSESS-A-3-10 [SecondaryHA] ssl 200 '-' (459 bytes) Mar 30 15:26:46:632 [140603195123456] 3 hub: TSESS-A-3-10 [SecondaryHA] is paired with [P-1947-10] Mar 30 15:26:46:632 [140603195123456] 1 hub: TSESS-A-3-10 [SecondaryHA] received 200 '-' (467 bytes) Mar 30 15:26:46:632 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:26:46:632 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:26:46:632 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:26:46:632 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:26:46:632 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA same proximity=0 status=0 secver=217 Mar 30 15:26:46:632 [140605283890944] 5 hub: mark_hubup_list_updated Mar 30 15:26:46:632 [140605283890944] 5 hub: hublist changed: /DOMAIN/Primary 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:46:632 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:46:632 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:46:632 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:46:632 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:46:632 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:46:632 [140605283890944] 5 hub: hublist changed: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:26:46:632 [140605283890944] 5 hub: hublist changed: /DOMAIN/PrimaryHA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:46:632 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:46:632 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:46:632 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:46:632 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:46:632 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:46:632 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:46:632 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:26:46:632 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:26:46:632 [140605283890944] 5 hub: hublist changed: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:26:46:632 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelBHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:26:46:632 [140605283890944] 5 hub: mark_hubup_list_updated end Mar 30 15:26:46:632 [140605283890944] 1 hub: PERF: hubup_check_for_updates: 0 sec hubhash_size: 18 Mar 30 15:26:46:632 [140605283890944] 1 hub: PERF: hubi_run_command: 0 sec hubhash_size: 18 Mar 30 15:26:46:632 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:26:46:633 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:26:46:633 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:26:46:633 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:26:46:633 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:26:46:634 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:26:46:634 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:26:46:634 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:26:46:635 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:26:46:635 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:26:46:635 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:26:46:636 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:26:46:636 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:26:46:636 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:26:46:637 [140603195123456] 5 hub: TSESS-A-3-10 [SecondaryHA] local 100 '_close' Mar 30 15:26:46:637 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:46:637 [140603195123456] 1 hub: TSESS-A-3-10 [SecondaryHA] sent 100 '_close' (296 bytes, 0 ms) Mar 30 15:26:46:637 [140603195123456] 3 hub: TSESS-A-3-10 [SecondaryHA] local session closed, suspending Mar 30 15:26:46:637 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:26:46:638 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:26:46:638 [140605283890944] 1 hub: PERF: hubi_check_security: 0 sec hubhash_size: 18 Mar 30 15:26:46:805 [140603203516160] 5 hub: TSESS-A-6-10 [Secondary] suspended Mar 30 15:26:46:821 [140603715176192] 5 hub: TSESS-A-5-43 [TunnelB] suspended Mar 30 15:26:46:837 [140603195123456] 5 hub: TSESS-A-3-10 [SecondaryHA] suspended Mar 30 15:26:46:959 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:26:46:987 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:26:46:996 [140605547460672] 2 hub: verify_login from ctrl 127.0.0.1/53680 Mar 30 15:26:46:996 [140605547460672] 1 hub: verify - [target] prid=controller cmd=probe_list ip=10.x.x.x perm=1 Mar 30 15:26:46:996 [140605547460672] 1 hub: verify - [source] id=ppm ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:47:604 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:26:47:604 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:47:922 [140603161552640] 5 hub: TSESS-P-4-10 [SecondaryHA] suspended Mar 30 15:26:47:935 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:26:47:959 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:26:47:997 [140605547460672] 4 hub: medium timeout Mar 30 15:26:47:997 [140605547460672] 3 hub: next_check [1585596696] now [1585596407] Mar 30 15:26:48:050 [140603153159936] 5 hub: TSESS-P-8-7 [Secondary] suspended Mar 30 15:26:48:278 [140603169945344] 5 hub: TSESS-P-7-10 [TunnelB] suspended Mar 30 15:26:48:699 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:26:49:407 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:51:209 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:51:910 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:26:51:910 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:26:51:910 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32976 Mar 30 15:26:51:910 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32976' delegated to TSESS-A-5 [140603715176192] Mar 30 15:26:51:910 [140603715176192] 3 hub: TSESS-A-5-44 [TunnelB] restarted (for local) Mar 30 15:26:51:915 [140603715176192] 5 hub: TSESS-A-5-44 [TunnelB] local 100 'get_info' Mar 30 15:26:51:915 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:51:915 [140603715176192] 1 hub: TSESS-A-5-44 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:26:51:926 [140603715176192] 1 hub: TSESS-A-5-44 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:26:51:926 [140603715176192] 3 hub: TSESS-A-5-44 [TunnelB] is paired with [P-298-44] Mar 30 15:26:51:926 [140603715176192] 1 hub: TSESS-A-5-44 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:26:51:926 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32978 Mar 30 15:26:51:926 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32978' delegated to TSESS-A-10 [140603706783488] Mar 30 15:26:51:926 [140603706783488] 3 hub: TSESS-A-10-32 [TunnelB] restarted (for local) Mar 30 15:26:51:931 [140603715176192] 5 hub: TSESS-A-5-44 [TunnelB] local 100 '_close' Mar 30 15:26:51:931 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:51:931 [140603715176192] 1 hub: TSESS-A-5-44 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:26:51:931 [140603715176192] 3 hub: TSESS-A-5-44 [TunnelB] local session closed, suspending Mar 30 15:26:51:931 [140603706783488] 5 hub: TSESS-A-10-32 [TunnelB] local 100 '_status' Mar 30 15:26:51:931 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:51:931 [140603706783488] 1 hub: TSESS-A-10-32 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:26:51:941 [140603706783488] 1 hub: TSESS-A-10-32 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:26:51:941 [140603706783488] 3 hub: TSESS-A-10-32 [TunnelB] is paired with [P-300-32] Mar 30 15:26:51:941 [140603706783488] 1 hub: TSESS-A-10-32 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:26:51:946 [140603706783488] 5 hub: TSESS-A-10-32 [TunnelB] local 100 '_close' Mar 30 15:26:51:946 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:26:51:947 [140603706783488] 1 hub: TSESS-A-10-32 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:26:51:947 [140603706783488] 3 hub: TSESS-A-10-32 [TunnelB] local session closed, suspending Mar 30 15:26:52:131 [140603715176192] 5 hub: TSESS-A-5-44 [TunnelB] suspended Mar 30 15:26:52:147 [140603706783488] 5 hub: TSESS-A-10-32 [TunnelB] suspended Mar 30 15:26:53:012 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:55:715 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:56:806 [140603203516160] 5 hub: TSESS-A-6-10 [Secondary] suspended Mar 30 15:26:56:837 [140603195123456] 5 hub: TSESS-A-3-10 [SecondaryHA] suspended Mar 30 15:26:56:959 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:26:56:987 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:26:57:518 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:26:57:604 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:26:57:916 [140605547460672] 4 hub: medium timeout Mar 30 15:26:57:916 [140605547460672] 3 hub: next_check [1585596696] now [1585596417] Mar 30 15:26:57:923 [140603161552640] 5 hub: TSESS-P-4-10 [SecondaryHA] suspended Mar 30 15:26:57:935 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:26:57:959 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:26:58:050 [140603153159936] 5 hub: TSESS-P-8-7 [Secondary] suspended Mar 30 15:26:58:278 [140603169945344] 5 hub: TSESS-P-7-10 [TunnelB] suspended Mar 30 15:26:58:699 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:26:59:320 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:01:123 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:01:953 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:27:01:953 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:27:01:953 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32984 Mar 30 15:27:01:953 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32984' delegated to TSESS-A-5 [140603715176192] Mar 30 15:27:01:953 [140603715176192] 3 hub: TSESS-A-5-45 [TunnelB] restarted (for local) Mar 30 15:27:01:958 [140603715176192] 5 hub: TSESS-A-5-45 [TunnelB] local 100 'get_info' Mar 30 15:27:01:958 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:01:958 [140603715176192] 1 hub: TSESS-A-5-45 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:27:01:969 [140603715176192] 1 hub: TSESS-A-5-45 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:27:01:969 [140603715176192] 3 hub: TSESS-A-5-45 [TunnelB] is paired with [P-298-45] Mar 30 15:27:01:969 [140603715176192] 1 hub: TSESS-A-5-45 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:27:01:969 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/32986 Mar 30 15:27:01:969 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/32986' delegated to TSESS-A-10 [140603706783488] Mar 30 15:27:01:969 [140603706783488] 3 hub: TSESS-A-10-33 [TunnelB] restarted (for local) Mar 30 15:27:01:974 [140603715176192] 5 hub: TSESS-A-5-45 [TunnelB] local 100 '_close' Mar 30 15:27:01:974 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:01:974 [140603715176192] 1 hub: TSESS-A-5-45 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:27:01:974 [140603715176192] 3 hub: TSESS-A-5-45 [TunnelB] local session closed, suspending Mar 30 15:27:01:974 [140603706783488] 5 hub: TSESS-A-10-33 [TunnelB] local 100 '_status' Mar 30 15:27:01:974 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:01:974 [140603706783488] 1 hub: TSESS-A-10-33 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:27:01:984 [140603706783488] 1 hub: TSESS-A-10-33 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:27:01:984 [140603706783488] 3 hub: TSESS-A-10-33 [TunnelB] is paired with [P-300-33] Mar 30 15:27:01:984 [140603706783488] 1 hub: TSESS-A-10-33 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:27:01:990 [140603706783488] 5 hub: TSESS-A-10-33 [TunnelB] local 100 '_close' Mar 30 15:27:01:990 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:01:990 [140603706783488] 1 hub: TSESS-A-10-33 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:27:01:990 [140603706783488] 3 hub: TSESS-A-10-33 [TunnelB] local session closed, suspending Mar 30 15:27:02:174 [140603715176192] 5 hub: TSESS-A-5-45 [TunnelB] suspended Mar 30 15:27:02:190 [140603706783488] 5 hub: TSESS-A-10-33 [TunnelB] suspended Mar 30 15:27:03:826 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:05:629 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:05:957 [140605547460672] 3 hub: check_session - session 0x20f9050 (127.0.0.1/48002) is a server Mar 30 15:27:05:957 [140605547460672] 2 hub: Profile - clayk 38 bytes (OK) Mar 30 15:27:06:806 [140603203516160] 5 hub: TSESS-A-6-10 [Secondary] suspended Mar 30 15:27:06:838 [140603195123456] 5 hub: TSESS-A-3-10 [SecondaryHA] suspended Mar 30 15:27:06:959 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:27:06:988 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:27:07:178 [140603740354304] 5 hub: TSESS-P-8 not internal command - thread will handle. mtype=100; sfd:23 Mar 30 15:27:07:178 [140603740354304] 2 hub: CTRL Secondary request 'hubup' passed to TSESS-P-8 [140603153159936] Mar 30 15:27:07:178 [140603153159936] 3 hub: TSESS-P-8-8 [Secondary] restarted (for remote) Mar 30 15:27:07:178 [140603153159936] 2 hub: TSESS-P-8 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:37:07 Mar 30 15:27:07:178 [140603153159936] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:27:07:178 [140603153159936] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:27:07:178 [140603153159936] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:27:07:178 [140603153159936] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:27:07:178 [140603153159936] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:27:07:178 [140603153159936] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:27:07:178 [140603153159936] 4 hub: CONNECT: 0x7fe0a0000d90(39) 127.0.0.1/53702->127.0.0.1/48002 Mar 30 15:27:07:178 [140603153159936] 3 hub: TSESS-P-8-8 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:27:07:178 [140603153159936] 1 hub: TSESS-P-8-8 [Secondary] received 100 'hubup' (683 bytes) Mar 30 15:27:07:178 [140605547460672] 4 hub: medium timeout Mar 30 15:27:07:178 [140605547460672] 3 hub: next_check [1585596696] now [1585596427] Mar 30 15:27:07:178 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53702 (s:388) Mar 30 15:27:07:178 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/53702] Mar 30 15:27:07:178 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:27:07:178 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:27:07:178 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/53702 Mar 30 15:27:07:178 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/53702 same proximity=0 status=0 secver=217 Mar 30 15:27:07:183 [140603153159936] 5 hub: TSESS-P-8-8 [Secondary] local 200 'OK' Mar 30 15:27:07:183 [140603153159936] 1 hub: TSESS-P-8-8 [Secondary] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:27:07:194 [140603153159936] 1 hub: TSESS-P-8-8 [Secondary] ssl 100 '_close' (296 bytes) Mar 30 15:27:07:194 [140603153159936] 3 hub: TSESS-P-8-8 [Secondary] is paired with [A-2121-8] Mar 30 15:27:07:194 [140603153159936] 1 hub: TSESS-P-8-8 [Secondary] received 100 '_close' (296 bytes) Mar 30 15:27:07:199 [140603153159936] 1 hub: TSESS-P-8-8 [Secondary] remote session closed Mar 30 15:27:07:199 [140603153159936] 3 hub: TSESS-P-8-8 [Secondary] exiting msg loop (remote close) Mar 30 15:27:07:199 [140603153159936] 5 hub: TSESS-P-8-8 [Secondary] suspended Mar 30 15:27:07:430 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:07:604 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:27:07:923 [140603161552640] 5 hub: TSESS-P-4-10 [SecondaryHA] suspended Mar 30 15:27:07:936 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:27:07:959 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:27:08:053 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:27:08:053 [140603731961600] 2 hub: CTRL SecondaryHA request 'hubup' passed to TSESS-P-4 [140603161552640] Mar 30 15:27:08:053 [140603161552640] 3 hub: TSESS-P-4-11 [SecondaryHA] restarted (for remote) Mar 30 15:27:08:053 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:37:07 Mar 30 15:27:08:053 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:27:08:053 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:27:08:053 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:27:08:053 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:27:08:053 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:27:08:053 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:27:08:053 [140603161552640] 4 hub: CONNECT: 0x7fe09c000d20(39) 127.0.0.1/53704->127.0.0.1/48002 Mar 30 15:27:08:053 [140603161552640] 3 hub: TSESS-P-4-11 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:27:08:053 [140603161552640] 1 hub: TSESS-P-4-11 [SecondaryHA] received 100 'hubup' (689 bytes) Mar 30 15:27:08:053 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53704 (s:392) Mar 30 15:27:08:053 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/53704] Mar 30 15:27:08:053 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:27:08:053 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:27:08:053 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/53704 Mar 30 15:27:08:053 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/53704 same proximity=0 status=0 secver=217 Mar 30 15:27:08:058 [140603161552640] 5 hub: TSESS-P-4-11 [SecondaryHA] local 200 'OK' Mar 30 15:27:08:058 [140603161552640] 1 hub: TSESS-P-4-11 [SecondaryHA] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:27:08:069 [140603161552640] 1 hub: TSESS-P-4-11 [SecondaryHA] ssl 100 '_close' (302 bytes) Mar 30 15:27:08:069 [140603161552640] 3 hub: TSESS-P-4-11 [SecondaryHA] is paired with [A-1948-11] Mar 30 15:27:08:069 [140603161552640] 1 hub: TSESS-P-4-11 [SecondaryHA] received 100 '_close' (298 bytes) Mar 30 15:27:08:074 [140603161552640] 1 hub: TSESS-P-4-11 [SecondaryHA] remote session closed Mar 30 15:27:08:074 [140603161552640] 3 hub: TSESS-P-4-11 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:27:08:074 [140603161552640] 5 hub: TSESS-P-4-11 [SecondaryHA] suspended Mar 30 15:27:08:278 [140603169945344] 5 hub: TSESS-P-7-10 [TunnelB] suspended Mar 30 15:27:08:341 [140603723568896] 5 hub: TSESS-P-7 not internal command - thread will handle. mtype=100; sfd:22 Mar 30 15:27:08:341 [140603723568896] 2 hub: CTRL TunnelB request 'hubup' passed to TSESS-P-7 [140603169945344] Mar 30 15:27:08:342 [140603169945344] 3 hub: TSESS-P-7-11 [TunnelB] restarted (for remote) Mar 30 15:27:08:342 [140603169945344] 2 hub: TSESS-P-7 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:37:08 Mar 30 15:27:08:342 [140603169945344] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:27:08:342 [140603169945344] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:27:08:342 [140603169945344] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:27:08:342 [140603169945344] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:27:08:342 [140603169945344] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:27:08:342 [140603169945344] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:27:08:342 [140603169945344] 4 hub: CONNECT: 0x7fe0a8000d20(39) 127.0.0.1/53706->127.0.0.1/48002 Mar 30 15:27:08:342 [140603169945344] 3 hub: TSESS-P-7-11 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:27:08:342 [140603169945344] 1 hub: TSESS-P-7-11 [TunnelB] received 100 'hubup' (673 bytes) Mar 30 15:27:08:342 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53706 (s:378) Mar 30 15:27:08:342 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/53706] Mar 30 15:27:08:342 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:27:08:342 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:27:08:342 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/53706 Mar 30 15:27:08:342 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/53706 same proximity=0 status=0 secver=217 Mar 30 15:27:08:347 [140603169945344] 5 hub: TSESS-P-7-11 [TunnelB] local 200 'OK' Mar 30 15:27:08:347 [140603169945344] 1 hub: TSESS-P-7-11 [TunnelB] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:27:08:357 [140603169945344] 1 hub: TSESS-P-7-11 [TunnelB] ssl 100 '_close' (284 bytes) Mar 30 15:27:08:357 [140603169945344] 3 hub: TSESS-P-7-11 [TunnelB] is paired with [A-299-11] Mar 30 15:27:08:357 [140603169945344] 1 hub: TSESS-P-7-11 [TunnelB] received 100 '_close' (296 bytes) Mar 30 15:27:08:362 [140603169945344] 1 hub: TSESS-P-7-11 [TunnelB] remote session closed Mar 30 15:27:08:362 [140603169945344] 3 hub: TSESS-P-7-11 [TunnelB] exiting msg loop (remote close) Mar 30 15:27:08:363 [140603169945344] 5 hub: TSESS-P-7-11 [TunnelB] suspended Mar 30 15:27:08:699 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:27:09:233 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:11:035 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:11:993 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:27:11:993 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:27:11:993 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33002 Mar 30 15:27:11:993 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33002' delegated to TSESS-A-5 [140603715176192] Mar 30 15:27:11:993 [140603715176192] 3 hub: TSESS-A-5-46 [TunnelB] restarted (for local) Mar 30 15:27:11:998 [140603715176192] 5 hub: TSESS-A-5-46 [TunnelB] local 100 'get_info' Mar 30 15:27:11:999 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:11:999 [140603715176192] 1 hub: TSESS-A-5-46 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:27:12:009 [140603715176192] 1 hub: TSESS-A-5-46 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:27:12:009 [140603715176192] 3 hub: TSESS-A-5-46 [TunnelB] is paired with [P-298-46] Mar 30 15:27:12:009 [140603715176192] 1 hub: TSESS-A-5-46 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:27:12:009 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33004 Mar 30 15:27:12:009 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33004' delegated to TSESS-A-10 [140603706783488] Mar 30 15:27:12:009 [140603706783488] 3 hub: TSESS-A-10-34 [TunnelB] restarted (for local) Mar 30 15:27:12:014 [140603715176192] 5 hub: TSESS-A-5-46 [TunnelB] local 100 '_close' Mar 30 15:27:12:014 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:12:014 [140603715176192] 1 hub: TSESS-A-5-46 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:27:12:014 [140603715176192] 3 hub: TSESS-A-5-46 [TunnelB] local session closed, suspending Mar 30 15:27:12:014 [140603706783488] 5 hub: TSESS-A-10-34 [TunnelB] local 100 '_status' Mar 30 15:27:12:014 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:12:014 [140603706783488] 1 hub: TSESS-A-10-34 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:27:12:024 [140603706783488] 1 hub: TSESS-A-10-34 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:27:12:025 [140603706783488] 3 hub: TSESS-A-10-34 [TunnelB] is paired with [P-300-34] Mar 30 15:27:12:025 [140603706783488] 1 hub: TSESS-A-10-34 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:27:12:030 [140603706783488] 5 hub: TSESS-A-10-34 [TunnelB] local 100 '_close' Mar 30 15:27:12:030 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:12:030 [140603706783488] 1 hub: TSESS-A-10-34 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:27:12:030 [140603706783488] 3 hub: TSESS-A-10-34 [TunnelB] local session closed, suspending Mar 30 15:27:12:214 [140603715176192] 5 hub: TSESS-A-5-46 [TunnelB] suspended Mar 30 15:27:12:230 [140603706783488] 5 hub: TSESS-A-10-34 [TunnelB] suspended Mar 30 15:27:13:739 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:15:541 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:16:670 [140605283890944] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:27:16:670 [140605283890944] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:27:16:670 [140605283890944] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:27:16:670 [140605283890944] 1 hub: PERF: hubup_check_interval: 0 sec hubhash_size: 18 Mar 30 15:27:16:670 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:670 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:670 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:670 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:670 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:670 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:670 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:27:16:670 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:670 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:670 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:670 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:670 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:670 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:670 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:670 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:670 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:27:16:670 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:27:16:670 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:27:16:670 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/47378->127.0.0.1/48007 Mar 30 15:27:16:670 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/47378 Mar 30 15:27:16:670 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/47378' delegated to TSESS-A-6 [140603203516160] Mar 30 15:27:16:670 [140603203516160] 3 hub: TSESS-A-6-11 [Secondary] restarted (for local) Mar 30 15:27:16:676 [140603203516160] 5 hub: TSESS-A-6-11 [Secondary] local 100 'hubup' Mar 30 15:27:16:676 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:16:676 [140603203516160] 1 hub: TSESS-A-6-11 [Secondary] sent 100 'hubup' (687 bytes, 0 ms) Mar 30 15:27:16:686 [140603203516160] 1 hub: TSESS-A-6-11 [Secondary] ssl 200 '-' (455 bytes) Mar 30 15:27:16:686 [140603203516160] 3 hub: TSESS-A-6-11 [Secondary] is paired with [P-2120-11] Mar 30 15:27:16:686 [140603203516160] 1 hub: TSESS-A-6-11 [Secondary] received 200 '-' (463 bytes) Mar 30 15:27:16:686 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:27:16:686 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:27:16:686 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:27:16:686 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:27:16:686 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary same proximity=0 status=0 secver=217 Mar 30 15:27:16:686 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:686 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:686 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:686 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:686 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:686 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:686 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:27:16:686 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:686 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:686 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:686 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:686 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:686 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:686 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:686 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:686 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:27:16:686 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:27:16:686 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:27:16:686 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/33010->127.0.0.1/48009 Mar 30 15:27:16:686 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33010 Mar 30 15:27:16:686 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33010' delegated to TSESS-A-5 [140603715176192] Mar 30 15:27:16:687 [140603715176192] 3 hub: TSESS-A-5-47 [TunnelB] restarted (for local) Mar 30 15:27:16:691 [140603203516160] 5 hub: TSESS-A-6-11 [Secondary] local 100 '_close' Mar 30 15:27:16:691 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:16:691 [140603203516160] 1 hub: TSESS-A-6-11 [Secondary] sent 100 '_close' (294 bytes, 0 ms) Mar 30 15:27:16:691 [140603203516160] 3 hub: TSESS-A-6-11 [Secondary] local session closed, suspending Mar 30 15:27:16:692 [140603715176192] 5 hub: TSESS-A-5-47 [TunnelB] local 100 'hubup' Mar 30 15:27:16:692 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:16:692 [140603715176192] 1 hub: TSESS-A-5-47 [TunnelB] sent 100 'hubup' (685 bytes, 0 ms) Mar 30 15:27:16:702 [140603715176192] 1 hub: TSESS-A-5-47 [TunnelB] ssl 200 '-' (443 bytes) Mar 30 15:27:16:702 [140603715176192] 3 hub: TSESS-A-5-47 [TunnelB] is paired with [P-298-47] Mar 30 15:27:16:702 [140603715176192] 1 hub: TSESS-A-5-47 [TunnelB] received 200 '-' (451 bytes) Mar 30 15:27:16:702 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:27:16:702 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:27:16:702 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:27:16:702 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:27:16:702 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB same proximity=0 status=0 secver=217 Mar 30 15:27:16:702 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:702 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:702 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:702 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:702 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:702 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:702 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:27:16:702 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:702 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:702 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:702 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:702 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:702 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:702 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:702 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:16:702 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:27:16:702 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:27:16:702 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:27:16:702 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/40430->127.0.0.1/48008 Mar 30 15:27:16:702 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/40430 Mar 30 15:27:16:702 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/40430' delegated to TSESS-A-3 [140603195123456] Mar 30 15:27:16:702 [140603195123456] 3 hub: TSESS-A-3-11 [SecondaryHA] restarted (for local) Mar 30 15:27:16:707 [140603715176192] 5 hub: TSESS-A-5-47 [TunnelB] local 100 '_close' Mar 30 15:27:16:707 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:16:707 [140603715176192] 1 hub: TSESS-A-5-47 [TunnelB] sent 100 '_close' (292 bytes, 0 ms) Mar 30 15:27:16:707 [140603715176192] 3 hub: TSESS-A-5-47 [TunnelB] local session closed, suspending Mar 30 15:27:16:708 [140603195123456] 5 hub: TSESS-A-3-11 [SecondaryHA] local 100 'hubup' Mar 30 15:27:16:708 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:16:708 [140603195123456] 1 hub: TSESS-A-3-11 [SecondaryHA] sent 100 'hubup' (689 bytes, 0 ms) Mar 30 15:27:16:718 [140603195123456] 1 hub: TSESS-A-3-11 [SecondaryHA] ssl 200 '-' (459 bytes) Mar 30 15:27:16:718 [140603195123456] 3 hub: TSESS-A-3-11 [SecondaryHA] is paired with [P-1947-11] Mar 30 15:27:16:718 [140603195123456] 1 hub: TSESS-A-3-11 [SecondaryHA] received 200 '-' (467 bytes) Mar 30 15:27:16:718 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:27:16:718 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:27:16:718 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:27:16:718 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:27:16:718 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA same proximity=0 status=0 secver=217 Mar 30 15:27:16:718 [140605283890944] 5 hub: mark_hubup_list_updated Mar 30 15:27:16:718 [140605283890944] 5 hub: hublist changed: /DOMAIN/Primary 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:16:718 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:16:718 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:16:718 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:16:718 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:16:718 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:16:718 [140605283890944] 5 hub: hublist changed: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:27:16:718 [140605283890944] 5 hub: hublist changed: /DOMAIN/PrimaryHA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:16:718 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:16:718 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:16:718 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:16:718 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:16:718 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:16:718 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:16:718 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:16:718 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:27:16:718 [140605283890944] 5 hub: hublist changed: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:27:16:718 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelBHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:27:16:718 [140605283890944] 5 hub: mark_hubup_list_updated end Mar 30 15:27:16:718 [140605283890944] 1 hub: PERF: hubup_check_for_updates: 0 sec hubhash_size: 18 Mar 30 15:27:16:718 [140605283890944] 1 hub: PERF: hubi_run_command: 0 sec hubhash_size: 18 Mar 30 15:27:16:718 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:27:16:719 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:27:16:719 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:27:16:719 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:27:16:719 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:27:16:720 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:27:16:720 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:27:16:721 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:27:16:721 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:27:16:722 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:27:16:722 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:27:16:722 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:27:16:722 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:27:16:723 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:27:16:723 [140603195123456] 5 hub: TSESS-A-3-11 [SecondaryHA] local 100 '_close' Mar 30 15:27:16:723 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:16:723 [140603195123456] 1 hub: TSESS-A-3-11 [SecondaryHA] sent 100 '_close' (296 bytes, 0 ms) Mar 30 15:27:16:723 [140603195123456] 3 hub: TSESS-A-3-11 [SecondaryHA] local session closed, suspending Mar 30 15:27:16:723 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:27:16:724 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:27:16:725 [140605283890944] 1 hub: PERF: hubi_check_security: 0 sec hubhash_size: 18 Mar 30 15:27:16:891 [140603203516160] 5 hub: TSESS-A-6-11 [Secondary] suspended Mar 30 15:27:16:907 [140603715176192] 5 hub: TSESS-A-5-47 [TunnelB] suspended Mar 30 15:27:16:923 [140603195123456] 5 hub: TSESS-A-3-11 [SecondaryHA] suspended Mar 30 15:27:16:959 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:27:16:988 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:27:17:199 [140603153159936] 5 hub: TSESS-P-8-8 [Secondary] suspended Mar 30 15:27:17:344 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:17:604 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:27:17:936 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:27:17:959 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:27:17:999 [140605547460672] 4 hub: medium timeout Mar 30 15:27:18:000 [140605547460672] 3 hub: next_check [1585596696] now [1585596437] Mar 30 15:27:18:074 [140603161552640] 5 hub: TSESS-P-4-11 [SecondaryHA] suspended Mar 30 15:27:18:363 [140603169945344] 5 hub: TSESS-P-7-11 [TunnelB] suspended Mar 30 15:27:18:699 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:27:19:147 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:21:031 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:27:21:031 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:27:21:031 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33018 Mar 30 15:27:21:031 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33018' delegated to TSESS-A-5 [140603715176192] Mar 30 15:27:21:031 [140603715176192] 3 hub: TSESS-A-5-48 [TunnelB] restarted (for local) Mar 30 15:27:21:036 [140603715176192] 5 hub: TSESS-A-5-48 [TunnelB] local 100 'get_info' Mar 30 15:27:21:036 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:21:036 [140603715176192] 1 hub: TSESS-A-5-48 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:27:21:047 [140603715176192] 1 hub: TSESS-A-5-48 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:27:21:047 [140603715176192] 3 hub: TSESS-A-5-48 [TunnelB] is paired with [P-298-48] Mar 30 15:27:21:047 [140603715176192] 1 hub: TSESS-A-5-48 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:27:21:047 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33020 Mar 30 15:27:21:047 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33020' delegated to TSESS-A-10 [140603706783488] Mar 30 15:27:21:047 [140603706783488] 3 hub: TSESS-A-10-35 [TunnelB] restarted (for local) Mar 30 15:27:21:052 [140603715176192] 5 hub: TSESS-A-5-48 [TunnelB] local 100 '_close' Mar 30 15:27:21:052 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:21:052 [140603715176192] 1 hub: TSESS-A-5-48 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:27:21:052 [140603715176192] 3 hub: TSESS-A-5-48 [TunnelB] local session closed, suspending Mar 30 15:27:21:052 [140603706783488] 5 hub: TSESS-A-10-35 [TunnelB] local 100 '_status' Mar 30 15:27:21:052 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:21:052 [140603706783488] 1 hub: TSESS-A-10-35 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:27:21:063 [140603706783488] 1 hub: TSESS-A-10-35 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:27:21:063 [140603706783488] 3 hub: TSESS-A-10-35 [TunnelB] is paired with [P-300-35] Mar 30 15:27:21:063 [140603706783488] 1 hub: TSESS-A-10-35 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:27:21:068 [140603706783488] 5 hub: TSESS-A-10-35 [TunnelB] local 100 '_close' Mar 30 15:27:21:068 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:21:068 [140603706783488] 1 hub: TSESS-A-10-35 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:27:21:068 [140603706783488] 3 hub: TSESS-A-10-35 [TunnelB] local session closed, suspending Mar 30 15:27:21:252 [140603715176192] 5 hub: TSESS-A-5-48 [TunnelB] suspended Mar 30 15:27:21:268 [140603706783488] 5 hub: TSESS-A-10-35 [TunnelB] suspended Mar 30 15:27:21:850 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:23:652 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:25:456 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:26:892 [140603203516160] 5 hub: TSESS-A-6-11 [Secondary] suspended Mar 30 15:27:26:924 [140603195123456] 5 hub: TSESS-A-3-11 [SecondaryHA] suspended Mar 30 15:27:26:960 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:27:26:988 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:27:27:037 [140605547460672] 4 hub: medium timeout Mar 30 15:27:27:037 [140605547460672] 3 hub: next_check [1585596696] now [1585596447] Mar 30 15:27:27:199 [140603153159936] 5 hub: TSESS-P-8-8 [Secondary] suspended Mar 30 15:27:27:258 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:27:605 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:27:27:936 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:27:27:959 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:27:28:074 [140603161552640] 5 hub: TSESS-P-4-11 [SecondaryHA] suspended Mar 30 15:27:28:363 [140603169945344] 5 hub: TSESS-P-7-11 [TunnelB] suspended Mar 30 15:27:28:699 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:27:29:061 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:31:073 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:27:31:073 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:27:31:073 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33026 Mar 30 15:27:31:073 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33026' delegated to TSESS-A-5 [140603715176192] Mar 30 15:27:31:073 [140603715176192] 3 hub: TSESS-A-5-49 [TunnelB] restarted (for local) Mar 30 15:27:31:078 [140603715176192] 5 hub: TSESS-A-5-49 [TunnelB] local 100 'get_info' Mar 30 15:27:31:078 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:31:078 [140603715176192] 1 hub: TSESS-A-5-49 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:27:31:089 [140603715176192] 1 hub: TSESS-A-5-49 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:27:31:089 [140603715176192] 3 hub: TSESS-A-5-49 [TunnelB] is paired with [P-298-49] Mar 30 15:27:31:089 [140603715176192] 1 hub: TSESS-A-5-49 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:27:31:089 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33028 Mar 30 15:27:31:089 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33028' delegated to TSESS-A-10 [140603706783488] Mar 30 15:27:31:089 [140603706783488] 3 hub: TSESS-A-10-36 [TunnelB] restarted (for local) Mar 30 15:27:31:094 [140603715176192] 5 hub: TSESS-A-5-49 [TunnelB] local 100 '_close' Mar 30 15:27:31:094 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:31:094 [140603715176192] 1 hub: TSESS-A-5-49 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:27:31:094 [140603715176192] 3 hub: TSESS-A-5-49 [TunnelB] local session closed, suspending Mar 30 15:27:31:094 [140603706783488] 5 hub: TSESS-A-10-36 [TunnelB] local 100 '_status' Mar 30 15:27:31:094 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:31:094 [140603706783488] 1 hub: TSESS-A-10-36 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:27:31:104 [140603706783488] 1 hub: TSESS-A-10-36 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:27:31:104 [140603706783488] 3 hub: TSESS-A-10-36 [TunnelB] is paired with [P-300-36] Mar 30 15:27:31:104 [140603706783488] 1 hub: TSESS-A-10-36 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:27:31:109 [140603706783488] 5 hub: TSESS-A-10-36 [TunnelB] local 100 '_close' Mar 30 15:27:31:110 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:31:110 [140603706783488] 1 hub: TSESS-A-10-36 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:27:31:110 [140603706783488] 3 hub: TSESS-A-10-36 [TunnelB] local session closed, suspending Mar 30 15:27:31:294 [140603715176192] 5 hub: TSESS-A-5-49 [TunnelB] suspended Mar 30 15:27:31:310 [140603706783488] 5 hub: TSESS-A-10-36 [TunnelB] suspended Mar 30 15:27:31:764 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:33:567 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:34:036 [140604042360576] 5 hub: There are 0 objects waiting on 8 threads Mar 30 15:27:35:077 [140605547460672] 3 hub: check_session - session 0x20f9050 (127.0.0.1/48002) is a server Mar 30 15:27:35:077 [140605547460672] 2 hub: Profile - clayk 38 bytes (OK) Mar 30 15:27:35:369 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:36:892 [140603203516160] 5 hub: TSESS-A-6-11 [Secondary] suspended Mar 30 15:27:36:924 [140603195123456] 5 hub: TSESS-A-3-11 [SecondaryHA] suspended Mar 30 15:27:36:960 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:27:36:988 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:27:37:079 [140605547460672] 4 hub: medium timeout Mar 30 15:27:37:079 [140605547460672] 3 hub: next_check [1585596696] now [1585596457] Mar 30 15:27:37:172 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:37:199 [140603153159936] 5 hub: TSESS-P-8-8 [Secondary] suspended Mar 30 15:27:37:228 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:27:37:228 [140603731961600] 2 hub: CTRL SecondaryHA request 'hubup' passed to TSESS-P-4 [140603161552640] Mar 30 15:27:37:228 [140603161552640] 3 hub: TSESS-P-4-12 [SecondaryHA] restarted (for remote) Mar 30 15:27:37:228 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:37:37 Mar 30 15:27:37:228 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:27:37:228 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:27:37:228 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:27:37:228 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:27:37:228 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:27:37:228 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:27:37:229 [140603161552640] 4 hub: CONNECT: 0x7fe09c000d70(39) 127.0.0.1/53740->127.0.0.1/48002 Mar 30 15:27:37:229 [140603161552640] 3 hub: TSESS-P-4-12 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:27:37:229 [140603161552640] 1 hub: TSESS-P-4-12 [SecondaryHA] received 100 'hubup' (689 bytes) Mar 30 15:27:37:229 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53740 (s:392) Mar 30 15:27:37:229 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/53740] Mar 30 15:27:37:229 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:27:37:229 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:27:37:229 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/53740 Mar 30 15:27:37:229 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/53740 same proximity=0 status=0 secver=217 Mar 30 15:27:37:234 [140603161552640] 5 hub: TSESS-P-4-12 [SecondaryHA] local 200 'OK' Mar 30 15:27:37:234 [140603161552640] 1 hub: TSESS-P-4-12 [SecondaryHA] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:27:37:244 [140603161552640] 1 hub: TSESS-P-4-12 [SecondaryHA] ssl 100 '_close' (302 bytes) Mar 30 15:27:37:244 [140603161552640] 3 hub: TSESS-P-4-12 [SecondaryHA] is paired with [A-1948-12] Mar 30 15:27:37:244 [140603161552640] 1 hub: TSESS-P-4-12 [SecondaryHA] received 100 '_close' (298 bytes) Mar 30 15:27:37:249 [140603161552640] 1 hub: TSESS-P-4-12 [SecondaryHA] remote session closed Mar 30 15:27:37:249 [140603161552640] 3 hub: TSESS-P-4-12 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:27:37:249 [140603161552640] 5 hub: TSESS-P-4-12 [SecondaryHA] suspended Mar 30 15:27:37:343 [140603740354304] 5 hub: TSESS-P-8 not internal command - thread will handle. mtype=100; sfd:23 Mar 30 15:27:37:343 [140603740354304] 2 hub: CTRL Secondary request 'hubup' passed to TSESS-P-8 [140603153159936] Mar 30 15:27:37:343 [140603153159936] 3 hub: TSESS-P-8-9 [Secondary] restarted (for remote) Mar 30 15:27:37:343 [140603153159936] 2 hub: TSESS-P-8 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:37:37 Mar 30 15:27:37:343 [140603153159936] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:27:37:344 [140603153159936] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:27:37:344 [140603153159936] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:27:37:344 [140603153159936] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:27:37:344 [140603153159936] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:27:37:344 [140603153159936] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:27:37:344 [140603153159936] 4 hub: CONNECT: 0x7fe0a0000d90(39) 127.0.0.1/53742->127.0.0.1/48002 Mar 30 15:27:37:344 [140603153159936] 3 hub: TSESS-P-8-9 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:27:37:344 [140603153159936] 1 hub: TSESS-P-8-9 [Secondary] received 100 'hubup' (1209 bytes) Mar 30 15:27:37:344 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53742 (s:914) Mar 30 15:27:37:344 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/53742] Mar 30 15:27:37:344 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:27:37:344 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:27:37:344 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/53742 Mar 30 15:27:37:344 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/53742 same proximity=0 status=0 secver=217 Mar 30 15:27:37:344 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-11] domain [DOMAIN] addr [(null)] ip [100.64.1.21] port [48002] robotname [a0f8us052nim751] version [9.20 [Build 9.20.1048, Aug 8 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-11] source [/DOMAIN/Secondary] from [127.0.0.1/53742] Mar 30 15:27:37:344 [140605547460672] 4 hub: hubup source: /DOMAIN/Secondary Mar 30 15:27:37:344 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-11 Mar 30 15:27:37:344 [140605547460672] 5 hub: hubinfo_update - same proximity but different source (/DOMAIN/Secondary) Mar 30 15:27:37:349 [140603153159936] 5 hub: TSESS-P-8-9 [Secondary] local 200 'OK' Mar 30 15:27:37:349 [140603153159936] 1 hub: TSESS-P-8-9 [Secondary] sent 200 'OK' (456 bytes, 0 ms) Mar 30 15:27:37:359 [140603153159936] 1 hub: TSESS-P-8-9 [Secondary] ssl 100 '_close' (296 bytes) Mar 30 15:27:37:359 [140603153159936] 3 hub: TSESS-P-8-9 [Secondary] is paired with [A-2121-9] Mar 30 15:27:37:359 [140603153159936] 1 hub: TSESS-P-8-9 [Secondary] received 100 '_close' (296 bytes) Mar 30 15:27:37:364 [140603153159936] 1 hub: TSESS-P-8-9 [Secondary] remote session closed Mar 30 15:27:37:364 [140603153159936] 3 hub: TSESS-P-8-9 [Secondary] exiting msg loop (remote close) Mar 30 15:27:37:364 [140603153159936] 5 hub: TSESS-P-8-9 [Secondary] suspended Mar 30 15:27:37:605 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:27:37:936 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:27:37:960 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:27:38:363 [140603169945344] 5 hub: TSESS-P-7-11 [TunnelB] suspended Mar 30 15:27:38:424 [140603723568896] 5 hub: TSESS-P-7 not internal command - thread will handle. mtype=100; sfd:22 Mar 30 15:27:38:424 [140603723568896] 2 hub: CTRL TunnelB request 'hubup' passed to TSESS-P-7 [140603169945344] Mar 30 15:27:38:424 [140603169945344] 3 hub: TSESS-P-7-12 [TunnelB] restarted (for remote) Mar 30 15:27:38:424 [140603169945344] 2 hub: TSESS-P-7 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:37:38 Mar 30 15:27:38:424 [140603169945344] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:27:38:424 [140603169945344] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:27:38:424 [140603169945344] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:27:38:424 [140603169945344] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:27:38:424 [140603169945344] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:27:38:424 [140603169945344] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:27:38:424 [140603169945344] 4 hub: CONNECT: 0x7fe0a8000d20(39) 127.0.0.1/53744->127.0.0.1/48002 Mar 30 15:27:38:424 [140603169945344] 3 hub: TSESS-P-7-12 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:27:38:424 [140603169945344] 1 hub: TSESS-P-7-12 [TunnelB] received 100 'hubup' (673 bytes) Mar 30 15:27:38:424 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53744 (s:378) Mar 30 15:27:38:424 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/53744] Mar 30 15:27:38:424 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:27:38:424 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:27:38:424 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/53744 Mar 30 15:27:38:424 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/53744 same proximity=0 status=0 secver=217 Mar 30 15:27:38:429 [140603169945344] 5 hub: TSESS-P-7-12 [TunnelB] local 200 'OK' Mar 30 15:27:38:429 [140603169945344] 1 hub: TSESS-P-7-12 [TunnelB] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:27:38:440 [140603169945344] 1 hub: TSESS-P-7-12 [TunnelB] ssl 100 '_close' (284 bytes) Mar 30 15:27:38:440 [140603169945344] 3 hub: TSESS-P-7-12 [TunnelB] is paired with [A-299-12] Mar 30 15:27:38:440 [140603169945344] 1 hub: TSESS-P-7-12 [TunnelB] received 100 '_close' (296 bytes) Mar 30 15:27:38:445 [140603169945344] 1 hub: TSESS-P-7-12 [TunnelB] remote session closed Mar 30 15:27:38:445 [140603169945344] 3 hub: TSESS-P-7-12 [TunnelB] exiting msg loop (remote close) Mar 30 15:27:38:445 [140603169945344] 5 hub: TSESS-P-7-12 [TunnelB] suspended Mar 30 15:27:38:700 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:27:39:875 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:41:116 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:27:41:116 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:27:41:116 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33042 Mar 30 15:27:41:116 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33042' delegated to TSESS-A-5 [140603715176192] Mar 30 15:27:41:116 [140603715176192] 3 hub: TSESS-A-5-50 [TunnelB] restarted (for local) Mar 30 15:27:41:121 [140603715176192] 5 hub: TSESS-A-5-50 [TunnelB] local 100 'get_info' Mar 30 15:27:41:121 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:41:121 [140603715176192] 1 hub: TSESS-A-5-50 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:27:41:131 [140603715176192] 1 hub: TSESS-A-5-50 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:27:41:131 [140603715176192] 3 hub: TSESS-A-5-50 [TunnelB] is paired with [P-298-50] Mar 30 15:27:41:132 [140603715176192] 1 hub: TSESS-A-5-50 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:27:41:132 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33044 Mar 30 15:27:41:132 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33044' delegated to TSESS-A-10 [140603706783488] Mar 30 15:27:41:132 [140603706783488] 3 hub: TSESS-A-10-37 [TunnelB] restarted (for local) Mar 30 15:27:41:137 [140603715176192] 5 hub: TSESS-A-5-50 [TunnelB] local 100 '_close' Mar 30 15:27:41:137 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:41:137 [140603715176192] 1 hub: TSESS-A-5-50 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:27:41:137 [140603715176192] 3 hub: TSESS-A-5-50 [TunnelB] local session closed, suspending Mar 30 15:27:41:137 [140603706783488] 5 hub: TSESS-A-10-37 [TunnelB] local 100 '_status' Mar 30 15:27:41:137 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:41:137 [140603706783488] 1 hub: TSESS-A-10-37 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:27:41:147 [140603706783488] 1 hub: TSESS-A-10-37 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:27:41:147 [140603706783488] 3 hub: TSESS-A-10-37 [TunnelB] is paired with [P-300-37] Mar 30 15:27:41:147 [140603706783488] 1 hub: TSESS-A-10-37 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:27:41:152 [140603706783488] 5 hub: TSESS-A-10-37 [TunnelB] local 100 '_close' Mar 30 15:27:41:152 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:41:152 [140603706783488] 1 hub: TSESS-A-10-37 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:27:41:152 [140603706783488] 3 hub: TSESS-A-10-37 [TunnelB] local session closed, suspending Mar 30 15:27:41:337 [140603715176192] 5 hub: TSESS-A-5-50 [TunnelB] suspended Mar 30 15:27:41:353 [140603706783488] 5 hub: TSESS-A-10-37 [TunnelB] suspended Mar 30 15:27:41:678 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:43:480 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:45:283 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:46:755 [140605283890944] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:27:46:756 [140605283890944] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:27:46:756 [140605283890944] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:27:46:756 [140605283890944] 1 hub: PERF: hubup_check_interval: 0 sec hubhash_size: 18 Mar 30 15:27:46:756 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:756 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:756 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:756 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:756 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:756 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:756 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:27:46:756 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:756 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:756 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:756 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:756 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:756 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:756 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:756 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:756 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:27:46:756 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:27:46:756 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:27:46:756 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/47420->127.0.0.1/48007 Mar 30 15:27:46:756 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/47420 Mar 30 15:27:46:756 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/47420' delegated to TSESS-A-6 [140603203516160] Mar 30 15:27:46:756 [140603203516160] 3 hub: TSESS-A-6-12 [Secondary] restarted (for local) Mar 30 15:27:46:761 [140603203516160] 5 hub: TSESS-A-6-12 [Secondary] local 100 'hubup' Mar 30 15:27:46:761 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:46:761 [140603203516160] 1 hub: TSESS-A-6-12 [Secondary] sent 100 'hubup' (687 bytes, 0 ms) Mar 30 15:27:46:772 [140603203516160] 1 hub: TSESS-A-6-12 [Secondary] ssl 200 '-' (455 bytes) Mar 30 15:27:46:772 [140603203516160] 3 hub: TSESS-A-6-12 [Secondary] is paired with [P-2120-12] Mar 30 15:27:46:772 [140603203516160] 1 hub: TSESS-A-6-12 [Secondary] received 200 '-' (463 bytes) Mar 30 15:27:46:772 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:27:46:772 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:27:46:772 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:27:46:772 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:27:46:772 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary same proximity=0 status=0 secver=217 Mar 30 15:27:46:772 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:772 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:772 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:772 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:772 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:772 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:772 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:27:46:772 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:772 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:772 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:772 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:772 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:772 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:772 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:772 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:772 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:27:46:772 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:27:46:772 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:27:46:772 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/33052->127.0.0.1/48009 Mar 30 15:27:46:772 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33052 Mar 30 15:27:46:772 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33052' delegated to TSESS-A-5 [140603715176192] Mar 30 15:27:46:772 [140603715176192] 3 hub: TSESS-A-5-51 [TunnelB] restarted (for local) Mar 30 15:27:46:777 [140603203516160] 5 hub: TSESS-A-6-12 [Secondary] local 100 '_close' Mar 30 15:27:46:777 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:46:777 [140603203516160] 1 hub: TSESS-A-6-12 [Secondary] sent 100 '_close' (294 bytes, 0 ms) Mar 30 15:27:46:777 [140603203516160] 3 hub: TSESS-A-6-12 [Secondary] local session closed, suspending Mar 30 15:27:46:777 [140603715176192] 5 hub: TSESS-A-5-51 [TunnelB] local 100 'hubup' Mar 30 15:27:46:777 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:46:777 [140603715176192] 1 hub: TSESS-A-5-51 [TunnelB] sent 100 'hubup' (685 bytes, 0 ms) Mar 30 15:27:46:787 [140603715176192] 1 hub: TSESS-A-5-51 [TunnelB] ssl 200 '-' (443 bytes) Mar 30 15:27:46:788 [140603715176192] 3 hub: TSESS-A-5-51 [TunnelB] is paired with [P-298-51] Mar 30 15:27:46:788 [140603715176192] 1 hub: TSESS-A-5-51 [TunnelB] received 200 '-' (451 bytes) Mar 30 15:27:46:788 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:27:46:788 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:27:46:788 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:27:46:788 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:27:46:788 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB same proximity=0 status=0 secver=217 Mar 30 15:27:46:788 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:788 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:788 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:788 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:788 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:788 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:788 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:27:46:788 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:788 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:788 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:788 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:788 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:788 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:788 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:788 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:27:46:788 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:27:46:788 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:27:46:788 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:27:46:788 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/40472->127.0.0.1/48008 Mar 30 15:27:46:788 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/40472 Mar 30 15:27:46:788 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/40472' delegated to TSESS-A-3 [140603195123456] Mar 30 15:27:46:788 [140603195123456] 3 hub: TSESS-A-3-12 [SecondaryHA] restarted (for local) Mar 30 15:27:46:793 [140603715176192] 5 hub: TSESS-A-5-51 [TunnelB] local 100 '_close' Mar 30 15:27:46:793 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:46:793 [140603715176192] 1 hub: TSESS-A-5-51 [TunnelB] sent 100 '_close' (292 bytes, 0 ms) Mar 30 15:27:46:793 [140603715176192] 3 hub: TSESS-A-5-51 [TunnelB] local session closed, suspending Mar 30 15:27:46:793 [140603195123456] 5 hub: TSESS-A-3-12 [SecondaryHA] local 100 'hubup' Mar 30 15:27:46:793 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:46:793 [140603195123456] 1 hub: TSESS-A-3-12 [SecondaryHA] sent 100 'hubup' (689 bytes, 0 ms) Mar 30 15:27:46:803 [140603195123456] 1 hub: TSESS-A-3-12 [SecondaryHA] ssl 200 '-' (459 bytes) Mar 30 15:27:46:803 [140603195123456] 3 hub: TSESS-A-3-12 [SecondaryHA] is paired with [P-1947-12] Mar 30 15:27:46:803 [140603195123456] 1 hub: TSESS-A-3-12 [SecondaryHA] received 200 '-' (467 bytes) Mar 30 15:27:46:803 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:27:46:804 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:27:46:804 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:27:46:804 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:27:46:804 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA same proximity=0 status=0 secver=217 Mar 30 15:27:46:804 [140605283890944] 5 hub: mark_hubup_list_updated Mar 30 15:27:46:804 [140605283890944] 5 hub: hublist changed: /DOMAIN/Primary 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:46:804 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:46:804 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:46:804 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:46:804 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:46:804 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:46:804 [140605283890944] 5 hub: hublist changed: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:27:46:804 [140605283890944] 5 hub: hublist changed: /DOMAIN/PrimaryHA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:46:804 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:46:804 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:46:804 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:46:804 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:46:804 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:46:804 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:46:804 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:27:46:804 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:27:46:804 [140605283890944] 5 hub: hublist changed: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:27:46:804 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelBHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:27:46:804 [140605283890944] 5 hub: mark_hubup_list_updated end Mar 30 15:27:46:804 [140605283890944] 1 hub: PERF: hubup_check_for_updates: 0 sec hubhash_size: 18 Mar 30 15:27:46:804 [140605283890944] 1 hub: PERF: hubi_run_command: 0 sec hubhash_size: 18 Mar 30 15:27:46:804 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:27:46:804 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:27:46:804 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:27:46:805 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:27:46:805 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:27:46:805 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:27:46:806 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:27:46:806 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:27:46:806 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:27:46:807 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:27:46:807 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:27:46:808 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:27:46:808 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:27:46:808 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:27:46:809 [140603195123456] 5 hub: TSESS-A-3-12 [SecondaryHA] local 100 '_close' Mar 30 15:27:46:809 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:46:809 [140603195123456] 1 hub: TSESS-A-3-12 [SecondaryHA] sent 100 '_close' (296 bytes, 0 ms) Mar 30 15:27:46:809 [140603195123456] 3 hub: TSESS-A-3-12 [SecondaryHA] local session closed, suspending Mar 30 15:27:46:809 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:27:46:809 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:27:46:810 [140605283890944] 1 hub: PERF: hubi_check_security: 0 sec hubhash_size: 18 Mar 30 15:27:46:960 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:27:46:977 [140603203516160] 5 hub: TSESS-A-6-12 [Secondary] suspended Mar 30 15:27:46:988 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:27:46:993 [140603715176192] 5 hub: TSESS-A-5-51 [TunnelB] suspended Mar 30 15:27:47:004 [140605547460672] 4 hub: medium timeout Mar 30 15:27:47:004 [140605547460672] 3 hub: next_check [1585596696] now [1585596467] Mar 30 15:27:47:004 [140605547460672] 2 hub: verify_login from ctrl 127.0.0.1/53782 Mar 30 15:27:47:004 [140605547460672] 1 hub: verify - [target] prid=controller cmd=probe_list ip=10.x.x.x perm=1 Mar 30 15:27:47:004 [140605547460672] 1 hub: verify - [source] id=ppm ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:47:010 [140603195123456] 5 hub: TSESS-A-3-12 [SecondaryHA] suspended Mar 30 15:27:47:086 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:47:249 [140603161552640] 5 hub: TSESS-P-4-12 [SecondaryHA] suspended Mar 30 15:27:47:364 [140603153159936] 5 hub: TSESS-P-8-9 [Secondary] suspended Mar 30 15:27:47:605 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:27:47:936 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:27:47:960 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:27:48:445 [140603169945344] 5 hub: TSESS-P-7-12 [TunnelB] suspended Mar 30 15:27:48:700 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:27:49:789 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:51:158 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:27:51:158 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:27:51:158 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33078 Mar 30 15:27:51:158 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33078' delegated to TSESS-A-5 [140603715176192] Mar 30 15:27:51:158 [140603715176192] 3 hub: TSESS-A-5-52 [TunnelB] restarted (for local) Mar 30 15:27:51:164 [140603715176192] 5 hub: TSESS-A-5-52 [TunnelB] local 100 'get_info' Mar 30 15:27:51:164 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:51:164 [140603715176192] 1 hub: TSESS-A-5-52 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:27:51:174 [140603715176192] 1 hub: TSESS-A-5-52 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:27:51:174 [140603715176192] 3 hub: TSESS-A-5-52 [TunnelB] is paired with [P-298-52] Mar 30 15:27:51:174 [140603715176192] 1 hub: TSESS-A-5-52 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:27:51:174 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33080 Mar 30 15:27:51:174 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33080' delegated to TSESS-A-10 [140603706783488] Mar 30 15:27:51:174 [140603706783488] 3 hub: TSESS-A-10-38 [TunnelB] restarted (for local) Mar 30 15:27:51:179 [140603715176192] 5 hub: TSESS-A-5-52 [TunnelB] local 100 '_close' Mar 30 15:27:51:179 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:51:179 [140603715176192] 1 hub: TSESS-A-5-52 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:27:51:179 [140603715176192] 3 hub: TSESS-A-5-52 [TunnelB] local session closed, suspending Mar 30 15:27:51:179 [140603706783488] 5 hub: TSESS-A-10-38 [TunnelB] local 100 '_status' Mar 30 15:27:51:179 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:51:179 [140603706783488] 1 hub: TSESS-A-10-38 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:27:51:190 [140603706783488] 1 hub: TSESS-A-10-38 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:27:51:190 [140603706783488] 3 hub: TSESS-A-10-38 [TunnelB] is paired with [P-300-38] Mar 30 15:27:51:190 [140603706783488] 1 hub: TSESS-A-10-38 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:27:51:195 [140603706783488] 5 hub: TSESS-A-10-38 [TunnelB] local 100 '_close' Mar 30 15:27:51:195 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:27:51:195 [140603706783488] 1 hub: TSESS-A-10-38 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:27:51:195 [140603706783488] 3 hub: TSESS-A-10-38 [TunnelB] local session closed, suspending Mar 30 15:27:51:380 [140603715176192] 5 hub: TSESS-A-5-52 [TunnelB] suspended Mar 30 15:27:51:395 [140603706783488] 5 hub: TSESS-A-10-38 [TunnelB] suspended Mar 30 15:27:51:591 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:53:394 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:55:196 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:56:960 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:27:56:977 [140603203516160] 5 hub: TSESS-A-6-12 [Secondary] suspended Mar 30 15:27:56:988 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:27:57:010 [140603195123456] 5 hub: TSESS-A-3-12 [SecondaryHA] suspended Mar 30 15:27:57:165 [140605547460672] 4 hub: medium timeout Mar 30 15:27:57:165 [140605547460672] 3 hub: next_check [1585596696] now [1585596477] Mar 30 15:27:57:249 [140603161552640] 5 hub: TSESS-P-4-12 [SecondaryHA] suspended Mar 30 15:27:57:365 [140603153159936] 5 hub: TSESS-P-8-9 [Secondary] suspended Mar 30 15:27:57:605 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:27:57:899 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:27:57:937 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:27:57:960 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:27:58:445 [140603169945344] 5 hub: TSESS-P-7-12 [TunnelB] suspended Mar 30 15:27:58:700 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:27:59:702 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:01:201 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:28:01:202 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:28:01:202 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33086 Mar 30 15:28:01:202 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33086' delegated to TSESS-A-5 [140603715176192] Mar 30 15:28:01:202 [140603715176192] 3 hub: TSESS-A-5-53 [TunnelB] restarted (for local) Mar 30 15:28:01:207 [140603715176192] 5 hub: TSESS-A-5-53 [TunnelB] local 100 'get_info' Mar 30 15:28:01:207 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:01:207 [140603715176192] 1 hub: TSESS-A-5-53 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:28:01:217 [140603715176192] 1 hub: TSESS-A-5-53 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:28:01:217 [140603715176192] 3 hub: TSESS-A-5-53 [TunnelB] is paired with [P-298-53] Mar 30 15:28:01:218 [140603715176192] 1 hub: TSESS-A-5-53 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:28:01:218 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33088 Mar 30 15:28:01:218 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33088' delegated to TSESS-A-10 [140603706783488] Mar 30 15:28:01:218 [140603706783488] 3 hub: TSESS-A-10-39 [TunnelB] restarted (for local) Mar 30 15:28:01:223 [140603715176192] 5 hub: TSESS-A-5-53 [TunnelB] local 100 '_close' Mar 30 15:28:01:223 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:01:223 [140603715176192] 1 hub: TSESS-A-5-53 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:28:01:223 [140603715176192] 3 hub: TSESS-A-5-53 [TunnelB] local session closed, suspending Mar 30 15:28:01:223 [140603706783488] 5 hub: TSESS-A-10-39 [TunnelB] local 100 '_status' Mar 30 15:28:01:223 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:01:223 [140603706783488] 1 hub: TSESS-A-10-39 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:28:01:233 [140603706783488] 1 hub: TSESS-A-10-39 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:28:01:233 [140603706783488] 3 hub: TSESS-A-10-39 [TunnelB] is paired with [P-300-39] Mar 30 15:28:01:233 [140603706783488] 1 hub: TSESS-A-10-39 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:28:01:238 [140603706783488] 5 hub: TSESS-A-10-39 [TunnelB] local 100 '_close' Mar 30 15:28:01:238 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:01:238 [140603706783488] 1 hub: TSESS-A-10-39 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:28:01:238 [140603706783488] 3 hub: TSESS-A-10-39 [TunnelB] local session closed, suspending Mar 30 15:28:01:423 [140603715176192] 5 hub: TSESS-A-5-53 [TunnelB] suspended Mar 30 15:28:01:439 [140603706783488] 5 hub: TSESS-A-10-39 [TunnelB] suspended Mar 30 15:28:01:504 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:03:308 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:05:110 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:05:206 [140605547460672] 3 hub: check_session - session 0x20f9050 (127.0.0.1/48002) is a server Mar 30 15:28:05:206 [140605547460672] 2 hub: Profile - clayk 38 bytes (OK) Mar 30 15:28:06:960 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:28:06:977 [140603203516160] 5 hub: TSESS-A-6-12 [Secondary] suspended Mar 30 15:28:06:988 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:28:07:010 [140603195123456] 5 hub: TSESS-A-3-12 [SecondaryHA] suspended Mar 30 15:28:07:055 [140603740354304] 3 hub: CTRL Secondary heartbeat sent on TSESS-P-21; heartbeat: 300; state[0x24] Mar 30 15:28:07:208 [140605547460672] 4 hub: medium timeout Mar 30 15:28:07:208 [140605547460672] 3 hub: next_check [1585596696] now [1585596487] Mar 30 15:28:07:250 [140603161552640] 5 hub: TSESS-P-4-12 [SecondaryHA] suspended Mar 30 15:28:07:365 [140603153159936] 5 hub: TSESS-P-8-9 [Secondary] suspended Mar 30 15:28:07:375 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:28:07:375 [140603731961600] 2 hub: CTRL SecondaryHA request 'hubup' passed to TSESS-P-4 [140603161552640] Mar 30 15:28:07:375 [140603161552640] 3 hub: TSESS-P-4-13 [SecondaryHA] restarted (for remote) Mar 30 15:28:07:375 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:38:07 Mar 30 15:28:07:376 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:28:07:376 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:28:07:376 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:28:07:376 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:28:07:376 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:28:07:376 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:28:07:376 [140603161552640] 4 hub: CONNECT: 0x7fe09c000cb0(39) 127.0.0.1/53800->127.0.0.1/48002 Mar 30 15:28:07:376 [140603161552640] 3 hub: TSESS-P-4-13 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:28:07:376 [140603161552640] 1 hub: TSESS-P-4-13 [SecondaryHA] received 100 'hubup' (1217 bytes) Mar 30 15:28:07:376 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53800 (s:920) Mar 30 15:28:07:376 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/53800] Mar 30 15:28:07:376 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:28:07:376 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:28:07:376 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/53800 Mar 30 15:28:07:376 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/53800 same proximity=0 status=0 secver=217 Mar 30 15:28:07:376 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-11] domain [DOMAIN] addr [(null)] ip [100.64.1.21] port [48002] robotname [a0f8us052nim751] version [9.20 [Build 9.20.1048, Aug 8 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-11] source [/DOMAIN/SecondaryHA] from [127.0.0.1/53800] Mar 30 15:28:07:376 [140605547460672] 4 hub: hubup source: /DOMAIN/SecondaryHA Mar 30 15:28:07:376 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-11 Mar 30 15:28:07:376 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/A0F8-USDA-FMMI-11/a0f8us052nim751/hub, from 127.0.0.1/53800 Mar 30 15:28:07:376 [140605547460672] 2 hub: hubinfo_update - DOMAIN/A0F8-USDA-FMMI-11 from=127.0.0.1/53800 CHANGED proximity=3 status=0 secver=217 Mar 30 15:28:07:376 [140605547460672] 5 hub: hubinfo_update source - lookup /DOMAIN/SecondaryHA ok Mar 30 15:28:07:376 [140605547460672] 2 hub: ch_hublist /DOMAIN/A0F8-USDA-FMMI-11/a0f8us052nim751/hub Mar 30 15:28:07:376 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:28:07:381 [140603161552640] 5 hub: TSESS-P-4-13 [SecondaryHA] local 200 'OK' Mar 30 15:28:07:381 [140603161552640] 1 hub: TSESS-P-4-13 [SecondaryHA] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:28:07:391 [140603161552640] 1 hub: TSESS-P-4-13 [SecondaryHA] ssl 100 '_close' (302 bytes) Mar 30 15:28:07:391 [140603161552640] 3 hub: TSESS-P-4-13 [SecondaryHA] is paired with [A-1948-13] Mar 30 15:28:07:391 [140603161552640] 1 hub: TSESS-P-4-13 [SecondaryHA] received 100 '_close' (298 bytes) Mar 30 15:28:07:396 [140603161552640] 1 hub: TSESS-P-4-13 [SecondaryHA] remote session closed Mar 30 15:28:07:396 [140603161552640] 3 hub: TSESS-P-4-13 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:28:07:396 [140603161552640] 5 hub: TSESS-P-4-13 [SecondaryHA] suspended Mar 30 15:28:07:487 [140603740354304] 5 hub: TSESS-P-8 not internal command - thread will handle. mtype=100; sfd:23 Mar 30 15:28:07:487 [140603740354304] 2 hub: CTRL Secondary request 'hubup' passed to TSESS-P-8 [140603153159936] Mar 30 15:28:07:487 [140603153159936] 3 hub: TSESS-P-8-10 [Secondary] restarted (for remote) Mar 30 15:28:07:487 [140603153159936] 2 hub: TSESS-P-8 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:38:07 Mar 30 15:28:07:487 [140603153159936] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:28:07:487 [140603153159936] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:28:07:487 [140603153159936] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:28:07:487 [140603153159936] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:28:07:487 [140603153159936] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:28:07:487 [140603153159936] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:28:07:487 [140603153159936] 4 hub: CONNECT: 0x7fe0a0000d90(39) 127.0.0.1/53802->127.0.0.1/48002 Mar 30 15:28:07:487 [140603153159936] 3 hub: TSESS-P-8-10 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:28:07:487 [140603153159936] 1 hub: TSESS-P-8-10 [Secondary] received 100 'hubup' (685 bytes) Mar 30 15:28:07:487 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53802 (s:388) Mar 30 15:28:07:487 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/53802] Mar 30 15:28:07:487 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:28:07:487 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:28:07:487 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/53802 Mar 30 15:28:07:487 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/53802 same proximity=0 status=0 secver=217 Mar 30 15:28:07:492 [140603153159936] 5 hub: TSESS-P-8-10 [Secondary] local 200 'OK' Mar 30 15:28:07:493 [140603153159936] 1 hub: TSESS-P-8-10 [Secondary] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:28:07:503 [140603153159936] 1 hub: TSESS-P-8-10 [Secondary] ssl 100 '_close' (298 bytes) Mar 30 15:28:07:503 [140603153159936] 3 hub: TSESS-P-8-10 [Secondary] is paired with [A-2121-10] Mar 30 15:28:07:503 [140603153159936] 1 hub: TSESS-P-8-10 [Secondary] received 100 '_close' (298 bytes) Mar 30 15:28:07:508 [140603153159936] 1 hub: TSESS-P-8-10 [Secondary] remote session closed Mar 30 15:28:07:508 [140603153159936] 3 hub: TSESS-P-8-10 [Secondary] exiting msg loop (remote close) Mar 30 15:28:07:508 [140603153159936] 5 hub: TSESS-P-8-10 [Secondary] suspended Mar 30 15:28:07:605 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:28:07:814 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:07:937 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:28:07:960 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:28:08:445 [140603169945344] 5 hub: TSESS-P-7-12 [TunnelB] suspended Mar 30 15:28:08:507 [140603723568896] 5 hub: TSESS-P-7 not internal command - thread will handle. mtype=100; sfd:22 Mar 30 15:28:08:507 [140603723568896] 2 hub: CTRL TunnelB request 'hubup' passed to TSESS-P-7 [140603169945344] Mar 30 15:28:08:507 [140603169945344] 3 hub: TSESS-P-7-13 [TunnelB] restarted (for remote) Mar 30 15:28:08:507 [140603169945344] 2 hub: TSESS-P-7 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:38:08 Mar 30 15:28:08:507 [140603169945344] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:28:08:508 [140603169945344] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:28:08:508 [140603169945344] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:28:08:508 [140603169945344] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:28:08:508 [140603169945344] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:28:08:508 [140603169945344] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:28:08:508 [140603169945344] 4 hub: CONNECT: 0x7fe0a8000d20(39) 127.0.0.1/53804->127.0.0.1/48002 Mar 30 15:28:08:508 [140603169945344] 3 hub: TSESS-P-7-13 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:28:08:508 [140603169945344] 1 hub: TSESS-P-7-13 [TunnelB] received 100 'hubup' (1197 bytes) Mar 30 15:28:08:508 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53804 (s:902) Mar 30 15:28:08:508 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/53804] Mar 30 15:28:08:508 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:28:08:508 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:28:08:508 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/53804 Mar 30 15:28:08:508 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/53804 same proximity=0 status=0 secver=217 Mar 30 15:28:08:508 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-11] domain [DOMAIN] addr [(null)] ip [100.64.1.21] port [48002] robotname [a0f8us052nim751] version [9.20 [Build 9.20.1048, Aug 8 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-11] source [/DOMAIN/TunnelB] from [127.0.0.1/53804] Mar 30 15:28:08:508 [140605547460672] 4 hub: hubup source: /DOMAIN/TunnelB Mar 30 15:28:08:508 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-11 Mar 30 15:28:08:508 [140605547460672] 5 hub: hubinfo_update - for /DOMAIN/A0F8-USDA-FMMI-11/a0f8us052nim751/hub from 127.0.0.1/53804, proximity 4 higher than current 3, ignore update.... Mar 30 15:28:08:513 [140603169945344] 5 hub: TSESS-P-7-13 [TunnelB] local 200 'OK' Mar 30 15:28:08:513 [140603169945344] 1 hub: TSESS-P-7-13 [TunnelB] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:28:08:523 [140603169945344] 1 hub: TSESS-P-7-13 [TunnelB] ssl 100 '_close' (284 bytes) Mar 30 15:28:08:523 [140603169945344] 3 hub: TSESS-P-7-13 [TunnelB] is paired with [A-299-13] Mar 30 15:28:08:523 [140603169945344] 1 hub: TSESS-P-7-13 [TunnelB] received 100 '_close' (296 bytes) Mar 30 15:28:08:528 [140603169945344] 1 hub: TSESS-P-7-13 [TunnelB] remote session closed Mar 30 15:28:08:528 [140603169945344] 3 hub: TSESS-P-7-13 [TunnelB] exiting msg loop (remote close) Mar 30 15:28:08:528 [140603169945344] 5 hub: TSESS-P-7-13 [TunnelB] suspended Mar 30 15:28:08:700 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:28:09:616 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:11:243 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:28:11:243 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:28:11:243 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33100 Mar 30 15:28:11:243 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33100' delegated to TSESS-A-5 [140603715176192] Mar 30 15:28:11:244 [140603715176192] 3 hub: TSESS-A-5-54 [TunnelB] restarted (for local) Mar 30 15:28:11:249 [140603715176192] 5 hub: TSESS-A-5-54 [TunnelB] local 100 'get_info' Mar 30 15:28:11:249 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:11:249 [140603715176192] 1 hub: TSESS-A-5-54 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:28:11:259 [140603715176192] 1 hub: TSESS-A-5-54 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:28:11:259 [140603715176192] 3 hub: TSESS-A-5-54 [TunnelB] is paired with [P-298-54] Mar 30 15:28:11:259 [140603715176192] 1 hub: TSESS-A-5-54 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:28:11:259 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33102 Mar 30 15:28:11:259 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33102' delegated to TSESS-A-10 [140603706783488] Mar 30 15:28:11:259 [140603706783488] 3 hub: TSESS-A-10-40 [TunnelB] restarted (for local) Mar 30 15:28:11:264 [140603715176192] 5 hub: TSESS-A-5-54 [TunnelB] local 100 '_close' Mar 30 15:28:11:264 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:11:264 [140603715176192] 1 hub: TSESS-A-5-54 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:28:11:264 [140603715176192] 3 hub: TSESS-A-5-54 [TunnelB] local session closed, suspending Mar 30 15:28:11:264 [140603706783488] 5 hub: TSESS-A-10-40 [TunnelB] local 100 '_status' Mar 30 15:28:11:264 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:11:264 [140603706783488] 1 hub: TSESS-A-10-40 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:28:11:275 [140603706783488] 1 hub: TSESS-A-10-40 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:28:11:275 [140603706783488] 3 hub: TSESS-A-10-40 [TunnelB] is paired with [P-300-40] Mar 30 15:28:11:275 [140603706783488] 1 hub: TSESS-A-10-40 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:28:11:280 [140603706783488] 5 hub: TSESS-A-10-40 [TunnelB] local 100 '_close' Mar 30 15:28:11:280 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:11:280 [140603706783488] 1 hub: TSESS-A-10-40 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:28:11:280 [140603706783488] 3 hub: TSESS-A-10-40 [TunnelB] local session closed, suspending Mar 30 15:28:11:419 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:11:465 [140603715176192] 5 hub: TSESS-A-5-54 [TunnelB] suspended Mar 30 15:28:11:480 [140603706783488] 5 hub: TSESS-A-10-40 [TunnelB] suspended Mar 30 15:28:13:221 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:15:024 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:16:841 [140605283890944] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:28:16:841 [140605283890944] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:28:16:841 [140605283890944] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:28:16:841 [140605283890944] 3 hub: hubhash_write - 0 hubs saved to cache Mar 30 15:28:16:841 [140605283890944] 1 hub: PERF: hubup_check_interval: 0 sec hubhash_size: 18 Mar 30 15:28:16:841 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:842 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:842 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:842 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:842 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:842 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:842 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:28:16:842 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:842 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:842 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:842 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:842 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:842 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:842 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:842 [140605283890944] 2 hub: hubupdatelist->Secondary[+]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:16:842 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:28:16:842 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:28:16:842 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:28:16:842 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/47478->127.0.0.1/48007 Mar 30 15:28:16:842 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/47478 Mar 30 15:28:16:842 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/47478' delegated to TSESS-A-6 [140603203516160] Mar 30 15:28:16:842 [140603203516160] 3 hub: TSESS-A-6-13 [Secondary] restarted (for local) Mar 30 15:28:16:847 [140603203516160] 5 hub: TSESS-A-6-13 [Secondary] local 100 'hubup' Mar 30 15:28:16:847 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:16:847 [140603203516160] 1 hub: TSESS-A-6-13 [Secondary] sent 100 'hubup' (1213 bytes, 0 ms) Mar 30 15:28:16:857 [140603203516160] 1 hub: TSESS-A-6-13 [Secondary] ssl 200 '-' (455 bytes) Mar 30 15:28:16:857 [140603203516160] 3 hub: TSESS-A-6-13 [Secondary] is paired with [P-2120-13] Mar 30 15:28:16:857 [140603203516160] 1 hub: TSESS-A-6-13 [Secondary] received 200 '-' (463 bytes) Mar 30 15:28:16:857 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:28:16:857 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:28:16:857 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:28:16:857 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:28:16:857 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary same proximity=0 status=0 secver=217 Mar 30 15:28:16:858 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:858 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:858 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:858 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:858 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:858 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:858 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:28:16:858 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:858 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:858 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:858 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:858 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:858 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:858 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:858 [140605283890944] 2 hub: hubupdatelist->TunnelB[+]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:16:858 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:28:16:858 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:28:16:858 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:28:16:858 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/33110->127.0.0.1/48009 Mar 30 15:28:16:858 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33110 Mar 30 15:28:16:858 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33110' delegated to TSESS-A-5 [140603715176192] Mar 30 15:28:16:858 [140603715176192] 3 hub: TSESS-A-5-55 [TunnelB] restarted (for local) Mar 30 15:28:16:862 [140603203516160] 5 hub: TSESS-A-6-13 [Secondary] local 100 '_close' Mar 30 15:28:16:863 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:16:863 [140603203516160] 1 hub: TSESS-A-6-13 [Secondary] sent 100 '_close' (294 bytes, 0 ms) Mar 30 15:28:16:863 [140603203516160] 3 hub: TSESS-A-6-13 [Secondary] local session closed, suspending Mar 30 15:28:16:863 [140603715176192] 5 hub: TSESS-A-5-55 [TunnelB] local 100 'hubup' Mar 30 15:28:16:863 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:16:863 [140603715176192] 1 hub: TSESS-A-5-55 [TunnelB] sent 100 'hubup' (1211 bytes, 0 ms) Mar 30 15:28:16:873 [140603715176192] 1 hub: TSESS-A-5-55 [TunnelB] ssl 200 '-' (443 bytes) Mar 30 15:28:16:873 [140603715176192] 3 hub: TSESS-A-5-55 [TunnelB] is paired with [P-298-55] Mar 30 15:28:16:873 [140603715176192] 1 hub: TSESS-A-5-55 [TunnelB] received 200 '-' (451 bytes) Mar 30 15:28:16:873 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:28:16:873 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:28:16:873 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:28:16:873 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:28:16:873 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB same proximity=0 status=0 secver=217 Mar 30 15:28:16:873 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:873 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:873 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:873 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:873 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:873 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:873 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:28:16:873 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:873 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:873 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:873 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:873 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:873 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:873 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:873 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:16:873 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:28:16:873 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:28:16:873 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:28:16:874 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/40530->127.0.0.1/48008 Mar 30 15:28:16:874 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/40530 Mar 30 15:28:16:874 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/40530' delegated to TSESS-A-3 [140603195123456] Mar 30 15:28:16:874 [140603195123456] 3 hub: TSESS-A-3-13 [SecondaryHA] restarted (for local) Mar 30 15:28:16:878 [140603715176192] 5 hub: TSESS-A-5-55 [TunnelB] local 100 '_close' Mar 30 15:28:16:878 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:16:878 [140603715176192] 1 hub: TSESS-A-5-55 [TunnelB] sent 100 '_close' (292 bytes, 0 ms) Mar 30 15:28:16:878 [140603715176192] 3 hub: TSESS-A-5-55 [TunnelB] local session closed, suspending Mar 30 15:28:16:879 [140603195123456] 5 hub: TSESS-A-3-13 [SecondaryHA] local 100 'hubup' Mar 30 15:28:16:879 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:16:879 [140603195123456] 1 hub: TSESS-A-3-13 [SecondaryHA] sent 100 'hubup' (689 bytes, 0 ms) Mar 30 15:28:16:889 [140603195123456] 1 hub: TSESS-A-3-13 [SecondaryHA] ssl 200 '-' (459 bytes) Mar 30 15:28:16:889 [140603195123456] 3 hub: TSESS-A-3-13 [SecondaryHA] is paired with [P-1947-13] Mar 30 15:28:16:889 [140603195123456] 1 hub: TSESS-A-3-13 [SecondaryHA] received 200 '-' (467 bytes) Mar 30 15:28:16:889 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:28:16:889 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:28:16:889 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:28:16:889 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:28:16:889 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA same proximity=0 status=0 secver=217 Mar 30 15:28:16:889 [140605283890944] 5 hub: mark_hubup_list_updated Mar 30 15:28:16:889 [140605283890944] 5 hub: hublist changed: /DOMAIN/Primary 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:16:889 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:16:889 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:16:889 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:16:889 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:16:889 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:16:889 [140605283890944] 5 hub: hublist changed: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:28:16:889 [140605283890944] 5 hub: hublist changed: /DOMAIN/PrimaryHA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:16:889 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:16:889 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:16:889 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:16:889 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:16:889 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:16:889 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:16:889 [140605283890944] 5 hub: hublist chnaged reset: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:16:889 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:28:16:889 [140605283890944] 5 hub: hublist changed: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:28:16:889 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelBHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:28:16:889 [140605283890944] 5 hub: mark_hubup_list_updated end Mar 30 15:28:16:889 [140605283890944] 1 hub: PERF: hubup_check_for_updates: 0 sec hubhash_size: 18 Mar 30 15:28:16:889 [140605283890944] 1 hub: PERF: hubi_run_command: 0 sec hubhash_size: 18 Mar 30 15:28:16:890 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:28:16:890 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:28:16:890 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:28:16:891 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:28:16:891 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:28:16:892 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:28:16:892 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:28:16:893 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:28:16:893 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:28:16:894 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:28:16:894 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:28:16:894 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:28:16:894 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:28:16:894 [140603195123456] 5 hub: TSESS-A-3-13 [SecondaryHA] local 100 '_close' Mar 30 15:28:16:894 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:16:894 [140603195123456] 1 hub: TSESS-A-3-13 [SecondaryHA] sent 100 '_close' (296 bytes, 0 ms) Mar 30 15:28:16:894 [140603195123456] 3 hub: TSESS-A-3-13 [SecondaryHA] local session closed, suspending Mar 30 15:28:16:895 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:28:16:895 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:28:16:896 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:28:16:897 [140605283890944] 1 hub: PERF: hubi_check_security: 0 sec hubhash_size: 18 Mar 30 15:28:16:960 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:28:16:988 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:28:17:063 [140603203516160] 5 hub: TSESS-A-6-13 [Secondary] suspended Mar 30 15:28:17:079 [140603715176192] 5 hub: TSESS-A-5-55 [TunnelB] suspended Mar 30 15:28:17:095 [140603195123456] 5 hub: TSESS-A-3-13 [SecondaryHA] suspended Mar 30 15:28:17:253 [140605547460672] 4 hub: medium timeout Mar 30 15:28:17:253 [140605547460672] 3 hub: next_check [1585596696] now [1585596497] Mar 30 15:28:17:397 [140603161552640] 5 hub: TSESS-P-4-13 [SecondaryHA] suspended Mar 30 15:28:17:508 [140603153159936] 5 hub: TSESS-P-8-10 [Secondary] suspended Mar 30 15:28:17:605 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:28:17:727 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:17:937 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:28:17:960 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:28:18:529 [140603169945344] 5 hub: TSESS-P-7-13 [TunnelB] suspended Mar 30 15:28:18:700 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:28:19:530 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:21:286 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:28:21:286 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:28:21:286 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33120 Mar 30 15:28:21:286 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33120' delegated to TSESS-A-5 [140603715176192] Mar 30 15:28:21:286 [140603715176192] 3 hub: TSESS-A-5-56 [TunnelB] restarted (for local) Mar 30 15:28:21:292 [140603715176192] 5 hub: TSESS-A-5-56 [TunnelB] local 100 'get_info' Mar 30 15:28:21:292 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:21:292 [140603715176192] 1 hub: TSESS-A-5-56 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:28:21:302 [140603715176192] 1 hub: TSESS-A-5-56 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:28:21:302 [140603715176192] 3 hub: TSESS-A-5-56 [TunnelB] is paired with [P-298-56] Mar 30 15:28:21:302 [140603715176192] 1 hub: TSESS-A-5-56 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:28:21:302 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33122 Mar 30 15:28:21:302 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33122' delegated to TSESS-A-10 [140603706783488] Mar 30 15:28:21:302 [140603706783488] 3 hub: TSESS-A-10-41 [TunnelB] restarted (for local) Mar 30 15:28:21:307 [140603715176192] 5 hub: TSESS-A-5-56 [TunnelB] local 100 '_close' Mar 30 15:28:21:307 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:21:307 [140603715176192] 1 hub: TSESS-A-5-56 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:28:21:307 [140603715176192] 3 hub: TSESS-A-5-56 [TunnelB] local session closed, suspending Mar 30 15:28:21:307 [140603706783488] 5 hub: TSESS-A-10-41 [TunnelB] local 100 '_status' Mar 30 15:28:21:307 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:21:307 [140603706783488] 1 hub: TSESS-A-10-41 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:28:21:318 [140603706783488] 1 hub: TSESS-A-10-41 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:28:21:318 [140603706783488] 3 hub: TSESS-A-10-41 [TunnelB] is paired with [P-300-41] Mar 30 15:28:21:318 [140603706783488] 1 hub: TSESS-A-10-41 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:28:21:323 [140603706783488] 5 hub: TSESS-A-10-41 [TunnelB] local 100 '_close' Mar 30 15:28:21:323 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:21:323 [140603706783488] 1 hub: TSESS-A-10-41 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:28:21:323 [140603706783488] 3 hub: TSESS-A-10-41 [TunnelB] local session closed, suspending Mar 30 15:28:21:332 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:21:507 [140603715176192] 5 hub: TSESS-A-5-56 [TunnelB] suspended Mar 30 15:28:21:523 [140603706783488] 5 hub: TSESS-A-10-41 [TunnelB] suspended Mar 30 15:28:23:135 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:25:838 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:26:960 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:28:26:988 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:28:27:063 [140603203516160] 5 hub: TSESS-A-6-13 [Secondary] suspended Mar 30 15:28:27:095 [140603195123456] 5 hub: TSESS-A-3-13 [SecondaryHA] suspended Mar 30 15:28:27:293 [140605547460672] 4 hub: medium timeout Mar 30 15:28:27:293 [140605547460672] 3 hub: next_check [1585596696] now [1585596507] Mar 30 15:28:27:397 [140603161552640] 5 hub: TSESS-P-4-13 [SecondaryHA] suspended Mar 30 15:28:27:509 [140603153159936] 5 hub: TSESS-P-8-10 [Secondary] suspended Mar 30 15:28:27:605 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:28:27:641 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:27:937 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:28:27:960 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:28:28:529 [140603169945344] 5 hub: TSESS-P-7-13 [TunnelB] suspended Mar 30 15:28:28:700 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:28:29:443 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:31:246 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:31:328 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:28:31:328 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:28:31:328 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33128 Mar 30 15:28:31:328 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33128' delegated to TSESS-A-5 [140603715176192] Mar 30 15:28:31:328 [140603715176192] 3 hub: TSESS-A-5-57 [TunnelB] restarted (for local) Mar 30 15:28:31:333 [140603715176192] 5 hub: TSESS-A-5-57 [TunnelB] local 100 'get_info' Mar 30 15:28:31:334 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:31:334 [140603715176192] 1 hub: TSESS-A-5-57 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:28:31:344 [140603715176192] 1 hub: TSESS-A-5-57 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:28:31:344 [140603715176192] 3 hub: TSESS-A-5-57 [TunnelB] is paired with [P-298-57] Mar 30 15:28:31:344 [140603715176192] 1 hub: TSESS-A-5-57 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:28:31:344 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33130 Mar 30 15:28:31:344 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33130' delegated to TSESS-A-10 [140603706783488] Mar 30 15:28:31:344 [140603706783488] 3 hub: TSESS-A-10-42 [TunnelB] restarted (for local) Mar 30 15:28:31:349 [140603715176192] 5 hub: TSESS-A-5-57 [TunnelB] local 100 '_close' Mar 30 15:28:31:349 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:31:349 [140603715176192] 1 hub: TSESS-A-5-57 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:28:31:349 [140603715176192] 3 hub: TSESS-A-5-57 [TunnelB] local session closed, suspending Mar 30 15:28:31:349 [140603706783488] 5 hub: TSESS-A-10-42 [TunnelB] local 100 '_status' Mar 30 15:28:31:349 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:31:349 [140603706783488] 1 hub: TSESS-A-10-42 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:28:31:360 [140603706783488] 1 hub: TSESS-A-10-42 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:28:31:360 [140603706783488] 3 hub: TSESS-A-10-42 [TunnelB] is paired with [P-300-42] Mar 30 15:28:31:360 [140603706783488] 1 hub: TSESS-A-10-42 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:28:31:365 [140603706783488] 5 hub: TSESS-A-10-42 [TunnelB] local 100 '_close' Mar 30 15:28:31:365 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:31:365 [140603706783488] 1 hub: TSESS-A-10-42 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:28:31:365 [140603706783488] 3 hub: TSESS-A-10-42 [TunnelB] local session closed, suspending Mar 30 15:28:31:549 [140603715176192] 5 hub: TSESS-A-5-57 [TunnelB] suspended Mar 30 15:28:31:565 [140603706783488] 5 hub: TSESS-A-10-42 [TunnelB] suspended Mar 30 15:28:33:048 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:34:065 [140604042360576] 5 hub: There are 0 objects waiting on 8 threads Mar 30 15:28:35:332 [140605547460672] 3 hub: check_session - session 0x20f9050 (127.0.0.1/48002) is a server Mar 30 15:28:35:332 [140605547460672] 2 hub: Profile - clayk 38 bytes (OK) Mar 30 15:28:35:752 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:36:960 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:28:36:989 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:28:37:063 [140603203516160] 5 hub: TSESS-A-6-13 [Secondary] suspended Mar 30 15:28:37:095 [140603195123456] 5 hub: TSESS-A-3-13 [SecondaryHA] suspended Mar 30 15:28:37:335 [140605547460672] 4 hub: medium timeout Mar 30 15:28:37:335 [140605547460672] 3 hub: next_check [1585596696] now [1585596517] Mar 30 15:28:37:397 [140603161552640] 5 hub: TSESS-P-4-13 [SecondaryHA] suspended Mar 30 15:28:37:499 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:28:37:499 [140603731961600] 2 hub: CTRL SecondaryHA request 'hubup' passed to TSESS-P-4 [140603161552640] Mar 30 15:28:37:499 [140603161552640] 3 hub: TSESS-P-4-14 [SecondaryHA] restarted (for remote) Mar 30 15:28:37:499 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:38:37 Mar 30 15:28:37:499 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:28:37:499 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:28:37:499 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:28:37:499 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:28:37:499 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:28:37:499 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:28:37:499 [140603161552640] 4 hub: CONNECT: 0x7fe09c000c60(39) 127.0.0.1/53842->127.0.0.1/48002 Mar 30 15:28:37:499 [140603161552640] 3 hub: TSESS-P-4-14 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:28:37:499 [140603161552640] 1 hub: TSESS-P-4-14 [SecondaryHA] received 100 'hubup' (689 bytes) Mar 30 15:28:37:499 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53842 (s:392) Mar 30 15:28:37:499 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/53842] Mar 30 15:28:37:499 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:28:37:499 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:28:37:499 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/53842 Mar 30 15:28:37:499 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/53842 same proximity=0 status=0 secver=217 Mar 30 15:28:37:504 [140603161552640] 5 hub: TSESS-P-4-14 [SecondaryHA] local 200 'OK' Mar 30 15:28:37:505 [140603161552640] 1 hub: TSESS-P-4-14 [SecondaryHA] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:28:37:509 [140603153159936] 5 hub: TSESS-P-8-10 [Secondary] suspended Mar 30 15:28:37:515 [140603161552640] 1 hub: TSESS-P-4-14 [SecondaryHA] ssl 100 '_close' (302 bytes) Mar 30 15:28:37:515 [140603161552640] 3 hub: TSESS-P-4-14 [SecondaryHA] is paired with [A-1948-14] Mar 30 15:28:37:515 [140603161552640] 1 hub: TSESS-P-4-14 [SecondaryHA] received 100 '_close' (298 bytes) Mar 30 15:28:37:520 [140603161552640] 1 hub: TSESS-P-4-14 [SecondaryHA] remote session closed Mar 30 15:28:37:520 [140603161552640] 3 hub: TSESS-P-4-14 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:28:37:520 [140603161552640] 5 hub: TSESS-P-4-14 [SecondaryHA] suspended Mar 30 15:28:37:554 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:37:606 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:28:37:656 [140603740354304] 5 hub: TSESS-P-8 not internal command - thread will handle. mtype=100; sfd:23 Mar 30 15:28:37:656 [140603740354304] 2 hub: CTRL Secondary request 'hubup' passed to TSESS-P-8 [140603153159936] Mar 30 15:28:37:656 [140603153159936] 3 hub: TSESS-P-8-11 [Secondary] restarted (for remote) Mar 30 15:28:37:656 [140603153159936] 2 hub: TSESS-P-8 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:38:37 Mar 30 15:28:37:656 [140603153159936] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:28:37:656 [140603153159936] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:28:37:656 [140603153159936] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:28:37:656 [140603153159936] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:28:37:656 [140603153159936] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:28:37:656 [140603153159936] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:28:37:656 [140603153159936] 4 hub: CONNECT: 0x7fe0a0000d90(39) 127.0.0.1/53844->127.0.0.1/48002 Mar 30 15:28:37:656 [140603153159936] 3 hub: TSESS-P-8-11 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:28:37:656 [140603153159936] 1 hub: TSESS-P-8-11 [Secondary] received 100 'hubup' (685 bytes) Mar 30 15:28:37:657 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53844 (s:388) Mar 30 15:28:37:657 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/53844] Mar 30 15:28:37:657 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:28:37:657 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:28:37:657 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/53844 Mar 30 15:28:37:657 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/53844 same proximity=0 status=0 secver=217 Mar 30 15:28:37:662 [140603153159936] 5 hub: TSESS-P-8-11 [Secondary] local 200 'OK' Mar 30 15:28:37:662 [140603153159936] 1 hub: TSESS-P-8-11 [Secondary] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:28:37:672 [140603153159936] 1 hub: TSESS-P-8-11 [Secondary] ssl 100 '_close' (298 bytes) Mar 30 15:28:37:672 [140603153159936] 3 hub: TSESS-P-8-11 [Secondary] is paired with [A-2121-11] Mar 30 15:28:37:672 [140603153159936] 1 hub: TSESS-P-8-11 [Secondary] received 100 '_close' (298 bytes) Mar 30 15:28:37:677 [140603153159936] 1 hub: TSESS-P-8-11 [Secondary] remote session closed Mar 30 15:28:37:677 [140603153159936] 3 hub: TSESS-P-8-11 [Secondary] exiting msg loop (remote close) Mar 30 15:28:37:677 [140603153159936] 5 hub: TSESS-P-8-11 [Secondary] suspended Mar 30 15:28:37:937 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:28:37:960 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:28:38:529 [140603169945344] 5 hub: TSESS-P-7-13 [TunnelB] suspended Mar 30 15:28:38:630 [140603723568896] 5 hub: TSESS-P-7 not internal command - thread will handle. mtype=100; sfd:22 Mar 30 15:28:38:630 [140603723568896] 2 hub: CTRL TunnelB request 'hubup' passed to TSESS-P-7 [140603169945344] Mar 30 15:28:38:630 [140603169945344] 3 hub: TSESS-P-7-14 [TunnelB] restarted (for remote) Mar 30 15:28:38:630 [140603169945344] 2 hub: TSESS-P-7 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:38:38 Mar 30 15:28:38:630 [140603169945344] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:28:38:630 [140603169945344] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:28:38:630 [140603169945344] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:28:38:631 [140603169945344] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:28:38:631 [140603169945344] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:28:38:631 [140603169945344] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:28:38:631 [140603169945344] 4 hub: CONNECT: 0x7fe0a8000d20(39) 127.0.0.1/53846->127.0.0.1/48002 Mar 30 15:28:38:631 [140603169945344] 3 hub: TSESS-P-7-14 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:28:38:631 [140603169945344] 1 hub: TSESS-P-7-14 [TunnelB] received 100 'hubup' (673 bytes) Mar 30 15:28:38:631 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53846 (s:378) Mar 30 15:28:38:631 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/53846] Mar 30 15:28:38:631 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:28:38:631 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:28:38:631 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/53846 Mar 30 15:28:38:631 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/53846 same proximity=0 status=0 secver=217 Mar 30 15:28:38:636 [140603169945344] 5 hub: TSESS-P-7-14 [TunnelB] local 200 'OK' Mar 30 15:28:38:636 [140603169945344] 1 hub: TSESS-P-7-14 [TunnelB] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:28:38:646 [140603169945344] 1 hub: TSESS-P-7-14 [TunnelB] ssl 100 '_close' (284 bytes) Mar 30 15:28:38:646 [140603169945344] 3 hub: TSESS-P-7-14 [TunnelB] is paired with [A-299-14] Mar 30 15:28:38:646 [140603169945344] 1 hub: TSESS-P-7-14 [TunnelB] received 100 '_close' (296 bytes) Mar 30 15:28:38:651 [140603169945344] 1 hub: TSESS-P-7-14 [TunnelB] remote session closed Mar 30 15:28:38:651 [140603169945344] 3 hub: TSESS-P-7-14 [TunnelB] exiting msg loop (remote close) Mar 30 15:28:38:651 [140603169945344] 5 hub: TSESS-P-7-14 [TunnelB] suspended Mar 30 15:28:38:700 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:28:39:357 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:41:159 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:41:371 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:28:41:371 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:28:41:371 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33142 Mar 30 15:28:41:371 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33142' delegated to TSESS-A-5 [140603715176192] Mar 30 15:28:41:372 [140603715176192] 3 hub: TSESS-A-5-58 [TunnelB] restarted (for local) Mar 30 15:28:41:377 [140603715176192] 5 hub: TSESS-A-5-58 [TunnelB] local 100 'get_info' Mar 30 15:28:41:377 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:41:377 [140603715176192] 1 hub: TSESS-A-5-58 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:28:41:387 [140603715176192] 1 hub: TSESS-A-5-58 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:28:41:387 [140603715176192] 3 hub: TSESS-A-5-58 [TunnelB] is paired with [P-298-58] Mar 30 15:28:41:387 [140603715176192] 1 hub: TSESS-A-5-58 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:28:41:387 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33144 Mar 30 15:28:41:387 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33144' delegated to TSESS-A-10 [140603706783488] Mar 30 15:28:41:387 [140603706783488] 3 hub: TSESS-A-10-43 [TunnelB] restarted (for local) Mar 30 15:28:41:392 [140603715176192] 5 hub: TSESS-A-5-58 [TunnelB] local 100 '_close' Mar 30 15:28:41:392 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:41:392 [140603715176192] 1 hub: TSESS-A-5-58 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:28:41:392 [140603715176192] 3 hub: TSESS-A-5-58 [TunnelB] local session closed, suspending Mar 30 15:28:41:393 [140603706783488] 5 hub: TSESS-A-10-43 [TunnelB] local 100 '_status' Mar 30 15:28:41:393 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:41:393 [140603706783488] 1 hub: TSESS-A-10-43 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:28:41:403 [140603706783488] 1 hub: TSESS-A-10-43 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:28:41:403 [140603706783488] 3 hub: TSESS-A-10-43 [TunnelB] is paired with [P-300-43] Mar 30 15:28:41:403 [140603706783488] 1 hub: TSESS-A-10-43 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:28:41:408 [140603706783488] 5 hub: TSESS-A-10-43 [TunnelB] local 100 '_close' Mar 30 15:28:41:408 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:41:408 [140603706783488] 1 hub: TSESS-A-10-43 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:28:41:408 [140603706783488] 3 hub: TSESS-A-10-43 [TunnelB] local session closed, suspending Mar 30 15:28:41:593 [140603715176192] 5 hub: TSESS-A-5-58 [TunnelB] suspended Mar 30 15:28:41:608 [140603706783488] 5 hub: TSESS-A-10-43 [TunnelB] suspended Mar 30 15:28:43:863 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:45:665 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:46:925 [140605283890944] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:28:46:925 [140605283890944] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:28:46:926 [140605283890944] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:28:46:926 [140605283890944] 1 hub: PERF: hubup_check_interval: 0 sec hubhash_size: 18 Mar 30 15:28:46:926 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:926 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:926 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:926 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:926 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:926 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:926 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:28:46:926 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:926 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:926 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:926 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:926 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:926 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:926 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:926 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:926 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:28:46:926 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:28:46:926 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:28:46:926 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/47520->127.0.0.1/48007 Mar 30 15:28:46:926 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/47520 Mar 30 15:28:46:926 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/47520' delegated to TSESS-A-6 [140603203516160] Mar 30 15:28:46:926 [140603203516160] 3 hub: TSESS-A-6-14 [Secondary] restarted (for local) Mar 30 15:28:46:931 [140603203516160] 5 hub: TSESS-A-6-14 [Secondary] local 100 'hubup' Mar 30 15:28:46:931 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:46:931 [140603203516160] 1 hub: TSESS-A-6-14 [Secondary] sent 100 'hubup' (687 bytes, 0 ms) Mar 30 15:28:46:941 [140603203516160] 1 hub: TSESS-A-6-14 [Secondary] ssl 200 '-' (455 bytes) Mar 30 15:28:46:941 [140603203516160] 3 hub: TSESS-A-6-14 [Secondary] is paired with [P-2120-14] Mar 30 15:28:46:941 [140603203516160] 1 hub: TSESS-A-6-14 [Secondary] received 200 '-' (463 bytes) Mar 30 15:28:46:942 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:28:46:942 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:28:46:942 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:28:46:942 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:28:46:942 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary same proximity=0 status=0 secver=217 Mar 30 15:28:46:942 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:942 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:942 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:942 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:942 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:942 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:942 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:28:46:942 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:942 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:942 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:942 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:942 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:942 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:942 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:942 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:942 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:28:46:942 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:28:46:942 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:28:46:942 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/33152->127.0.0.1/48009 Mar 30 15:28:46:942 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33152 Mar 30 15:28:46:942 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33152' delegated to TSESS-A-5 [140603715176192] Mar 30 15:28:46:942 [140603715176192] 3 hub: TSESS-A-5-59 [TunnelB] restarted (for local) Mar 30 15:28:46:947 [140603203516160] 5 hub: TSESS-A-6-14 [Secondary] local 100 '_close' Mar 30 15:28:46:947 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:46:947 [140603203516160] 1 hub: TSESS-A-6-14 [Secondary] sent 100 '_close' (294 bytes, 0 ms) Mar 30 15:28:46:947 [140603203516160] 3 hub: TSESS-A-6-14 [Secondary] local session closed, suspending Mar 30 15:28:46:947 [140603715176192] 5 hub: TSESS-A-5-59 [TunnelB] local 100 'hubup' Mar 30 15:28:46:947 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:46:947 [140603715176192] 1 hub: TSESS-A-5-59 [TunnelB] sent 100 'hubup' (685 bytes, 0 ms) Mar 30 15:28:46:957 [140603715176192] 1 hub: TSESS-A-5-59 [TunnelB] ssl 200 '-' (443 bytes) Mar 30 15:28:46:957 [140603715176192] 3 hub: TSESS-A-5-59 [TunnelB] is paired with [P-298-59] Mar 30 15:28:46:957 [140603715176192] 1 hub: TSESS-A-5-59 [TunnelB] received 200 '-' (451 bytes) Mar 30 15:28:46:958 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:28:46:958 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:28:46:958 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:28:46:958 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:28:46:958 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB same proximity=0 status=0 secver=217 Mar 30 15:28:46:958 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:958 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:958 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:958 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:958 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:958 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:958 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:28:46:958 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:958 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:958 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:958 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:958 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:958 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:958 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:958 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:28:46:958 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:28:46:958 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:28:46:958 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:28:46:958 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(39) 127.0.0.1/40572->127.0.0.1/48008 Mar 30 15:28:46:958 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/40572 Mar 30 15:28:46:958 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/40572' delegated to TSESS-A-3 [140603195123456] Mar 30 15:28:46:958 [140603195123456] 3 hub: TSESS-A-3-14 [SecondaryHA] restarted (for local) Mar 30 15:28:46:961 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:28:46:963 [140603715176192] 5 hub: TSESS-A-5-59 [TunnelB] local 100 '_close' Mar 30 15:28:46:963 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:46:963 [140603715176192] 1 hub: TSESS-A-5-59 [TunnelB] sent 100 '_close' (292 bytes, 0 ms) Mar 30 15:28:46:963 [140603715176192] 3 hub: TSESS-A-5-59 [TunnelB] local session closed, suspending Mar 30 15:28:46:963 [140603195123456] 5 hub: TSESS-A-3-14 [SecondaryHA] local 100 'hubup' Mar 30 15:28:46:963 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:46:963 [140603195123456] 1 hub: TSESS-A-3-14 [SecondaryHA] sent 100 'hubup' (689 bytes, 0 ms) Mar 30 15:28:46:974 [140603195123456] 1 hub: TSESS-A-3-14 [SecondaryHA] ssl 200 '-' (459 bytes) Mar 30 15:28:46:974 [140603195123456] 3 hub: TSESS-A-3-14 [SecondaryHA] is paired with [P-1947-14] Mar 30 15:28:46:974 [140603195123456] 1 hub: TSESS-A-3-14 [SecondaryHA] received 200 '-' (467 bytes) Mar 30 15:28:46:974 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:28:46:974 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:28:46:974 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:28:46:974 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:28:46:974 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA same proximity=0 status=0 secver=217 Mar 30 15:28:46:974 [140605283890944] 5 hub: mark_hubup_list_updated Mar 30 15:28:46:974 [140605283890944] 5 hub: hublist changed: /DOMAIN/Primary 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:46:974 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:46:974 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:46:974 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:46:974 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:46:974 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:46:974 [140605283890944] 5 hub: hublist changed: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:28:46:974 [140605283890944] 5 hub: hublist changed: /DOMAIN/PrimaryHA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:46:974 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:46:974 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:46:974 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:46:974 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:46:974 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:46:974 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:46:974 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:28:46:974 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:28:46:974 [140605283890944] 5 hub: hublist changed: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:28:46:974 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelBHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:28:46:974 [140605283890944] 5 hub: mark_hubup_list_updated end Mar 30 15:28:46:974 [140605283890944] 1 hub: PERF: hubup_check_for_updates: 0 sec hubhash_size: 18 Mar 30 15:28:46:974 [140605283890944] 1 hub: PERF: hubi_run_command: 0 sec hubhash_size: 18 Mar 30 15:28:46:974 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:28:46:975 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:28:46:975 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:28:46:975 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:28:46:975 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:28:46:976 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:28:46:976 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:28:46:977 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:28:46:977 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:28:46:977 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:28:46:977 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:28:46:978 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:28:46:978 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:28:46:978 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:28:46:979 [140603195123456] 5 hub: TSESS-A-3-14 [SecondaryHA] local 100 '_close' Mar 30 15:28:46:979 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:46:979 [140603195123456] 1 hub: TSESS-A-3-14 [SecondaryHA] sent 100 '_close' (296 bytes, 0 ms) Mar 30 15:28:46:979 [140603195123456] 3 hub: TSESS-A-3-14 [SecondaryHA] local session closed, suspending Mar 30 15:28:46:979 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:28:46:980 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:28:46:980 [140605283890944] 1 hub: PERF: hubi_check_security: 0 sec hubhash_size: 18 Mar 30 15:28:46:989 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:28:47:012 [140605547460672] 4 hub: medium timeout Mar 30 15:28:47:012 [140605547460672] 3 hub: next_check [1585596696] now [1585596527] Mar 30 15:28:47:012 [140605547460672] 2 hub: verify_login from ctrl 127.0.0.1/53882 Mar 30 15:28:47:012 [140605547460672] 1 hub: verify - [target] prid=controller cmd=probe_list ip=10.x.x.x perm=1 Mar 30 15:28:47:012 [140605547460672] 1 hub: verify - [source] id=ppm ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:47:147 [140603203516160] 5 hub: TSESS-A-6-14 [Secondary] suspended Mar 30 15:28:47:163 [140603715176192] 5 hub: TSESS-A-5-59 [TunnelB] suspended Mar 30 15:28:47:179 [140603195123456] 5 hub: TSESS-A-3-14 [SecondaryHA] suspended Mar 30 15:28:47:467 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:47:520 [140603161552640] 5 hub: TSESS-P-4-14 [SecondaryHA] suspended Mar 30 15:28:47:606 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:28:47:677 [140603153159936] 5 hub: TSESS-P-8-11 [Secondary] suspended Mar 30 15:28:47:937 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:28:47:961 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:28:48:651 [140603169945344] 5 hub: TSESS-P-7-14 [TunnelB] suspended Mar 30 15:28:48:701 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:28:49:270 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:51:072 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:51:414 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:28:51:414 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:28:51:414 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33178 Mar 30 15:28:51:414 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33178' delegated to TSESS-A-5 [140603715176192] Mar 30 15:28:51:414 [140603715176192] 3 hub: TSESS-A-5-60 [TunnelB] restarted (for local) Mar 30 15:28:51:419 [140603715176192] 5 hub: TSESS-A-5-60 [TunnelB] local 100 'get_info' Mar 30 15:28:51:419 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:51:419 [140603715176192] 1 hub: TSESS-A-5-60 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:28:51:430 [140603715176192] 1 hub: TSESS-A-5-60 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:28:51:430 [140603715176192] 3 hub: TSESS-A-5-60 [TunnelB] is paired with [P-298-60] Mar 30 15:28:51:430 [140603715176192] 1 hub: TSESS-A-5-60 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:28:51:430 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33180 Mar 30 15:28:51:430 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33180' delegated to TSESS-A-10 [140603706783488] Mar 30 15:28:51:430 [140603706783488] 3 hub: TSESS-A-10-44 [TunnelB] restarted (for local) Mar 30 15:28:51:435 [140603715176192] 5 hub: TSESS-A-5-60 [TunnelB] local 100 '_close' Mar 30 15:28:51:435 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:51:435 [140603715176192] 1 hub: TSESS-A-5-60 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:28:51:435 [140603715176192] 3 hub: TSESS-A-5-60 [TunnelB] local session closed, suspending Mar 30 15:28:51:435 [140603706783488] 5 hub: TSESS-A-10-44 [TunnelB] local 100 '_status' Mar 30 15:28:51:435 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:51:435 [140603706783488] 1 hub: TSESS-A-10-44 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:28:51:445 [140603706783488] 1 hub: TSESS-A-10-44 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:28:51:445 [140603706783488] 3 hub: TSESS-A-10-44 [TunnelB] is paired with [P-300-44] Mar 30 15:28:51:445 [140603706783488] 1 hub: TSESS-A-10-44 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:28:51:451 [140603706783488] 5 hub: TSESS-A-10-44 [TunnelB] local 100 '_close' Mar 30 15:28:51:451 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:28:51:451 [140603706783488] 1 hub: TSESS-A-10-44 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:28:51:451 [140603706783488] 3 hub: TSESS-A-10-44 [TunnelB] local session closed, suspending Mar 30 15:28:51:635 [140603715176192] 5 hub: TSESS-A-5-60 [TunnelB] suspended Mar 30 15:28:51:651 [140603706783488] 5 hub: TSESS-A-10-44 [TunnelB] suspended Mar 30 15:28:53:775 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:55:578 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:56:961 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:28:56:989 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:28:57:147 [140603203516160] 5 hub: TSESS-A-6-14 [Secondary] suspended Mar 30 15:28:57:179 [140603195123456] 5 hub: TSESS-A-3-14 [SecondaryHA] suspended Mar 30 15:28:57:380 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:28:57:420 [140605547460672] 4 hub: medium timeout Mar 30 15:28:57:420 [140605547460672] 3 hub: next_check [1585596696] now [1585596537] Mar 30 15:28:57:520 [140603161552640] 5 hub: TSESS-P-4-14 [SecondaryHA] suspended Mar 30 15:28:57:606 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:28:57:677 [140603153159936] 5 hub: TSESS-P-8-11 [Secondary] suspended Mar 30 15:28:57:937 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:28:57:961 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:28:58:651 [140603169945344] 5 hub: TSESS-P-7-14 [TunnelB] suspended Mar 30 15:28:58:701 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:28:59:182 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:01:457 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:29:01:457 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:29:01:457 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33186 Mar 30 15:29:01:457 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33186' delegated to TSESS-A-5 [140603715176192] Mar 30 15:29:01:457 [140603715176192] 3 hub: TSESS-A-5-61 [TunnelB] restarted (for local) Mar 30 15:29:01:462 [140603715176192] 5 hub: TSESS-A-5-61 [TunnelB] local 100 'get_info' Mar 30 15:29:01:462 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:01:462 [140603715176192] 1 hub: TSESS-A-5-61 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:29:01:473 [140603715176192] 1 hub: TSESS-A-5-61 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:29:01:473 [140603715176192] 3 hub: TSESS-A-5-61 [TunnelB] is paired with [P-298-61] Mar 30 15:29:01:473 [140603715176192] 1 hub: TSESS-A-5-61 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:29:01:473 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33188 Mar 30 15:29:01:473 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33188' delegated to TSESS-A-10 [140603706783488] Mar 30 15:29:01:473 [140603706783488] 3 hub: TSESS-A-10-45 [TunnelB] restarted (for local) Mar 30 15:29:01:478 [140603715176192] 5 hub: TSESS-A-5-61 [TunnelB] local 100 '_close' Mar 30 15:29:01:478 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:01:478 [140603715176192] 1 hub: TSESS-A-5-61 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:29:01:478 [140603715176192] 3 hub: TSESS-A-5-61 [TunnelB] local session closed, suspending Mar 30 15:29:01:478 [140603706783488] 5 hub: TSESS-A-10-45 [TunnelB] local 100 '_status' Mar 30 15:29:01:478 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:01:478 [140603706783488] 1 hub: TSESS-A-10-45 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:29:01:489 [140603706783488] 1 hub: TSESS-A-10-45 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:29:01:489 [140603706783488] 3 hub: TSESS-A-10-45 [TunnelB] is paired with [P-300-45] Mar 30 15:29:01:489 [140603706783488] 1 hub: TSESS-A-10-45 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:29:01:494 [140603706783488] 5 hub: TSESS-A-10-45 [TunnelB] local 100 '_close' Mar 30 15:29:01:494 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:01:494 [140603706783488] 1 hub: TSESS-A-10-45 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:29:01:494 [140603706783488] 3 hub: TSESS-A-10-45 [TunnelB] local session closed, suspending Mar 30 15:29:01:678 [140603715176192] 5 hub: TSESS-A-5-61 [TunnelB] suspended Mar 30 15:29:01:694 [140603706783488] 5 hub: TSESS-A-10-45 [TunnelB] suspended Mar 30 15:29:01:886 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:03:688 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:05:461 [140605547460672] 3 hub: check_session - session 0x20f9050 (127.0.0.1/48002) is a server Mar 30 15:29:05:461 [140605547460672] 2 hub: Profile - clayk 38 bytes (OK) Mar 30 15:29:05:490 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:06:961 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:29:06:989 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:29:07:147 [140603203516160] 5 hub: TSESS-A-6-14 [Secondary] suspended Mar 30 15:29:07:179 [140603195123456] 5 hub: TSESS-A-3-14 [SecondaryHA] suspended Mar 30 15:29:07:292 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:07:465 [140605547460672] 4 hub: medium timeout Mar 30 15:29:07:465 [140605547460672] 3 hub: next_check [1585596696] now [1585596547] Mar 30 15:29:07:520 [140603161552640] 5 hub: TSESS-P-4-14 [SecondaryHA] suspended Mar 30 15:29:07:606 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:29:07:619 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:29:07:619 [140603731961600] 2 hub: CTRL SecondaryHA request 'hubup' passed to TSESS-P-4 [140603161552640] Mar 30 15:29:07:619 [140603161552640] 3 hub: TSESS-P-4-15 [SecondaryHA] restarted (for remote) Mar 30 15:29:07:619 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:39:07 Mar 30 15:29:07:619 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:29:07:620 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:29:07:620 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:29:07:620 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:29:07:620 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:29:07:620 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:29:07:620 [140603161552640] 4 hub: CONNECT: 0x7fe09c000d70(39) 127.0.0.1/53900->127.0.0.1/48002 Mar 30 15:29:07:620 [140603161552640] 3 hub: TSESS-P-4-15 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:29:07:620 [140603161552640] 1 hub: TSESS-P-4-15 [SecondaryHA] received 100 'hubup' (689 bytes) Mar 30 15:29:07:620 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53900 (s:392) Mar 30 15:29:07:620 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/53900] Mar 30 15:29:07:620 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:29:07:620 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:29:07:620 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/53900 Mar 30 15:29:07:620 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/53900 same proximity=0 status=0 secver=217 Mar 30 15:29:07:625 [140603161552640] 5 hub: TSESS-P-4-15 [SecondaryHA] local 200 'OK' Mar 30 15:29:07:625 [140603161552640] 1 hub: TSESS-P-4-15 [SecondaryHA] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:29:07:635 [140603161552640] 1 hub: TSESS-P-4-15 [SecondaryHA] ssl 100 '_close' (302 bytes) Mar 30 15:29:07:635 [140603161552640] 3 hub: TSESS-P-4-15 [SecondaryHA] is paired with [A-1948-15] Mar 30 15:29:07:635 [140603161552640] 1 hub: TSESS-P-4-15 [SecondaryHA] received 100 '_close' (298 bytes) Mar 30 15:29:07:640 [140603161552640] 1 hub: TSESS-P-4-15 [SecondaryHA] remote session closed Mar 30 15:29:07:640 [140603161552640] 3 hub: TSESS-P-4-15 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:29:07:640 [140603161552640] 5 hub: TSESS-P-4-15 [SecondaryHA] suspended Mar 30 15:29:07:677 [140603153159936] 5 hub: TSESS-P-8-11 [Secondary] suspended Mar 30 15:29:07:794 [140603740354304] 5 hub: TSESS-P-8 not internal command - thread will handle. mtype=100; sfd:23 Mar 30 15:29:07:794 [140603740354304] 2 hub: CTRL Secondary request 'hubup' passed to TSESS-P-8 [140603153159936] Mar 30 15:29:07:794 [140603153159936] 3 hub: TSESS-P-8-12 [Secondary] restarted (for remote) Mar 30 15:29:07:794 [140603153159936] 2 hub: TSESS-P-8 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:39:07 Mar 30 15:29:07:794 [140603153159936] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:29:07:794 [140603153159936] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:29:07:794 [140603153159936] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:29:07:794 [140603153159936] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:29:07:794 [140603153159936] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:29:07:794 [140603153159936] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:29:07:794 [140603153159936] 4 hub: CONNECT: 0x7fe0a0000d90(39) 127.0.0.1/53902->127.0.0.1/48002 Mar 30 15:29:07:794 [140603153159936] 3 hub: TSESS-P-8-12 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:29:07:794 [140603153159936] 1 hub: TSESS-P-8-12 [Secondary] received 100 'hubup' (685 bytes) Mar 30 15:29:07:794 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53902 (s:388) Mar 30 15:29:07:794 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/53902] Mar 30 15:29:07:794 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:29:07:794 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:29:07:794 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/53902 Mar 30 15:29:07:795 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/53902 same proximity=0 status=0 secver=217 Mar 30 15:29:07:800 [140603153159936] 5 hub: TSESS-P-8-12 [Secondary] local 200 'OK' Mar 30 15:29:07:800 [140603153159936] 1 hub: TSESS-P-8-12 [Secondary] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:29:07:810 [140603153159936] 1 hub: TSESS-P-8-12 [Secondary] ssl 100 '_close' (298 bytes) Mar 30 15:29:07:810 [140603153159936] 3 hub: TSESS-P-8-12 [Secondary] is paired with [A-2121-12] Mar 30 15:29:07:810 [140603153159936] 1 hub: TSESS-P-8-12 [Secondary] received 100 '_close' (298 bytes) Mar 30 15:29:07:815 [140603153159936] 1 hub: TSESS-P-8-12 [Secondary] remote session closed Mar 30 15:29:07:815 [140603153159936] 3 hub: TSESS-P-8-12 [Secondary] exiting msg loop (remote close) Mar 30 15:29:07:815 [140603153159936] 5 hub: TSESS-P-8-12 [Secondary] suspended Mar 30 15:29:07:937 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:29:07:961 [140602649859840] 5 hub: TSESS-P-20-0 [Secondary] suspended Mar 30 15:29:08:651 [140603169945344] 5 hub: TSESS-P-7-14 [TunnelB] suspended Mar 30 15:29:08:701 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:29:08:716 [140603723568896] 5 hub: TSESS-P-7 not internal command - thread will handle. mtype=100; sfd:22 Mar 30 15:29:08:716 [140603723568896] 2 hub: CTRL TunnelB request 'hubup' passed to TSESS-P-7 [140603169945344] Mar 30 15:29:08:716 [140603169945344] 3 hub: TSESS-P-7-15 [TunnelB] restarted (for remote) Mar 30 15:29:08:716 [140603169945344] 2 hub: TSESS-P-7 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:39:08 Mar 30 15:29:08:716 [140603169945344] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:29:08:716 [140603169945344] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:29:08:716 [140603169945344] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:29:08:716 [140603169945344] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:29:08:716 [140603169945344] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:29:08:716 [140603169945344] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:29:08:716 [140603169945344] 4 hub: CONNECT: 0x7fe0a8000d20(39) 127.0.0.1/53904->127.0.0.1/48002 Mar 30 15:29:08:716 [140603169945344] 3 hub: TSESS-P-7-15 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:29:08:716 [140603169945344] 1 hub: TSESS-P-7-15 [TunnelB] received 100 'hubup' (673 bytes) Mar 30 15:29:08:716 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53904 (s:378) Mar 30 15:29:08:716 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/53904] Mar 30 15:29:08:716 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:29:08:716 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:29:08:716 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/53904 Mar 30 15:29:08:716 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/53904 same proximity=0 status=0 secver=217 Mar 30 15:29:08:721 [140603169945344] 5 hub: TSESS-P-7-15 [TunnelB] local 200 'OK' Mar 30 15:29:08:721 [140603169945344] 1 hub: TSESS-P-7-15 [TunnelB] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:29:08:731 [140603169945344] 1 hub: TSESS-P-7-15 [TunnelB] ssl 100 '_close' (284 bytes) Mar 30 15:29:08:731 [140603169945344] 3 hub: TSESS-P-7-15 [TunnelB] is paired with [A-299-15] Mar 30 15:29:08:732 [140603169945344] 1 hub: TSESS-P-7-15 [TunnelB] received 100 '_close' (296 bytes) Mar 30 15:29:08:737 [140603169945344] 1 hub: TSESS-P-7-15 [TunnelB] remote session closed Mar 30 15:29:08:737 [140603169945344] 3 hub: TSESS-P-7-15 [TunnelB] exiting msg loop (remote close) Mar 30 15:29:08:737 [140603169945344] 5 hub: TSESS-P-7-15 [TunnelB] suspended Mar 30 15:29:09:095 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:10:099 [140603740354304] 1 hub: TSESS-P-20 marked for removal (reason: Redundant) Mar 30 15:29:10:099 [140602649859840] 3 hub: TSESS-P-20-0 [Secondary] ended Mar 30 15:29:11:500 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:29:11:500 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:29:11:500 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33200 Mar 30 15:29:11:500 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33200' delegated to TSESS-A-5 [140603715176192] Mar 30 15:29:11:500 [140603715176192] 3 hub: TSESS-A-5-62 [TunnelB] restarted (for local) Mar 30 15:29:11:505 [140603715176192] 5 hub: TSESS-A-5-62 [TunnelB] local 100 'get_info' Mar 30 15:29:11:505 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:11:506 [140603715176192] 1 hub: TSESS-A-5-62 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:29:11:516 [140603715176192] 1 hub: TSESS-A-5-62 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:29:11:516 [140603715176192] 3 hub: TSESS-A-5-62 [TunnelB] is paired with [P-298-62] Mar 30 15:29:11:516 [140603715176192] 1 hub: TSESS-A-5-62 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:29:11:516 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33202 Mar 30 15:29:11:516 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33202' delegated to TSESS-A-10 [140603706783488] Mar 30 15:29:11:516 [140603706783488] 3 hub: TSESS-A-10-46 [TunnelB] restarted (for local) Mar 30 15:29:11:521 [140603715176192] 5 hub: TSESS-A-5-62 [TunnelB] local 100 '_close' Mar 30 15:29:11:521 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:11:521 [140603715176192] 1 hub: TSESS-A-5-62 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:29:11:521 [140603715176192] 3 hub: TSESS-A-5-62 [TunnelB] local session closed, suspending Mar 30 15:29:11:521 [140603706783488] 5 hub: TSESS-A-10-46 [TunnelB] local 100 '_status' Mar 30 15:29:11:521 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:11:521 [140603706783488] 1 hub: TSESS-A-10-46 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:29:11:532 [140603706783488] 1 hub: TSESS-A-10-46 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:29:11:532 [140603706783488] 3 hub: TSESS-A-10-46 [TunnelB] is paired with [P-300-46] Mar 30 15:29:11:532 [140603706783488] 1 hub: TSESS-A-10-46 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:29:11:537 [140603706783488] 5 hub: TSESS-A-10-46 [TunnelB] local 100 '_close' Mar 30 15:29:11:537 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:11:537 [140603706783488] 1 hub: TSESS-A-10-46 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:29:11:537 [140603706783488] 3 hub: TSESS-A-10-46 [TunnelB] local session closed, suspending Mar 30 15:29:11:721 [140603715176192] 5 hub: TSESS-A-5-62 [TunnelB] suspended Mar 30 15:29:11:737 [140603706783488] 5 hub: TSESS-A-10-46 [TunnelB] suspended Mar 30 15:29:11:798 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:13:056 [140603740354304] 2 hub: CTRL Secondary exit code for thread 140602649859840 was 0 Mar 30 15:29:13:056 [140603740354304] 2 hub: CTRL Secondary removing TSESS-P-20 [140602649859840] Mar 30 15:29:13:056 [140603740354304] 2 hub: TSESS-P-20 stats: bytes out=595,in=1071 Mar 30 15:29:13:056 [140603740354304] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:29:13:056 [140603740354304] 4 hub: TSESS-P-20 releasing resources Mar 30 15:29:13:056 [140603740354304] 4 hub: SSL_shutdown on TSESS-P-20: CLOSING SSL connection Mar 30 15:29:13:056 [140603740354304] 3 hub: SSL alert (write): warning: close notify Mar 30 15:29:13:056 [140603740354304] 3 hub: TSESS-P-20 [Secondary] SSL_shutdown succeeded (not acknowledged) Mar 30 15:29:13:056 [140603740354304] 4 hub: TSESS-P-20 [Secondary] resources released Mar 30 15:29:13:601 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:15:403 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:16:010 [140605283890944] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:29:16:011 [140605283890944] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:29:16:011 [140605283890944] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:29:16:011 [140605283890944] 1 hub: PERF: hubup_check_interval: 0 sec hubhash_size: 18 Mar 30 15:29:16:011 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:011 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:011 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:011 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:011 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:011 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:011 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:29:16:011 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:011 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:011 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:011 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:011 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:011 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:011 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:011 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:011 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:29:16:011 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:29:16:011 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:29:16:011 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(37) 127.0.0.1/47574->127.0.0.1/48007 Mar 30 15:29:16:011 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/47574 Mar 30 15:29:16:011 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/47574' delegated to TSESS-A-6 [140603203516160] Mar 30 15:29:16:011 [140603203516160] 3 hub: TSESS-A-6-15 [Secondary] restarted (for local) Mar 30 15:29:16:016 [140603203516160] 5 hub: TSESS-A-6-15 [Secondary] local 100 'hubup' Mar 30 15:29:16:016 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:16:016 [140603203516160] 1 hub: TSESS-A-6-15 [Secondary] sent 100 'hubup' (687 bytes, 0 ms) Mar 30 15:29:16:027 [140603203516160] 1 hub: TSESS-A-6-15 [Secondary] ssl 200 '-' (455 bytes) Mar 30 15:29:16:027 [140603203516160] 3 hub: TSESS-A-6-15 [Secondary] is paired with [P-2120-15] Mar 30 15:29:16:027 [140603203516160] 1 hub: TSESS-A-6-15 [Secondary] received 200 '-' (463 bytes) Mar 30 15:29:16:027 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:29:16:027 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:29:16:027 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:29:16:027 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:29:16:027 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary same proximity=0 status=0 secver=217 Mar 30 15:29:16:027 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:027 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:027 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:027 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:027 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:027 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:027 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:29:16:027 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:027 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:027 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:027 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:027 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:027 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:027 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:027 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:027 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:29:16:027 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:29:16:027 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:29:16:027 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(37) 127.0.0.1/33206->127.0.0.1/48009 Mar 30 15:29:16:027 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33206 Mar 30 15:29:16:027 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33206' delegated to TSESS-A-5 [140603715176192] Mar 30 15:29:16:027 [140603715176192] 3 hub: TSESS-A-5-63 [TunnelB] restarted (for local) Mar 30 15:29:16:032 [140603203516160] 5 hub: TSESS-A-6-15 [Secondary] local 100 '_close' Mar 30 15:29:16:032 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:16:032 [140603203516160] 1 hub: TSESS-A-6-15 [Secondary] sent 100 '_close' (294 bytes, 0 ms) Mar 30 15:29:16:032 [140603203516160] 3 hub: TSESS-A-6-15 [Secondary] local session closed, suspending Mar 30 15:29:16:032 [140603715176192] 5 hub: TSESS-A-5-63 [TunnelB] local 100 'hubup' Mar 30 15:29:16:032 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:16:032 [140603715176192] 1 hub: TSESS-A-5-63 [TunnelB] sent 100 'hubup' (685 bytes, 0 ms) Mar 30 15:29:16:043 [140603715176192] 1 hub: TSESS-A-5-63 [TunnelB] ssl 200 '-' (443 bytes) Mar 30 15:29:16:043 [140603715176192] 3 hub: TSESS-A-5-63 [TunnelB] is paired with [P-298-63] Mar 30 15:29:16:043 [140603715176192] 1 hub: TSESS-A-5-63 [TunnelB] received 200 '-' (451 bytes) Mar 30 15:29:16:043 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:29:16:043 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:29:16:043 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:29:16:043 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:29:16:043 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB same proximity=0 status=0 secver=217 Mar 30 15:29:16:043 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:043 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:043 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:043 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:043 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:043 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:043 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:29:16:043 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:043 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:043 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:043 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:043 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:043 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:043 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:043 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:16:043 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:29:16:043 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:29:16:043 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:29:16:043 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(37) 127.0.0.1/40626->127.0.0.1/48008 Mar 30 15:29:16:043 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/40626 Mar 30 15:29:16:043 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/40626' delegated to TSESS-A-3 [140603195123456] Mar 30 15:29:16:043 [140603195123456] 3 hub: TSESS-A-3-15 [SecondaryHA] restarted (for local) Mar 30 15:29:16:048 [140603715176192] 5 hub: TSESS-A-5-63 [TunnelB] local 100 '_close' Mar 30 15:29:16:048 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:16:048 [140603715176192] 1 hub: TSESS-A-5-63 [TunnelB] sent 100 '_close' (292 bytes, 0 ms) Mar 30 15:29:16:048 [140603715176192] 3 hub: TSESS-A-5-63 [TunnelB] local session closed, suspending Mar 30 15:29:16:048 [140603195123456] 5 hub: TSESS-A-3-15 [SecondaryHA] local 100 'hubup' Mar 30 15:29:16:048 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:16:048 [140603195123456] 1 hub: TSESS-A-3-15 [SecondaryHA] sent 100 'hubup' (689 bytes, 0 ms) Mar 30 15:29:16:059 [140603195123456] 1 hub: TSESS-A-3-15 [SecondaryHA] ssl 200 '-' (459 bytes) Mar 30 15:29:16:059 [140603195123456] 3 hub: TSESS-A-3-15 [SecondaryHA] is paired with [P-1947-15] Mar 30 15:29:16:059 [140603195123456] 1 hub: TSESS-A-3-15 [SecondaryHA] received 200 '-' (467 bytes) Mar 30 15:29:16:059 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:29:16:059 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:29:16:059 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:29:16:059 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:29:16:059 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA same proximity=0 status=0 secver=217 Mar 30 15:29:16:059 [140605283890944] 5 hub: mark_hubup_list_updated Mar 30 15:29:16:059 [140605283890944] 5 hub: hublist changed: /DOMAIN/Primary 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:16:059 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:16:059 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:16:059 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:16:059 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:16:059 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:16:059 [140605283890944] 5 hub: hublist changed: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:29:16:059 [140605283890944] 5 hub: hublist changed: /DOMAIN/PrimaryHA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:16:059 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:16:059 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:16:059 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:16:059 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:16:059 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:16:059 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:16:059 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:16:059 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:29:16:059 [140605283890944] 5 hub: hublist changed: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:29:16:059 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelBHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:29:16:059 [140605283890944] 5 hub: mark_hubup_list_updated end Mar 30 15:29:16:059 [140605283890944] 1 hub: PERF: hubup_check_for_updates: 0 sec hubhash_size: 18 Mar 30 15:29:16:059 [140605283890944] 1 hub: PERF: hubi_run_command: 0 sec hubhash_size: 18 Mar 30 15:29:16:059 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:29:16:059 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:29:16:060 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:29:16:060 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:29:16:060 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:29:16:061 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:29:16:061 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:29:16:062 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:29:16:062 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:29:16:063 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:29:16:063 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:29:16:064 [140603195123456] 5 hub: TSESS-A-3-15 [SecondaryHA] local 100 '_close' Mar 30 15:29:16:064 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:16:064 [140603195123456] 1 hub: TSESS-A-3-15 [SecondaryHA] sent 100 '_close' (296 bytes, 0 ms) Mar 30 15:29:16:064 [140603195123456] 3 hub: TSESS-A-3-15 [SecondaryHA] local session closed, suspending Mar 30 15:29:16:064 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:29:16:064 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:29:16:064 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:29:16:065 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:29:16:066 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:29:16:066 [140605283890944] 1 hub: PERF: hubi_check_security: 0 sec hubhash_size: 18 Mar 30 15:29:16:232 [140603203516160] 5 hub: TSESS-A-6-15 [Secondary] suspended Mar 30 15:29:16:248 [140603715176192] 5 hub: TSESS-A-5-63 [TunnelB] suspended Mar 30 15:29:16:264 [140603195123456] 5 hub: TSESS-A-3-15 [SecondaryHA] suspended Mar 30 15:29:16:961 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:29:16:989 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:29:17:206 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:17:506 [140605547460672] 4 hub: medium timeout Mar 30 15:29:17:506 [140605547460672] 3 hub: next_check [1585596696] now [1585596557] Mar 30 15:29:17:606 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:29:17:640 [140603161552640] 5 hub: TSESS-P-4-15 [SecondaryHA] suspended Mar 30 15:29:17:815 [140603153159936] 5 hub: TSESS-P-8-12 [Secondary] suspended Mar 30 15:29:17:938 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:29:18:701 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:29:18:737 [140603169945344] 5 hub: TSESS-P-7-15 [TunnelB] suspended Mar 30 15:29:19:008 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:21:543 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:29:21:543 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:29:21:543 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33216 Mar 30 15:29:21:543 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33216' delegated to TSESS-A-5 [140603715176192] Mar 30 15:29:21:543 [140603715176192] 3 hub: TSESS-A-5-64 [TunnelB] restarted (for local) Mar 30 15:29:21:549 [140603715176192] 5 hub: TSESS-A-5-64 [TunnelB] local 100 'get_info' Mar 30 15:29:21:549 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:21:549 [140603715176192] 1 hub: TSESS-A-5-64 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:29:21:559 [140603715176192] 1 hub: TSESS-A-5-64 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:29:21:559 [140603715176192] 3 hub: TSESS-A-5-64 [TunnelB] is paired with [P-298-64] Mar 30 15:29:21:559 [140603715176192] 1 hub: TSESS-A-5-64 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:29:21:559 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33218 Mar 30 15:29:21:559 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33218' delegated to TSESS-A-10 [140603706783488] Mar 30 15:29:21:559 [140603706783488] 3 hub: TSESS-A-10-47 [TunnelB] restarted (for local) Mar 30 15:29:21:564 [140603715176192] 5 hub: TSESS-A-5-64 [TunnelB] local 100 '_close' Mar 30 15:29:21:564 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:21:564 [140603715176192] 1 hub: TSESS-A-5-64 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:29:21:564 [140603715176192] 3 hub: TSESS-A-5-64 [TunnelB] local session closed, suspending Mar 30 15:29:21:564 [140603706783488] 5 hub: TSESS-A-10-47 [TunnelB] local 100 '_status' Mar 30 15:29:21:564 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:21:564 [140603706783488] 1 hub: TSESS-A-10-47 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:29:21:575 [140603706783488] 1 hub: TSESS-A-10-47 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:29:21:575 [140603706783488] 3 hub: TSESS-A-10-47 [TunnelB] is paired with [P-300-47] Mar 30 15:29:21:575 [140603706783488] 1 hub: TSESS-A-10-47 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:29:21:580 [140603706783488] 5 hub: TSESS-A-10-47 [TunnelB] local 100 '_close' Mar 30 15:29:21:580 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:21:580 [140603706783488] 1 hub: TSESS-A-10-47 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:29:21:580 [140603706783488] 3 hub: TSESS-A-10-47 [TunnelB] local session closed, suspending Mar 30 15:29:21:711 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:21:764 [140603715176192] 5 hub: TSESS-A-5-64 [TunnelB] suspended Mar 30 15:29:21:780 [140603706783488] 5 hub: TSESS-A-10-47 [TunnelB] suspended Mar 30 15:29:23:514 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:25:316 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:26:232 [140603203516160] 5 hub: TSESS-A-6-15 [Secondary] suspended Mar 30 15:29:26:264 [140603195123456] 5 hub: TSESS-A-3-15 [SecondaryHA] suspended Mar 30 15:29:26:961 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:29:26:989 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:29:27:119 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:27:550 [140605547460672] 4 hub: medium timeout Mar 30 15:29:27:550 [140605547460672] 3 hub: next_check [1585596696] now [1585596567] Mar 30 15:29:27:606 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:29:27:641 [140603161552640] 5 hub: TSESS-P-4-15 [SecondaryHA] suspended Mar 30 15:29:27:815 [140603153159936] 5 hub: TSESS-P-8-12 [Secondary] suspended Mar 30 15:29:27:938 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:29:28:701 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:29:28:737 [140603169945344] 5 hub: TSESS-P-7-15 [TunnelB] suspended Mar 30 15:29:29:822 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:31:586 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:29:31:586 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:29:31:586 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33224 Mar 30 15:29:31:586 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33224' delegated to TSESS-A-5 [140603715176192] Mar 30 15:29:31:586 [140603715176192] 3 hub: TSESS-A-5-65 [TunnelB] restarted (for local) Mar 30 15:29:31:591 [140603715176192] 5 hub: TSESS-A-5-65 [TunnelB] local 100 'get_info' Mar 30 15:29:31:591 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:31:592 [140603715176192] 1 hub: TSESS-A-5-65 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:29:31:602 [140603715176192] 1 hub: TSESS-A-5-65 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:29:31:602 [140603715176192] 3 hub: TSESS-A-5-65 [TunnelB] is paired with [P-298-65] Mar 30 15:29:31:602 [140603715176192] 1 hub: TSESS-A-5-65 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:29:31:602 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33226 Mar 30 15:29:31:602 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33226' delegated to TSESS-A-10 [140603706783488] Mar 30 15:29:31:602 [140603706783488] 3 hub: TSESS-A-10-48 [TunnelB] restarted (for local) Mar 30 15:29:31:607 [140603715176192] 5 hub: TSESS-A-5-65 [TunnelB] local 100 '_close' Mar 30 15:29:31:607 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:31:607 [140603715176192] 1 hub: TSESS-A-5-65 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:29:31:607 [140603715176192] 3 hub: TSESS-A-5-65 [TunnelB] local session closed, suspending Mar 30 15:29:31:607 [140603706783488] 5 hub: TSESS-A-10-48 [TunnelB] local 100 '_status' Mar 30 15:29:31:607 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:31:607 [140603706783488] 1 hub: TSESS-A-10-48 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:29:31:618 [140603706783488] 1 hub: TSESS-A-10-48 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:29:31:618 [140603706783488] 3 hub: TSESS-A-10-48 [TunnelB] is paired with [P-300-48] Mar 30 15:29:31:618 [140603706783488] 1 hub: TSESS-A-10-48 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:29:31:623 [140603706783488] 5 hub: TSESS-A-10-48 [TunnelB] local 100 '_close' Mar 30 15:29:31:623 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:31:623 [140603706783488] 1 hub: TSESS-A-10-48 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:29:31:623 [140603706783488] 3 hub: TSESS-A-10-48 [TunnelB] local session closed, suspending Mar 30 15:29:31:625 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:31:807 [140603715176192] 5 hub: TSESS-A-5-65 [TunnelB] suspended Mar 30 15:29:31:823 [140603706783488] 5 hub: TSESS-A-10-48 [TunnelB] suspended Mar 30 15:29:33:427 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:34:127 [140604042360576] 5 hub: There are 0 objects waiting on 8 threads Mar 30 15:29:35:230 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:35:592 [140605547460672] 3 hub: check_session - session 0x20f9050 (127.0.0.1/48002) is a server Mar 30 15:29:35:592 [140605547460672] 2 hub: Profile - clayk 38 bytes (OK) Mar 30 15:29:36:233 [140603203516160] 5 hub: TSESS-A-6-15 [Secondary] suspended Mar 30 15:29:36:264 [140603195123456] 5 hub: TSESS-A-3-15 [SecondaryHA] suspended Mar 30 15:29:36:961 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:29:36:989 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:29:37:032 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:37:594 [140605547460672] 4 hub: medium timeout Mar 30 15:29:37:594 [140605547460672] 3 hub: next_check [1585596696] now [1585596577] Mar 30 15:29:37:606 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:29:37:641 [140603161552640] 5 hub: TSESS-P-4-15 [SecondaryHA] suspended Mar 30 15:29:37:744 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:29:37:744 [140603731961600] 2 hub: CTRL SecondaryHA request 'hubup' passed to TSESS-P-4 [140603161552640] Mar 30 15:29:37:744 [140603161552640] 3 hub: TSESS-P-4-16 [SecondaryHA] restarted (for remote) Mar 30 15:29:37:744 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:39:37 Mar 30 15:29:37:744 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:29:37:744 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:29:37:744 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:29:37:744 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:29:37:744 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:29:37:744 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:29:37:745 [140603161552640] 4 hub: CONNECT: 0x7fe09c000c70(37) 127.0.0.1/53938->127.0.0.1/48002 Mar 30 15:29:37:745 [140603161552640] 3 hub: TSESS-P-4-16 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:29:37:745 [140603161552640] 1 hub: TSESS-P-4-16 [SecondaryHA] received 100 'hubup' (1215 bytes) Mar 30 15:29:37:745 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53938 (s:918) Mar 30 15:29:37:745 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/53938] Mar 30 15:29:37:745 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:29:37:745 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:29:37:745 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/53938 Mar 30 15:29:37:745 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/53938 same proximity=0 status=0 secver=217 Mar 30 15:29:37:745 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-12] domain [DOMAIN] addr [(null)] ip [100.64.1.22] port [48002] robotname [a0f8us052nim752] version [9.20 [Build 9.20.1048, Aug 8 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-12] source [/DOMAIN/SecondaryHA] from [127.0.0.1/53938] Mar 30 15:29:37:745 [140605547460672] 4 hub: hubup source: /DOMAIN/SecondaryHA Mar 30 15:29:37:745 [140605547460672] 4 hub: hubinfo create: /DOMAIN/A0F8-USDA-FMMI-12 100.64.1.22/48002 0x2145bd0 Mar 30 15:29:37:745 [140605547460672] 1 hub: hubinfo_update - new hubinfo for /DOMAIN/A0F8-USDA-FMMI-12 p=4 127.0.0.1/53938 Mar 30 15:29:37:745 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/A0F8-USDA-FMMI-12/a0f8us052nim752/hub, from 127.0.0.1/53938 Mar 30 15:29:37:745 [140605547460672] 2 hub: hubinfo_update - DOMAIN/A0F8-USDA-FMMI-12 from=127.0.0.1/53938 CHANGED proximity=4 status=0 secver=1 Mar 30 15:29:37:745 [140605547460672] 5 hub: hubinfo_update source - lookup /DOMAIN/SecondaryHA ok Mar 30 15:29:37:745 [140605547460672] 2 hub: ch_hublist /DOMAIN/A0F8-USDA-FMMI-12/a0f8us052nim752/hub Mar 30 15:29:37:745 [140605547460672] 5 hub: do_put_intern_post - domain [DOMAIN] robot [ROBOT_NAME] origin [TunnelBHA] source [10.x.x.x] Mar 30 15:29:37:750 [140603161552640] 5 hub: TSESS-P-4-16 [SecondaryHA] local 200 'OK' Mar 30 15:29:37:750 [140603161552640] 1 hub: TSESS-P-4-16 [SecondaryHA] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:29:37:760 [140603161552640] 1 hub: TSESS-P-4-16 [SecondaryHA] ssl 100 '_close' (302 bytes) Mar 30 15:29:37:760 [140603161552640] 3 hub: TSESS-P-4-16 [SecondaryHA] is paired with [A-1948-16] Mar 30 15:29:37:760 [140603161552640] 1 hub: TSESS-P-4-16 [SecondaryHA] received 100 '_close' (298 bytes) Mar 30 15:29:37:765 [140603161552640] 1 hub: TSESS-P-4-16 [SecondaryHA] remote session closed Mar 30 15:29:37:765 [140603161552640] 3 hub: TSESS-P-4-16 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:29:37:765 [140603161552640] 5 hub: TSESS-P-4-16 [SecondaryHA] suspended Mar 30 15:29:37:815 [140603153159936] 5 hub: TSESS-P-8-12 [Secondary] suspended Mar 30 15:29:37:938 [140603740354304] 5 hub: TSESS-P-8 not internal command - thread will handle. mtype=100; sfd:23 Mar 30 15:29:37:938 [140603740354304] 2 hub: CTRL Secondary request 'hubup' passed to TSESS-P-8 [140603153159936] Mar 30 15:29:37:938 [140603153159936] 3 hub: TSESS-P-8-13 [Secondary] restarted (for remote) Mar 30 15:29:37:938 [140603153159936] 2 hub: TSESS-P-8 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:39:37 Mar 30 15:29:37:938 [140603153159936] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:29:37:938 [140603153159936] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:29:37:938 [140603153159936] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:29:37:938 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:29:37:938 [140603153159936] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:29:37:938 [140603153159936] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:29:37:938 [140603153159936] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:29:37:938 [140603153159936] 4 hub: CONNECT: 0x7fe0a0000d90(37) 127.0.0.1/53940->127.0.0.1/48002 Mar 30 15:29:37:938 [140603153159936] 3 hub: TSESS-P-8-13 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:29:37:938 [140603153159936] 1 hub: TSESS-P-8-13 [Secondary] received 100 'hubup' (1209 bytes) Mar 30 15:29:37:938 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53940 (s:912) Mar 30 15:29:37:938 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/53940] Mar 30 15:29:37:938 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:29:37:938 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:29:37:938 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/53940 Mar 30 15:29:37:938 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/53940 same proximity=0 status=0 secver=217 Mar 30 15:29:37:938 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-12] domain [DOMAIN] addr [(null)] ip [100.64.1.22] port [48002] robotname [a0f8us052nim752] version [9.20 [Build 9.20.1048, Aug 8 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-12] source [/DOMAIN/Secondary] from [127.0.0.1/53940] Mar 30 15:29:37:938 [140605547460672] 4 hub: hubup source: /DOMAIN/Secondary Mar 30 15:29:37:938 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-12 Mar 30 15:29:37:938 [140605547460672] 5 hub: hubinfo_update - same proximity but different source (/DOMAIN/Secondary) Mar 30 15:29:37:943 [140603153159936] 5 hub: TSESS-P-8-13 [Secondary] local 200 'OK' Mar 30 15:29:37:943 [140603153159936] 1 hub: TSESS-P-8-13 [Secondary] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:29:37:954 [140603153159936] 1 hub: TSESS-P-8-13 [Secondary] ssl 100 '_close' (298 bytes) Mar 30 15:29:37:954 [140603153159936] 3 hub: TSESS-P-8-13 [Secondary] is paired with [A-2121-13] Mar 30 15:29:37:954 [140603153159936] 1 hub: TSESS-P-8-13 [Secondary] received 100 '_close' (298 bytes) Mar 30 15:29:37:959 [140603153159936] 1 hub: TSESS-P-8-13 [Secondary] remote session closed Mar 30 15:29:37:959 [140603153159936] 3 hub: TSESS-P-8-13 [Secondary] exiting msg loop (remote close) Mar 30 15:29:37:959 [140603153159936] 5 hub: TSESS-P-8-13 [Secondary] suspended Mar 30 15:29:38:701 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:29:38:737 [140603169945344] 5 hub: TSESS-P-7-15 [TunnelB] suspended Mar 30 15:29:38:801 [140603723568896] 5 hub: TSESS-P-7 not internal command - thread will handle. mtype=100; sfd:22 Mar 30 15:29:38:801 [140603723568896] 2 hub: CTRL TunnelB request 'hubup' passed to TSESS-P-7 [140603169945344] Mar 30 15:29:38:801 [140603169945344] 3 hub: TSESS-P-7-16 [TunnelB] restarted (for remote) Mar 30 15:29:38:801 [140603169945344] 2 hub: TSESS-P-7 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:39:38 Mar 30 15:29:38:801 [140603169945344] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:29:38:801 [140603169945344] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:29:38:801 [140603169945344] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:29:38:801 [140603169945344] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:29:38:801 [140603169945344] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:29:38:801 [140603169945344] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:29:38:802 [140603169945344] 4 hub: CONNECT: 0x7fe0a8000d20(37) 127.0.0.1/53942->127.0.0.1/48002 Mar 30 15:29:38:802 [140603169945344] 3 hub: TSESS-P-7-16 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:29:38:802 [140603169945344] 1 hub: TSESS-P-7-16 [TunnelB] received 100 'hubup' (1195 bytes) Mar 30 15:29:38:802 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53942 (s:900) Mar 30 15:29:38:802 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/53942] Mar 30 15:29:38:802 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:29:38:802 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:29:38:802 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/53942 Mar 30 15:29:38:802 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/53942 same proximity=0 status=0 secver=217 Mar 30 15:29:38:802 [140605547460672] 5 hub: hubinfo_update - name [A0F8-USDA-FMMI-12] domain [DOMAIN] addr [(null)] ip [100.64.1.22] port [48002] robotname [a0f8us052nim752] version [9.20 [Build 9.20.1048, Aug 8 2019]] status [0] ssl_mode [0] origin [A0F8-USDA-FMMI-12] source [/DOMAIN/TunnelB] from [127.0.0.1/53942] Mar 30 15:29:38:802 [140605547460672] 4 hub: hubup source: /DOMAIN/TunnelB Mar 30 15:29:38:802 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/A0F8-USDA-FMMI-12 Mar 30 15:29:38:802 [140605547460672] 5 hub: hubinfo_update - for /DOMAIN/A0F8-USDA-FMMI-12/a0f8us052nim752/hub from 127.0.0.1/53942, proximity 5 higher than current 4, ignore update.... Mar 30 15:29:38:807 [140603169945344] 5 hub: TSESS-P-7-16 [TunnelB] local 200 'OK' Mar 30 15:29:38:807 [140603169945344] 1 hub: TSESS-P-7-16 [TunnelB] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:29:38:817 [140603169945344] 1 hub: TSESS-P-7-16 [TunnelB] ssl 100 '_close' (284 bytes) Mar 30 15:29:38:817 [140603169945344] 3 hub: TSESS-P-7-16 [TunnelB] is paired with [A-299-16] Mar 30 15:29:38:817 [140603169945344] 1 hub: TSESS-P-7-16 [TunnelB] received 100 '_close' (296 bytes) Mar 30 15:29:38:822 [140603169945344] 1 hub: TSESS-P-7-16 [TunnelB] remote session closed Mar 30 15:29:38:822 [140603169945344] 3 hub: TSESS-P-7-16 [TunnelB] exiting msg loop (remote close) Mar 30 15:29:38:822 [140603169945344] 5 hub: TSESS-P-7-16 [TunnelB] suspended Mar 30 15:29:39:736 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:41:538 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:41:629 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:29:41:629 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:29:41:629 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33238 Mar 30 15:29:41:629 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33238' delegated to TSESS-A-5 [140603715176192] Mar 30 15:29:41:629 [140603715176192] 3 hub: TSESS-A-5-66 [TunnelB] restarted (for local) Mar 30 15:29:41:634 [140603715176192] 5 hub: TSESS-A-5-66 [TunnelB] local 100 'get_info' Mar 30 15:29:41:634 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:41:635 [140603715176192] 1 hub: TSESS-A-5-66 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:29:41:645 [140603715176192] 1 hub: TSESS-A-5-66 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:29:41:645 [140603715176192] 3 hub: TSESS-A-5-66 [TunnelB] is paired with [P-298-66] Mar 30 15:29:41:645 [140603715176192] 1 hub: TSESS-A-5-66 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:29:41:645 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33240 Mar 30 15:29:41:645 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33240' delegated to TSESS-A-10 [140603706783488] Mar 30 15:29:41:645 [140603706783488] 3 hub: TSESS-A-10-49 [TunnelB] restarted (for local) Mar 30 15:29:41:650 [140603715176192] 5 hub: TSESS-A-5-66 [TunnelB] local 100 '_close' Mar 30 15:29:41:650 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:41:650 [140603715176192] 1 hub: TSESS-A-5-66 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:29:41:650 [140603715176192] 3 hub: TSESS-A-5-66 [TunnelB] local session closed, suspending Mar 30 15:29:41:650 [140603706783488] 5 hub: TSESS-A-10-49 [TunnelB] local 100 '_status' Mar 30 15:29:41:650 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:41:650 [140603706783488] 1 hub: TSESS-A-10-49 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:29:41:660 [140603706783488] 1 hub: TSESS-A-10-49 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:29:41:661 [140603706783488] 3 hub: TSESS-A-10-49 [TunnelB] is paired with [P-300-49] Mar 30 15:29:41:661 [140603706783488] 1 hub: TSESS-A-10-49 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:29:41:666 [140603706783488] 5 hub: TSESS-A-10-49 [TunnelB] local 100 '_close' Mar 30 15:29:41:666 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:41:666 [140603706783488] 1 hub: TSESS-A-10-49 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:29:41:666 [140603706783488] 3 hub: TSESS-A-10-49 [TunnelB] local session closed, suspending Mar 30 15:29:41:850 [140603715176192] 5 hub: TSESS-A-5-66 [TunnelB] suspended Mar 30 15:29:41:866 [140603706783488] 5 hub: TSESS-A-10-49 [TunnelB] suspended Mar 30 15:29:43:340 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:45:143 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:46:098 [140605283890944] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:29:46:098 [140605283890944] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:29:46:098 [140605283890944] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:29:46:098 [140605283890944] 3 hub: hubhash_write - 0 hubs saved to cache Mar 30 15:29:46:098 [140605283890944] 1 hub: PERF: hubup_check_interval: 0 sec hubhash_size: 19 Mar 30 15:29:46:098 [140605283890944] 2 hub: hubupdatelist->Secondary[+]: /DOMAIN/A0F8-USDA-FMMI-12 100.64.1.22/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:46:098 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:098 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:098 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:098 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:098 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:098 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:098 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:29:46:098 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:098 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:098 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:098 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:098 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:098 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:098 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:098 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:098 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:29:46:098 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:29:46:098 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:29:46:099 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(37) 127.0.0.1/47618->127.0.0.1/48007 Mar 30 15:29:46:099 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/47618 Mar 30 15:29:46:099 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/47618' delegated to TSESS-A-6 [140603203516160] Mar 30 15:29:46:099 [140603203516160] 3 hub: TSESS-A-6-16 [Secondary] restarted (for local) Mar 30 15:29:46:104 [140603203516160] 5 hub: TSESS-A-6-16 [Secondary] local 100 'hubup' Mar 30 15:29:46:104 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:46:104 [140603203516160] 1 hub: TSESS-A-6-16 [Secondary] sent 100 'hubup' (1211 bytes, 0 ms) Mar 30 15:29:46:114 [140603203516160] 1 hub: TSESS-A-6-16 [Secondary] ssl 200 '-' (455 bytes) Mar 30 15:29:46:114 [140603203516160] 3 hub: TSESS-A-6-16 [Secondary] is paired with [P-2120-16] Mar 30 15:29:46:114 [140603203516160] 1 hub: TSESS-A-6-16 [Secondary] received 200 '-' (463 bytes) Mar 30 15:29:46:114 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:29:46:114 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:29:46:114 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:29:46:114 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:29:46:114 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary same proximity=0 status=0 secver=217 Mar 30 15:29:46:114 [140605283890944] 2 hub: hubupdatelist->TunnelB[+]: /DOMAIN/A0F8-USDA-FMMI-12 100.64.1.22/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:46:114 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:114 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:114 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:114 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:114 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:114 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:114 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:29:46:114 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:114 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:114 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:114 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:114 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:114 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:114 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:114 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:114 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:29:46:114 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:29:46:114 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:29:46:115 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(37) 127.0.0.1/33250->127.0.0.1/48009 Mar 30 15:29:46:115 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33250 Mar 30 15:29:46:115 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33250' delegated to TSESS-A-5 [140603715176192] Mar 30 15:29:46:115 [140603715176192] 3 hub: TSESS-A-5-67 [TunnelB] restarted (for local) Mar 30 15:29:46:119 [140603203516160] 5 hub: TSESS-A-6-16 [Secondary] local 100 '_close' Mar 30 15:29:46:119 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:46:119 [140603203516160] 1 hub: TSESS-A-6-16 [Secondary] sent 100 '_close' (294 bytes, 0 ms) Mar 30 15:29:46:119 [140603203516160] 3 hub: TSESS-A-6-16 [Secondary] local session closed, suspending Mar 30 15:29:46:120 [140603715176192] 5 hub: TSESS-A-5-67 [TunnelB] local 100 'hubup' Mar 30 15:29:46:120 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:46:120 [140603715176192] 1 hub: TSESS-A-5-67 [TunnelB] sent 100 'hubup' (1209 bytes, 0 ms) Mar 30 15:29:46:130 [140603715176192] 1 hub: TSESS-A-5-67 [TunnelB] ssl 200 '-' (443 bytes) Mar 30 15:29:46:130 [140603715176192] 3 hub: TSESS-A-5-67 [TunnelB] is paired with [P-298-67] Mar 30 15:29:46:130 [140603715176192] 1 hub: TSESS-A-5-67 [TunnelB] received 200 '-' (451 bytes) Mar 30 15:29:46:130 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:29:46:130 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:29:46:130 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:29:46:130 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:29:46:130 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB same proximity=0 status=0 secver=217 Mar 30 15:29:46:130 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-12 100.64.1.22/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:130 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:130 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:130 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:130 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:130 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:130 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:130 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:29:46:130 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:130 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:130 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:130 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:130 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:130 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:130 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:130 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:29:46:130 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:29:46:130 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:29:46:130 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:29:46:130 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(37) 127.0.0.1/40670->127.0.0.1/48008 Mar 30 15:29:46:131 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/40670 Mar 30 15:29:46:131 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/40670' delegated to TSESS-A-3 [140603195123456] Mar 30 15:29:46:131 [140603195123456] 3 hub: TSESS-A-3-16 [SecondaryHA] restarted (for local) Mar 30 15:29:46:135 [140603715176192] 5 hub: TSESS-A-5-67 [TunnelB] local 100 '_close' Mar 30 15:29:46:135 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:46:135 [140603715176192] 1 hub: TSESS-A-5-67 [TunnelB] sent 100 '_close' (292 bytes, 0 ms) Mar 30 15:29:46:135 [140603715176192] 3 hub: TSESS-A-5-67 [TunnelB] local session closed, suspending Mar 30 15:29:46:136 [140603195123456] 5 hub: TSESS-A-3-16 [SecondaryHA] local 100 'hubup' Mar 30 15:29:46:136 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:46:136 [140603195123456] 1 hub: TSESS-A-3-16 [SecondaryHA] sent 100 'hubup' (689 bytes, 0 ms) Mar 30 15:29:46:146 [140603195123456] 1 hub: TSESS-A-3-16 [SecondaryHA] ssl 200 '-' (459 bytes) Mar 30 15:29:46:146 [140603195123456] 3 hub: TSESS-A-3-16 [SecondaryHA] is paired with [P-1947-16] Mar 30 15:29:46:146 [140603195123456] 1 hub: TSESS-A-3-16 [SecondaryHA] received 200 '-' (467 bytes) Mar 30 15:29:46:146 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:29:46:146 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:29:46:146 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:29:46:146 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:29:46:146 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA same proximity=0 status=0 secver=217 Mar 30 15:29:46:146 [140605283890944] 5 hub: mark_hubup_list_updated Mar 30 15:29:46:146 [140605283890944] 5 hub: hublist chnaged reset: /DOMAIN/A0F8-USDA-FMMI-12 100.64.1.22/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:46:146 [140605283890944] 5 hub: hublist changed: /DOMAIN/Primary 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:46:146 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:46:146 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:46:146 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:46:146 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:46:146 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:46:146 [140605283890944] 5 hub: hublist changed: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:29:46:146 [140605283890944] 5 hub: hublist changed: /DOMAIN/PrimaryHA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:46:146 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:46:146 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:46:146 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:46:146 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:46:146 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:46:146 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:46:146 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:29:46:146 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:29:46:146 [140605283890944] 5 hub: hublist changed: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:29:46:146 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelBHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:29:46:146 [140605283890944] 5 hub: mark_hubup_list_updated end Mar 30 15:29:46:146 [140605283890944] 1 hub: PERF: hubup_check_for_updates: 0 sec hubhash_size: 19 Mar 30 15:29:46:146 [140605283890944] 1 hub: PERF: hubi_run_command: 0 sec hubhash_size: 19 Mar 30 15:29:46:146 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:29:46:147 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:29:46:147 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:29:46:147 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:29:46:148 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:29:46:148 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:29:46:148 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:29:46:149 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:29:46:149 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:29:46:149 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:29:46:149 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:29:46:150 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:29:46:150 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:29:46:151 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:29:46:151 [140603195123456] 5 hub: TSESS-A-3-16 [SecondaryHA] local 100 '_close' Mar 30 15:29:46:151 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:46:151 [140603195123456] 1 hub: TSESS-A-3-16 [SecondaryHA] sent 100 '_close' (296 bytes, 0 ms) Mar 30 15:29:46:151 [140603195123456] 3 hub: TSESS-A-3-16 [SecondaryHA] local session closed, suspending Mar 30 15:29:46:151 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:29:46:152 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:29:46:152 [140605283890944] 1 hub: PERF: hubi_check_security: 0 sec hubhash_size: 19 Mar 30 15:29:46:320 [140603203516160] 5 hub: TSESS-A-6-16 [Secondary] suspended Mar 30 15:29:46:336 [140603715176192] 5 hub: TSESS-A-5-67 [TunnelB] suspended Mar 30 15:29:46:351 [140603195123456] 5 hub: TSESS-A-3-16 [SecondaryHA] suspended Mar 30 15:29:46:961 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:29:46:990 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:29:47:021 [140605547460672] 4 hub: medium timeout Mar 30 15:29:47:021 [140605547460672] 3 hub: next_check [1585596696] now [1585596587] Mar 30 15:29:47:021 [140605547460672] 2 hub: verify_login from ctrl 127.0.0.1/53980 Mar 30 15:29:47:021 [140605547460672] 1 hub: verify - [target] prid=controller cmd=probe_list ip=10.x.x.x perm=1 Mar 30 15:29:47:021 [140605547460672] 1 hub: verify - [source] id=ppm ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:47:607 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:29:47:765 [140603161552640] 5 hub: TSESS-P-4-16 [SecondaryHA] suspended Mar 30 15:29:47:846 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:47:938 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:29:47:959 [140603153159936] 5 hub: TSESS-P-8-13 [Secondary] suspended Mar 30 15:29:48:702 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:29:48:822 [140603169945344] 5 hub: TSESS-P-7-16 [TunnelB] suspended Mar 30 15:29:49:648 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:51:451 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:51:672 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:29:51:672 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:29:51:673 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33276 Mar 30 15:29:51:673 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33276' delegated to TSESS-A-5 [140603715176192] Mar 30 15:29:51:673 [140603715176192] 3 hub: TSESS-A-5-68 [TunnelB] restarted (for local) Mar 30 15:29:51:678 [140603715176192] 5 hub: TSESS-A-5-68 [TunnelB] local 100 'get_info' Mar 30 15:29:51:678 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:51:678 [140603715176192] 1 hub: TSESS-A-5-68 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:29:51:688 [140603715176192] 1 hub: TSESS-A-5-68 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:29:51:688 [140603715176192] 3 hub: TSESS-A-5-68 [TunnelB] is paired with [P-298-68] Mar 30 15:29:51:688 [140603715176192] 1 hub: TSESS-A-5-68 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:29:51:689 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33278 Mar 30 15:29:51:689 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33278' delegated to TSESS-A-10 [140603706783488] Mar 30 15:29:51:689 [140603706783488] 3 hub: TSESS-A-10-50 [TunnelB] restarted (for local) Mar 30 15:29:51:693 [140603715176192] 5 hub: TSESS-A-5-68 [TunnelB] local 100 '_close' Mar 30 15:29:51:693 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:51:694 [140603715176192] 1 hub: TSESS-A-5-68 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:29:51:694 [140603715176192] 3 hub: TSESS-A-5-68 [TunnelB] local session closed, suspending Mar 30 15:29:51:694 [140603706783488] 5 hub: TSESS-A-10-50 [TunnelB] local 100 '_status' Mar 30 15:29:51:694 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:51:694 [140603706783488] 1 hub: TSESS-A-10-50 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:29:51:704 [140603706783488] 1 hub: TSESS-A-10-50 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:29:51:704 [140603706783488] 3 hub: TSESS-A-10-50 [TunnelB] is paired with [P-300-50] Mar 30 15:29:51:704 [140603706783488] 1 hub: TSESS-A-10-50 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:29:51:709 [140603706783488] 5 hub: TSESS-A-10-50 [TunnelB] local 100 '_close' Mar 30 15:29:51:709 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:29:51:709 [140603706783488] 1 hub: TSESS-A-10-50 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:29:51:709 [140603706783488] 3 hub: TSESS-A-10-50 [TunnelB] local session closed, suspending Mar 30 15:29:51:894 [140603715176192] 5 hub: TSESS-A-5-68 [TunnelB] suspended Mar 30 15:29:51:910 [140603706783488] 5 hub: TSESS-A-10-50 [TunnelB] suspended Mar 30 15:29:53:254 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:55:057 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:56:320 [140603203516160] 5 hub: TSESS-A-6-16 [Secondary] suspended Mar 30 15:29:56:352 [140603195123456] 5 hub: TSESS-A-3-16 [SecondaryHA] suspended Mar 30 15:29:56:962 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:29:56:990 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:29:57:607 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:29:57:675 [140605547460672] 4 hub: medium timeout Mar 30 15:29:57:676 [140605547460672] 3 hub: next_check [1585596696] now [1585596597] Mar 30 15:29:57:760 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:29:57:765 [140603161552640] 5 hub: TSESS-P-4-16 [SecondaryHA] suspended Mar 30 15:29:57:938 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:29:57:959 [140603153159936] 5 hub: TSESS-P-8-13 [Secondary] suspended Mar 30 15:29:58:702 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:29:58:823 [140603169945344] 5 hub: TSESS-P-7-16 [TunnelB] suspended Mar 30 15:29:59:563 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:01:365 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:01:680 [140605547460672] 1 hub: perf_wfile: sameday = 1 Mar 30 15:30:01:716 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:30:01:717 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:30:01:717 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33284 Mar 30 15:30:01:717 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33284' delegated to TSESS-A-5 [140603715176192] Mar 30 15:30:01:717 [140603715176192] 3 hub: TSESS-A-5-69 [TunnelB] restarted (for local) Mar 30 15:30:01:724 [140603715176192] 5 hub: TSESS-A-5-69 [TunnelB] local 100 'get_info' Mar 30 15:30:01:724 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:01:724 [140603715176192] 1 hub: TSESS-A-5-69 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:30:01:734 [140603715176192] 1 hub: TSESS-A-5-69 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:30:01:735 [140603715176192] 3 hub: TSESS-A-5-69 [TunnelB] is paired with [P-298-69] Mar 30 15:30:01:735 [140603715176192] 1 hub: TSESS-A-5-69 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:30:01:736 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33286 Mar 30 15:30:01:736 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33286' delegated to TSESS-A-10 [140603706783488] Mar 30 15:30:01:737 [140603706783488] 3 hub: TSESS-A-10-51 [TunnelB] restarted (for local) Mar 30 15:30:01:740 [140603715176192] 5 hub: TSESS-A-5-69 [TunnelB] local 100 '_close' Mar 30 15:30:01:740 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:01:740 [140603715176192] 1 hub: TSESS-A-5-69 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:30:01:740 [140603715176192] 3 hub: TSESS-A-5-69 [TunnelB] local session closed, suspending Mar 30 15:30:01:742 [140603706783488] 5 hub: TSESS-A-10-51 [TunnelB] local 100 '_status' Mar 30 15:30:01:742 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:01:742 [140603706783488] 1 hub: TSESS-A-10-51 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:30:01:754 [140603706783488] 1 hub: TSESS-A-10-51 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:30:01:754 [140603706783488] 3 hub: TSESS-A-10-51 [TunnelB] is paired with [P-300-51] Mar 30 15:30:01:754 [140603706783488] 1 hub: TSESS-A-10-51 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:30:01:759 [140603706783488] 5 hub: TSESS-A-10-51 [TunnelB] local 100 '_close' Mar 30 15:30:01:759 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:01:759 [140603706783488] 1 hub: TSESS-A-10-51 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:30:01:759 [140603706783488] 3 hub: TSESS-A-10-51 [TunnelB] local session closed, suspending Mar 30 15:30:01:940 [140603715176192] 5 hub: TSESS-A-5-69 [TunnelB] suspended Mar 30 15:30:01:960 [140603706783488] 5 hub: TSESS-A-10-51 [TunnelB] suspended Mar 30 15:30:03:168 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:05:721 [140605547460672] 3 hub: check_session - session 0x20f9050 (127.0.0.1/48002) is a server Mar 30 15:30:05:721 [140605547460672] 2 hub: Profile - clayk 38 bytes (OK) Mar 30 15:30:05:871 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:06:320 [140603203516160] 5 hub: TSESS-A-6-16 [Secondary] suspended Mar 30 15:30:06:352 [140603195123456] 5 hub: TSESS-A-3-16 [SecondaryHA] suspended Mar 30 15:30:06:962 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:30:06:990 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:30:07:083 [140603740354304] 5 hub: TSESS-P-8 not internal command - thread will handle. mtype=100; sfd:23 Mar 30 15:30:07:083 [140603740354304] 2 hub: CTRL Secondary request 'hubup' passed to TSESS-P-8 [140603153159936] Mar 30 15:30:07:083 [140603153159936] 3 hub: TSESS-P-8-14 [Secondary] restarted (for remote) Mar 30 15:30:07:083 [140603153159936] 2 hub: TSESS-P-8 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:40:07 Mar 30 15:30:07:083 [140603153159936] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:30:07:083 [140603153159936] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:30:07:083 [140603153159936] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:30:07:083 [140603153159936] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:30:07:083 [140603153159936] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:30:07:083 [140603153159936] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:30:07:083 [140603153159936] 4 hub: CONNECT: 0x7fe0a0000d90(37) 127.0.0.1/53998->127.0.0.1/48002 Mar 30 15:30:07:083 [140603153159936] 3 hub: TSESS-P-8-14 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:30:07:083 [140603153159936] 1 hub: TSESS-P-8-14 [Secondary] received 100 'hubup' (685 bytes) Mar 30 15:30:07:083 [140605547460672] 4 hub: medium timeout Mar 30 15:30:07:083 [140605547460672] 3 hub: next_check [1585596696] now [1585596607] Mar 30 15:30:07:083 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/53998 (s:388) Mar 30 15:30:07:083 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/53998] Mar 30 15:30:07:083 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:30:07:083 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:30:07:083 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/53998 Mar 30 15:30:07:083 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/53998 same proximity=0 status=0 secver=217 Mar 30 15:30:07:088 [140603153159936] 5 hub: TSESS-P-8-14 [Secondary] local 200 'OK' Mar 30 15:30:07:088 [140603153159936] 1 hub: TSESS-P-8-14 [Secondary] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:30:07:099 [140603153159936] 1 hub: TSESS-P-8-14 [Secondary] ssl 100 '_close' (298 bytes) Mar 30 15:30:07:099 [140603153159936] 3 hub: TSESS-P-8-14 [Secondary] is paired with [A-2121-14] Mar 30 15:30:07:099 [140603153159936] 1 hub: TSESS-P-8-14 [Secondary] received 100 '_close' (298 bytes) Mar 30 15:30:07:104 [140603153159936] 1 hub: TSESS-P-8-14 [Secondary] remote session closed Mar 30 15:30:07:104 [140603153159936] 3 hub: TSESS-P-8-14 [Secondary] exiting msg loop (remote close) Mar 30 15:30:07:104 [140603153159936] 5 hub: TSESS-P-8-14 [Secondary] suspended Mar 30 15:30:07:607 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:30:07:674 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:07:766 [140603161552640] 5 hub: TSESS-P-4-16 [SecondaryHA] suspended Mar 30 15:30:07:895 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:30:07:895 [140603731961600] 2 hub: CTRL SecondaryHA request 'hubup' passed to TSESS-P-4 [140603161552640] Mar 30 15:30:07:895 [140603161552640] 3 hub: TSESS-P-4-17 [SecondaryHA] restarted (for remote) Mar 30 15:30:07:895 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:40:07 Mar 30 15:30:07:895 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:30:07:896 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:30:07:896 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:30:07:896 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:30:07:896 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:30:07:896 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:30:07:896 [140603161552640] 4 hub: CONNECT: 0x7fe09c000cb0(37) 127.0.0.1/54000->127.0.0.1/48002 Mar 30 15:30:07:896 [140603161552640] 3 hub: TSESS-P-4-17 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:30:07:896 [140603161552640] 1 hub: TSESS-P-4-17 [SecondaryHA] received 100 'hubup' (689 bytes) Mar 30 15:30:07:896 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/54000 (s:392) Mar 30 15:30:07:896 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/54000] Mar 30 15:30:07:896 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:30:07:896 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:30:07:896 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/54000 Mar 30 15:30:07:896 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/54000 same proximity=0 status=0 secver=217 Mar 30 15:30:07:901 [140603161552640] 5 hub: TSESS-P-4-17 [SecondaryHA] local 200 'OK' Mar 30 15:30:07:901 [140603161552640] 1 hub: TSESS-P-4-17 [SecondaryHA] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:30:07:912 [140603161552640] 1 hub: TSESS-P-4-17 [SecondaryHA] ssl 100 '_close' (302 bytes) Mar 30 15:30:07:912 [140603161552640] 3 hub: TSESS-P-4-17 [SecondaryHA] is paired with [A-1948-17] Mar 30 15:30:07:913 [140603161552640] 1 hub: TSESS-P-4-17 [SecondaryHA] received 100 '_close' (298 bytes) Mar 30 15:30:07:918 [140603161552640] 1 hub: TSESS-P-4-17 [SecondaryHA] remote session closed Mar 30 15:30:07:918 [140603161552640] 3 hub: TSESS-P-4-17 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:30:07:918 [140603161552640] 5 hub: TSESS-P-4-17 [SecondaryHA] suspended Mar 30 15:30:07:938 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:30:08:702 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:30:08:823 [140603169945344] 5 hub: TSESS-P-7-16 [TunnelB] suspended Mar 30 15:30:08:886 [140603723568896] 5 hub: TSESS-P-7 not internal command - thread will handle. mtype=100; sfd:22 Mar 30 15:30:08:886 [140603723568896] 2 hub: CTRL TunnelB request 'hubup' passed to TSESS-P-7 [140603169945344] Mar 30 15:30:08:886 [140603169945344] 3 hub: TSESS-P-7-17 [TunnelB] restarted (for remote) Mar 30 15:30:08:886 [140603169945344] 2 hub: TSESS-P-7 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:40:08 Mar 30 15:30:08:886 [140603169945344] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:30:08:886 [140603169945344] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:30:08:886 [140603169945344] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:30:08:886 [140603169945344] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:30:08:886 [140603169945344] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:30:08:886 [140603169945344] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:30:08:887 [140603169945344] 4 hub: CONNECT: 0x7fe0a8000d20(37) 127.0.0.1/54002->127.0.0.1/48002 Mar 30 15:30:08:887 [140603169945344] 3 hub: TSESS-P-7-17 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:30:08:887 [140603169945344] 1 hub: TSESS-P-7-17 [TunnelB] received 100 'hubup' (673 bytes) Mar 30 15:30:08:887 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/54002 (s:378) Mar 30 15:30:08:887 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/54002] Mar 30 15:30:08:887 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:30:08:887 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:30:08:887 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/54002 Mar 30 15:30:08:887 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/54002 same proximity=0 status=0 secver=217 Mar 30 15:30:08:892 [140603169945344] 5 hub: TSESS-P-7-17 [TunnelB] local 200 'OK' Mar 30 15:30:08:892 [140603169945344] 1 hub: TSESS-P-7-17 [TunnelB] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:30:08:902 [140603169945344] 1 hub: TSESS-P-7-17 [TunnelB] ssl 100 '_close' (284 bytes) Mar 30 15:30:08:902 [140603169945344] 3 hub: TSESS-P-7-17 [TunnelB] is paired with [A-299-17] Mar 30 15:30:08:902 [140603169945344] 1 hub: TSESS-P-7-17 [TunnelB] received 100 '_close' (296 bytes) Mar 30 15:30:08:907 [140603169945344] 1 hub: TSESS-P-7-17 [TunnelB] remote session closed Mar 30 15:30:08:907 [140603169945344] 3 hub: TSESS-P-7-17 [TunnelB] exiting msg loop (remote close) Mar 30 15:30:08:907 [140603169945344] 5 hub: TSESS-P-7-17 [TunnelB] suspended Mar 30 15:30:09:476 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:11:279 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:11:767 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:30:11:767 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:30:11:767 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33302 Mar 30 15:30:11:767 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33302' delegated to TSESS-A-5 [140603715176192] Mar 30 15:30:11:767 [140603715176192] 3 hub: TSESS-A-5-70 [TunnelB] restarted (for local) Mar 30 15:30:11:773 [140603715176192] 5 hub: TSESS-A-5-70 [TunnelB] local 100 'get_info' Mar 30 15:30:11:773 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:11:773 [140603715176192] 1 hub: TSESS-A-5-70 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:30:11:783 [140603715176192] 1 hub: TSESS-A-5-70 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:30:11:783 [140603715176192] 3 hub: TSESS-A-5-70 [TunnelB] is paired with [P-298-70] Mar 30 15:30:11:783 [140603715176192] 1 hub: TSESS-A-5-70 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:30:11:783 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33304 Mar 30 15:30:11:783 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33304' delegated to TSESS-A-10 [140603706783488] Mar 30 15:30:11:783 [140603706783488] 3 hub: TSESS-A-10-52 [TunnelB] restarted (for local) Mar 30 15:30:11:788 [140603715176192] 5 hub: TSESS-A-5-70 [TunnelB] local 100 '_close' Mar 30 15:30:11:788 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:11:788 [140603715176192] 1 hub: TSESS-A-5-70 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:30:11:788 [140603715176192] 3 hub: TSESS-A-5-70 [TunnelB] local session closed, suspending Mar 30 15:30:11:789 [140603706783488] 5 hub: TSESS-A-10-52 [TunnelB] local 100 '_status' Mar 30 15:30:11:789 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:11:789 [140603706783488] 1 hub: TSESS-A-10-52 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:30:11:799 [140603706783488] 1 hub: TSESS-A-10-52 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:30:11:799 [140603706783488] 3 hub: TSESS-A-10-52 [TunnelB] is paired with [P-300-52] Mar 30 15:30:11:799 [140603706783488] 1 hub: TSESS-A-10-52 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:30:11:804 [140603706783488] 5 hub: TSESS-A-10-52 [TunnelB] local 100 '_close' Mar 30 15:30:11:804 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:11:804 [140603706783488] 1 hub: TSESS-A-10-52 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:30:11:804 [140603706783488] 3 hub: TSESS-A-10-52 [TunnelB] local session closed, suspending Mar 30 15:30:11:989 [140603715176192] 5 hub: TSESS-A-5-70 [TunnelB] suspended Mar 30 15:30:12:004 [140603706783488] 5 hub: TSESS-A-10-52 [TunnelB] suspended Mar 30 15:30:12:506 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:30:12:506 [140603731961600] 2 hub: CTRL SecondaryHA request 'getrobots' passed to TSESS-P-4 [140603161552640] Mar 30 15:30:12:506 [140603161552640] 3 hub: TSESS-P-4-18 [SecondaryHA] restarted (for remote) Mar 30 15:30:12:506 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=administrator hub=SecondaryHA ip=10.x.x.x type=0 expire=2020-03-30 21:30:08 Mar 30 15:30:12:506 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:30:12:506 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:30:12:506 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:30:12:506 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:30:12:506 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:30:12:506 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:30:12:506 [140603161552640] 4 hub: CONNECT: 0x7fe09c000cb0(37) 127.0.0.1/54016->127.0.0.1/48002 Mar 30 15:30:12:506 [140603161552640] 3 hub: TSESS-P-4-18 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:30:12:506 [140603161552640] 1 hub: TSESS-P-4-18 [SecondaryHA] received 100 'getrobots' (331 bytes) Mar 30 15:30:12:506 [140605547460672] 2 hub: nimSockaddr2Ip - got 127.0.0.1, using robotip = 127.0.0.1 Mar 30 15:30:12:506 [140605547460672] 4 hub: nimSockaddr2Ip - found 127.0.0.1 Mar 30 15:30:12:506 [140605547460672] 2 hub: nimVerifyLogin hub Mar 30 15:30:12:506 [140605547460672] 1 hub: verify - [target] prid=hub cmd=getrobots ip=127.0.0.1 perm=1 Mar 30 15:30:12:506 [140605547460672] 1 hub: verify - [source] id=administrator ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:12:506 [140605547460672] 1 hub: getrobots - name=* detail=1 from 127.0.0.1/54016 (robots=1) Mar 30 15:30:12:506 [140605547460672] 2 hub: getrobots - used 0 ms on 1 robots, pds size=6894 Mar 30 15:30:12:511 [140603161552640] 5 hub: TSESS-P-4-18 [SecondaryHA] local 200 'OK' Mar 30 15:30:12:511 [140603161552640] 1 hub: TSESS-P-4-18 [SecondaryHA] sent 200 'OK' (6957 bytes, 0 ms) Mar 30 15:30:12:583 [140603161552640] 1 hub: TSESS-P-4-18 [SecondaryHA] ssl 100 '_close' (314 bytes) Mar 30 15:30:12:583 [140603161552640] 3 hub: TSESS-P-4-18 [SecondaryHA] is paired with [A-1948-18] Mar 30 15:30:12:583 [140603161552640] 1 hub: TSESS-P-4-18 [SecondaryHA] received 100 '_close' (314 bytes) Mar 30 15:30:12:588 [140603161552640] 1 hub: TSESS-P-4-18 [SecondaryHA] remote session closed Mar 30 15:30:12:588 [140603161552640] 3 hub: TSESS-P-4-18 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:30:12:588 [140603161552640] 5 hub: TSESS-P-4-18 [SecondaryHA] suspended Mar 30 15:30:13:081 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:15:785 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:16:184 [140605283890944] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:30:16:184 [140605283890944] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:30:16:184 [140605283890944] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:30:16:184 [140605283890944] 1 hub: PERF: hubup_check_interval: 0 sec hubhash_size: 19 Mar 30 15:30:16:184 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-12 100.64.1.22/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:184 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:184 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:184 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:184 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:184 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:184 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:184 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:30:16:184 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:184 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:184 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:184 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:184 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:184 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:184 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:184 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:184 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:30:16:184 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:30:16:184 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:30:16:185 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(37) 127.0.0.1/47678->127.0.0.1/48007 Mar 30 15:30:16:185 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/47678 Mar 30 15:30:16:185 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/47678' delegated to TSESS-A-6 [140603203516160] Mar 30 15:30:16:185 [140603203516160] 3 hub: TSESS-A-6-17 [Secondary] restarted (for local) Mar 30 15:30:16:190 [140603203516160] 5 hub: TSESS-A-6-17 [Secondary] local 100 'hubup' Mar 30 15:30:16:190 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:16:190 [140603203516160] 1 hub: TSESS-A-6-17 [Secondary] sent 100 'hubup' (687 bytes, 0 ms) Mar 30 15:30:16:200 [140603203516160] 1 hub: TSESS-A-6-17 [Secondary] ssl 200 '-' (455 bytes) Mar 30 15:30:16:200 [140603203516160] 3 hub: TSESS-A-6-17 [Secondary] is paired with [P-2120-17] Mar 30 15:30:16:200 [140603203516160] 1 hub: TSESS-A-6-17 [Secondary] received 200 '-' (463 bytes) Mar 30 15:30:16:200 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:30:16:200 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:30:16:200 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:30:16:200 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:30:16:200 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary same proximity=0 status=0 secver=217 Mar 30 15:30:16:200 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-12 100.64.1.22/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:200 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:200 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:200 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:200 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:200 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:200 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:200 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:30:16:200 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:200 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:200 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:200 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:200 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:200 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:200 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:200 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:200 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:30:16:200 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:30:16:200 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:30:16:201 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(37) 127.0.0.1/33310->127.0.0.1/48009 Mar 30 15:30:16:201 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33310 Mar 30 15:30:16:201 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33310' delegated to TSESS-A-5 [140603715176192] Mar 30 15:30:16:201 [140603715176192] 3 hub: TSESS-A-5-71 [TunnelB] restarted (for local) Mar 30 15:30:16:205 [140603203516160] 5 hub: TSESS-A-6-17 [Secondary] local 100 '_close' Mar 30 15:30:16:205 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:16:205 [140603203516160] 1 hub: TSESS-A-6-17 [Secondary] sent 100 '_close' (294 bytes, 0 ms) Mar 30 15:30:16:205 [140603203516160] 3 hub: TSESS-A-6-17 [Secondary] local session closed, suspending Mar 30 15:30:16:206 [140603715176192] 5 hub: TSESS-A-5-71 [TunnelB] local 100 'hubup' Mar 30 15:30:16:206 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:16:206 [140603715176192] 1 hub: TSESS-A-5-71 [TunnelB] sent 100 'hubup' (685 bytes, 0 ms) Mar 30 15:30:16:216 [140603715176192] 1 hub: TSESS-A-5-71 [TunnelB] ssl 200 '-' (443 bytes) Mar 30 15:30:16:216 [140603715176192] 3 hub: TSESS-A-5-71 [TunnelB] is paired with [P-298-71] Mar 30 15:30:16:216 [140603715176192] 1 hub: TSESS-A-5-71 [TunnelB] received 200 '-' (451 bytes) Mar 30 15:30:16:216 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:30:16:216 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:30:16:216 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:30:16:216 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:30:16:216 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB same proximity=0 status=0 secver=217 Mar 30 15:30:16:216 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-12 100.64.1.22/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:216 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:216 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:216 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:216 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:216 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:216 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:216 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:30:16:216 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:216 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:216 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:216 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:216 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:216 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:216 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:216 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:16:216 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:30:16:216 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:30:16:216 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:30:16:217 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(37) 127.0.0.1/40730->127.0.0.1/48008 Mar 30 15:30:16:217 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/40730 Mar 30 15:30:16:217 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/40730' delegated to TSESS-A-3 [140603195123456] Mar 30 15:30:16:217 [140603195123456] 3 hub: TSESS-A-3-17 [SecondaryHA] restarted (for local) Mar 30 15:30:16:221 [140603715176192] 5 hub: TSESS-A-5-71 [TunnelB] local 100 '_close' Mar 30 15:30:16:221 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:16:221 [140603715176192] 1 hub: TSESS-A-5-71 [TunnelB] sent 100 '_close' (292 bytes, 0 ms) Mar 30 15:30:16:221 [140603715176192] 3 hub: TSESS-A-5-71 [TunnelB] local session closed, suspending Mar 30 15:30:16:222 [140603195123456] 5 hub: TSESS-A-3-17 [SecondaryHA] local 100 'hubup' Mar 30 15:30:16:222 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:16:222 [140603195123456] 1 hub: TSESS-A-3-17 [SecondaryHA] sent 100 'hubup' (689 bytes, 0 ms) Mar 30 15:30:16:232 [140603195123456] 1 hub: TSESS-A-3-17 [SecondaryHA] ssl 200 '-' (459 bytes) Mar 30 15:30:16:232 [140603195123456] 3 hub: TSESS-A-3-17 [SecondaryHA] is paired with [P-1947-17] Mar 30 15:30:16:232 [140603195123456] 1 hub: TSESS-A-3-17 [SecondaryHA] received 200 '-' (467 bytes) Mar 30 15:30:16:232 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:30:16:232 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:30:16:232 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:30:16:232 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:30:16:232 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA same proximity=0 status=0 secver=217 Mar 30 15:30:16:232 [140605283890944] 5 hub: mark_hubup_list_updated Mar 30 15:30:16:232 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-12 100.64.1.22/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:16:232 [140605283890944] 5 hub: hublist changed: /DOMAIN/Primary 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:16:232 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:16:232 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:16:232 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:16:232 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:16:232 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:16:232 [140605283890944] 5 hub: hublist changed: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:30:16:232 [140605283890944] 5 hub: hublist changed: /DOMAIN/PrimaryHA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:16:232 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:16:232 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:16:232 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:16:232 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:16:232 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:16:232 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:16:232 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:16:232 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:30:16:232 [140605283890944] 5 hub: hublist changed: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:30:16:232 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelBHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:30:16:232 [140605283890944] 5 hub: mark_hubup_list_updated end Mar 30 15:30:16:232 [140605283890944] 1 hub: PERF: hubup_check_for_updates: 0 sec hubhash_size: 19 Mar 30 15:30:16:233 [140605283890944] 1 hub: PERF: hubi_run_command: 0 sec hubhash_size: 19 Mar 30 15:30:16:233 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:30:16:233 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:30:16:234 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:30:16:234 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:30:16:234 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:30:16:235 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:30:16:235 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:30:16:236 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:30:16:236 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:30:16:236 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:30:16:236 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:30:16:237 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:30:16:237 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:30:16:237 [140603195123456] 5 hub: TSESS-A-3-17 [SecondaryHA] local 100 '_close' Mar 30 15:30:16:237 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:16:237 [140603195123456] 1 hub: TSESS-A-3-17 [SecondaryHA] sent 100 '_close' (296 bytes, 0 ms) Mar 30 15:30:16:237 [140603195123456] 3 hub: TSESS-A-3-17 [SecondaryHA] local session closed, suspending Mar 30 15:30:16:238 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:30:16:238 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:30:16:239 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:30:16:240 [140605283890944] 1 hub: PERF: hubi_check_security: 0 sec hubhash_size: 19 Mar 30 15:30:16:406 [140603203516160] 5 hub: TSESS-A-6-17 [Secondary] suspended Mar 30 15:30:16:422 [140603715176192] 5 hub: TSESS-A-5-71 [TunnelB] suspended Mar 30 15:30:16:435 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:30:16:435 [140603731961600] 2 hub: CTRL SecondaryHA request 'probe_list' passed to TSESS-P-4 [140603161552640] Mar 30 15:30:16:435 [140603161552640] 3 hub: TSESS-P-4-19 [SecondaryHA] restarted (for remote) Mar 30 15:30:16:435 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=administrator hub=SecondaryHA ip=10.x.x.x type=0 expire=2020-03-30 21:30:07 Mar 30 15:30:16:435 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:30:16:435 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:30:16:435 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:30:16:435 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/controller Mar 30 15:30:16:435 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/controller 127.0.0.1/48000 Mar 30 15:30:16:435 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/controller 127.0.0.1/48000 - 0 ms (nns) Mar 30 15:30:16:435 [140603161552640] 4 hub: CONNECT: 0x7fe09c000c90(37) 127.0.0.1/45128->127.0.0.1/48000 Mar 30 15:30:16:435 [140603161552640] 3 hub: TSESS-P-4-19 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/controller (127.0.0.1/48000) Mar 30 15:30:16:435 [140603161552640] 1 hub: TSESS-P-4-19 [SecondaryHA] received 100 'probe_list' (326 bytes) Mar 30 15:30:16:436 [140605547460672] 2 hub: verify_login from ctrl 127.0.0.1/54026 Mar 30 15:30:16:436 [140605547460672] 1 hub: verify - [target] prid=controller cmd=probe_list ip=10.x.x.x perm=1 Mar 30 15:30:16:436 [140605547460672] 1 hub: verify - [source] id=administrator ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:16:438 [140603195123456] 5 hub: TSESS-A-3-17 [SecondaryHA] suspended Mar 30 15:30:16:441 [140603161552640] 5 hub: TSESS-P-4-19 [SecondaryHA] local 200 'OK' Mar 30 15:30:16:441 [140603161552640] 1 hub: TSESS-P-4-19 [SecondaryHA] sent 200 'OK' (5030 bytes, 0 ms) Mar 30 15:30:16:512 [140603161552640] 1 hub: TSESS-P-4-19 [SecondaryHA] ssl 100 '_close' (321 bytes) Mar 30 15:30:16:512 [140603161552640] 3 hub: TSESS-P-4-19 [SecondaryHA] is paired with [A-1948-19] Mar 30 15:30:16:512 [140603161552640] 1 hub: TSESS-P-4-19 [SecondaryHA] received 100 '_close' (321 bytes) Mar 30 15:30:16:517 [140603161552640] 1 hub: TSESS-P-4-19 [SecondaryHA] remote session closed Mar 30 15:30:16:517 [140603161552640] 3 hub: TSESS-P-4-19 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:30:16:517 [140603161552640] 5 hub: TSESS-P-4-19 [SecondaryHA] suspended Mar 30 15:30:16:962 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:30:16:990 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:30:17:104 [140603153159936] 5 hub: TSESS-P-8-14 [Secondary] suspended Mar 30 15:30:17:437 [140605547460672] 4 hub: medium timeout Mar 30 15:30:17:437 [140605547460672] 3 hub: next_check [1585596696] now [1585596617] Mar 30 15:30:17:587 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:17:607 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:30:17:939 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:30:18:702 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:30:18:907 [140603169945344] 5 hub: TSESS-P-7-17 [TunnelB] suspended Mar 30 15:30:19:390 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:21:192 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:21:810 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:30:21:810 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:30:21:811 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33322 Mar 30 15:30:21:811 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33322' delegated to TSESS-A-5 [140603715176192] Mar 30 15:30:21:811 [140603715176192] 3 hub: TSESS-A-5-72 [TunnelB] restarted (for local) Mar 30 15:30:21:816 [140603715176192] 5 hub: TSESS-A-5-72 [TunnelB] local 100 'get_info' Mar 30 15:30:21:816 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:21:816 [140603715176192] 1 hub: TSESS-A-5-72 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:30:21:826 [140603715176192] 1 hub: TSESS-A-5-72 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:30:21:826 [140603715176192] 3 hub: TSESS-A-5-72 [TunnelB] is paired with [P-298-72] Mar 30 15:30:21:826 [140603715176192] 1 hub: TSESS-A-5-72 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:30:21:827 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33324 Mar 30 15:30:21:827 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33324' delegated to TSESS-A-10 [140603706783488] Mar 30 15:30:21:827 [140603706783488] 3 hub: TSESS-A-10-53 [TunnelB] restarted (for local) Mar 30 15:30:21:831 [140603715176192] 5 hub: TSESS-A-5-72 [TunnelB] local 100 '_close' Mar 30 15:30:21:831 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:21:832 [140603715176192] 1 hub: TSESS-A-5-72 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:30:21:832 [140603715176192] 3 hub: TSESS-A-5-72 [TunnelB] local session closed, suspending Mar 30 15:30:21:832 [140603706783488] 5 hub: TSESS-A-10-53 [TunnelB] local 100 '_status' Mar 30 15:30:21:832 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:21:832 [140603706783488] 1 hub: TSESS-A-10-53 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:30:21:842 [140603706783488] 1 hub: TSESS-A-10-53 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:30:21:842 [140603706783488] 3 hub: TSESS-A-10-53 [TunnelB] is paired with [P-300-53] Mar 30 15:30:21:842 [140603706783488] 1 hub: TSESS-A-10-53 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:30:21:847 [140603706783488] 5 hub: TSESS-A-10-53 [TunnelB] local 100 '_close' Mar 30 15:30:21:847 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:21:847 [140603706783488] 1 hub: TSESS-A-10-53 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:30:21:847 [140603706783488] 3 hub: TSESS-A-10-53 [TunnelB] local session closed, suspending Mar 30 15:30:22:032 [140603715176192] 5 hub: TSESS-A-5-72 [TunnelB] suspended Mar 30 15:30:22:048 [140603706783488] 5 hub: TSESS-A-10-53 [TunnelB] suspended Mar 30 15:30:23:896 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:25:698 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:26:406 [140603203516160] 5 hub: TSESS-A-6-17 [Secondary] suspended Mar 30 15:30:26:438 [140603195123456] 5 hub: TSESS-A-3-17 [SecondaryHA] suspended Mar 30 15:30:26:517 [140603161552640] 5 hub: TSESS-P-4-19 [SecondaryHA] suspended Mar 30 15:30:26:962 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:30:26:990 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:30:27:104 [140603153159936] 5 hub: TSESS-P-8-14 [Secondary] suspended Mar 30 15:30:27:501 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:27:607 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:30:27:820 [140605547460672] 4 hub: medium timeout Mar 30 15:30:27:820 [140605547460672] 3 hub: next_check [1585596696] now [1585596627] Mar 30 15:30:27:939 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:30:28:702 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:30:28:907 [140603169945344] 5 hub: TSESS-P-7-17 [TunnelB] suspended Mar 30 15:30:29:303 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:31:106 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:31:854 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:30:31:854 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:30:31:854 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33334 Mar 30 15:30:31:854 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33334' delegated to TSESS-A-5 [140603715176192] Mar 30 15:30:31:854 [140603715176192] 3 hub: TSESS-A-5-73 [TunnelB] restarted (for local) Mar 30 15:30:31:859 [140603715176192] 5 hub: TSESS-A-5-73 [TunnelB] local 100 'get_info' Mar 30 15:30:31:859 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:31:859 [140603715176192] 1 hub: TSESS-A-5-73 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:30:31:869 [140603715176192] 1 hub: TSESS-A-5-73 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:30:31:870 [140603715176192] 3 hub: TSESS-A-5-73 [TunnelB] is paired with [P-298-73] Mar 30 15:30:31:870 [140603715176192] 1 hub: TSESS-A-5-73 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:30:31:870 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33336 Mar 30 15:30:31:870 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33336' delegated to TSESS-A-10 [140603706783488] Mar 30 15:30:31:870 [140603706783488] 3 hub: TSESS-A-10-54 [TunnelB] restarted (for local) Mar 30 15:30:31:875 [140603715176192] 5 hub: TSESS-A-5-73 [TunnelB] local 100 '_close' Mar 30 15:30:31:875 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:31:875 [140603715176192] 1 hub: TSESS-A-5-73 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:30:31:875 [140603715176192] 3 hub: TSESS-A-5-73 [TunnelB] local session closed, suspending Mar 30 15:30:31:875 [140603706783488] 5 hub: TSESS-A-10-54 [TunnelB] local 100 '_status' Mar 30 15:30:31:875 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:31:875 [140603706783488] 1 hub: TSESS-A-10-54 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:30:31:885 [140603706783488] 1 hub: TSESS-A-10-54 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:30:31:886 [140603706783488] 3 hub: TSESS-A-10-54 [TunnelB] is paired with [P-300-54] Mar 30 15:30:31:886 [140603706783488] 1 hub: TSESS-A-10-54 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:30:31:891 [140603706783488] 5 hub: TSESS-A-10-54 [TunnelB] local 100 '_close' Mar 30 15:30:31:891 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:31:891 [140603706783488] 1 hub: TSESS-A-10-54 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:30:31:891 [140603706783488] 3 hub: TSESS-A-10-54 [TunnelB] local session closed, suspending Mar 30 15:30:32:075 [140603715176192] 5 hub: TSESS-A-5-73 [TunnelB] suspended Mar 30 15:30:32:091 [140603706783488] 5 hub: TSESS-A-10-54 [TunnelB] suspended Mar 30 15:30:33:809 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:34:192 [140604042360576] 5 hub: There are 0 objects waiting on 8 threads Mar 30 15:30:35:612 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:35:858 [140605547460672] 3 hub: check_session - session 0x20f9050 (127.0.0.1/48002) is a server Mar 30 15:30:35:859 [140605547460672] 2 hub: Profile - clayk 38 bytes (OK) Mar 30 15:30:36:406 [140603203516160] 5 hub: TSESS-A-6-17 [Secondary] suspended Mar 30 15:30:36:438 [140603195123456] 5 hub: TSESS-A-3-17 [SecondaryHA] suspended Mar 30 15:30:36:517 [140603161552640] 5 hub: TSESS-P-4-19 [SecondaryHA] suspended Mar 30 15:30:36:962 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:30:36:990 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:30:37:104 [140603153159936] 5 hub: TSESS-P-8-14 [Secondary] suspended Mar 30 15:30:37:251 [140603740354304] 5 hub: TSESS-P-8 not internal command - thread will handle. mtype=100; sfd:23 Mar 30 15:30:37:251 [140603740354304] 2 hub: CTRL Secondary request 'hubup' passed to TSESS-P-8 [140603153159936] Mar 30 15:30:37:251 [140603153159936] 3 hub: TSESS-P-8-15 [Secondary] restarted (for remote) Mar 30 15:30:37:251 [140603153159936] 2 hub: TSESS-P-8 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:40:37 Mar 30 15:30:37:251 [140603153159936] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:30:37:251 [140603153159936] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:30:37:251 [140603153159936] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:30:37:251 [140603153159936] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:30:37:251 [140603153159936] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:30:37:251 [140603153159936] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:30:37:252 [140603153159936] 4 hub: CONNECT: 0x7fe0a0000d90(37) 127.0.0.1/54048->127.0.0.1/48002 Mar 30 15:30:37:252 [140603153159936] 3 hub: TSESS-P-8-15 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:30:37:252 [140603153159936] 1 hub: TSESS-P-8-15 [Secondary] received 100 'hubup' (685 bytes) Mar 30 15:30:37:252 [140605547460672] 4 hub: medium timeout Mar 30 15:30:37:252 [140605547460672] 3 hub: next_check [1585596696] now [1585596637] Mar 30 15:30:37:252 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/54048 (s:388) Mar 30 15:30:37:252 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/54048] Mar 30 15:30:37:252 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:30:37:252 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:30:37:252 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/54048 Mar 30 15:30:37:252 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/54048 same proximity=0 status=0 secver=217 Mar 30 15:30:37:257 [140603153159936] 5 hub: TSESS-P-8-15 [Secondary] local 200 'OK' Mar 30 15:30:37:257 [140603153159936] 1 hub: TSESS-P-8-15 [Secondary] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:30:37:267 [140603153159936] 1 hub: TSESS-P-8-15 [Secondary] ssl 100 '_close' (298 bytes) Mar 30 15:30:37:267 [140603153159936] 3 hub: TSESS-P-8-15 [Secondary] is paired with [A-2121-15] Mar 30 15:30:37:267 [140603153159936] 1 hub: TSESS-P-8-15 [Secondary] received 100 '_close' (298 bytes) Mar 30 15:30:37:273 [140603153159936] 1 hub: TSESS-P-8-15 [Secondary] remote session closed Mar 30 15:30:37:273 [140603153159936] 3 hub: TSESS-P-8-15 [Secondary] exiting msg loop (remote close) Mar 30 15:30:37:273 [140603153159936] 5 hub: TSESS-P-8-15 [Secondary] suspended Mar 30 15:30:37:414 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:37:607 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:30:37:939 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:30:38:033 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:30:38:033 [140603731961600] 2 hub: CTRL SecondaryHA request 'hubup' passed to TSESS-P-4 [140603161552640] Mar 30 15:30:38:033 [140603161552640] 3 hub: TSESS-P-4-20 [SecondaryHA] restarted (for remote) Mar 30 15:30:38:033 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:40:37 Mar 30 15:30:38:033 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:30:38:033 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:30:38:033 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:30:38:033 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:30:38:033 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:30:38:033 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:30:38:034 [140603161552640] 4 hub: CONNECT: 0x7fe09c000e60(37) 127.0.0.1/54050->127.0.0.1/48002 Mar 30 15:30:38:034 [140603161552640] 3 hub: TSESS-P-4-20 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:30:38:034 [140603161552640] 1 hub: TSESS-P-4-20 [SecondaryHA] received 100 'hubup' (689 bytes) Mar 30 15:30:38:034 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/54050 (s:392) Mar 30 15:30:38:034 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/54050] Mar 30 15:30:38:034 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:30:38:034 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:30:38:034 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/54050 Mar 30 15:30:38:034 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/54050 same proximity=0 status=0 secver=217 Mar 30 15:30:38:039 [140603161552640] 5 hub: TSESS-P-4-20 [SecondaryHA] local 200 'OK' Mar 30 15:30:38:039 [140603161552640] 1 hub: TSESS-P-4-20 [SecondaryHA] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:30:38:050 [140603161552640] 1 hub: TSESS-P-4-20 [SecondaryHA] ssl 100 '_close' (302 bytes) Mar 30 15:30:38:050 [140603161552640] 3 hub: TSESS-P-4-20 [SecondaryHA] is paired with [A-1948-20] Mar 30 15:30:38:050 [140603161552640] 1 hub: TSESS-P-4-20 [SecondaryHA] received 100 '_close' (298 bytes) Mar 30 15:30:38:055 [140603161552640] 1 hub: TSESS-P-4-20 [SecondaryHA] remote session closed Mar 30 15:30:38:055 [140603161552640] 3 hub: TSESS-P-4-20 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:30:38:055 [140603161552640] 5 hub: TSESS-P-4-20 [SecondaryHA] suspended Mar 30 15:30:38:702 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:30:38:908 [140603169945344] 5 hub: TSESS-P-7-17 [TunnelB] suspended Mar 30 15:30:38:971 [140603723568896] 5 hub: TSESS-P-7 not internal command - thread will handle. mtype=100; sfd:22 Mar 30 15:30:38:971 [140603723568896] 2 hub: CTRL TunnelB request 'hubup' passed to TSESS-P-7 [140603169945344] Mar 30 15:30:38:971 [140603169945344] 3 hub: TSESS-P-7-18 [TunnelB] restarted (for remote) Mar 30 15:30:38:971 [140603169945344] 2 hub: TSESS-P-7 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:40:38 Mar 30 15:30:38:971 [140603169945344] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:30:38:971 [140603169945344] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:30:38:971 [140603169945344] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:30:38:971 [140603169945344] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:30:38:971 [140603169945344] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:30:38:971 [140603169945344] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:30:38:971 [140603169945344] 4 hub: CONNECT: 0x7fe0a8000d20(37) 127.0.0.1/54052->127.0.0.1/48002 Mar 30 15:30:38:971 [140603169945344] 3 hub: TSESS-P-7-18 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:30:38:971 [140603169945344] 1 hub: TSESS-P-7-18 [TunnelB] received 100 'hubup' (673 bytes) Mar 30 15:30:38:971 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/54052 (s:378) Mar 30 15:30:38:971 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/54052] Mar 30 15:30:38:971 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:30:38:971 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:30:38:971 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/54052 Mar 30 15:30:38:971 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/54052 same proximity=0 status=0 secver=217 Mar 30 15:30:38:976 [140603169945344] 5 hub: TSESS-P-7-18 [TunnelB] local 200 'OK' Mar 30 15:30:38:976 [140603169945344] 1 hub: TSESS-P-7-18 [TunnelB] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:30:38:987 [140603169945344] 1 hub: TSESS-P-7-18 [TunnelB] ssl 100 '_close' (284 bytes) Mar 30 15:30:38:987 [140603169945344] 3 hub: TSESS-P-7-18 [TunnelB] is paired with [A-299-18] Mar 30 15:30:38:987 [140603169945344] 1 hub: TSESS-P-7-18 [TunnelB] received 100 '_close' (296 bytes) Mar 30 15:30:38:992 [140603169945344] 1 hub: TSESS-P-7-18 [TunnelB] remote session closed Mar 30 15:30:38:992 [140603169945344] 3 hub: TSESS-P-7-18 [TunnelB] exiting msg loop (remote close) Mar 30 15:30:38:992 [140603169945344] 5 hub: TSESS-P-7-18 [TunnelB] suspended Mar 30 15:30:39:216 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:41:019 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:41:889 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:30:41:889 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:30:41:889 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33352 Mar 30 15:30:41:889 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33352' delegated to TSESS-A-5 [140603715176192] Mar 30 15:30:41:889 [140603715176192] 3 hub: TSESS-A-5-74 [TunnelB] restarted (for local) Mar 30 15:30:41:895 [140603715176192] 5 hub: TSESS-A-5-74 [TunnelB] local 100 'get_info' Mar 30 15:30:41:895 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:41:895 [140603715176192] 1 hub: TSESS-A-5-74 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:30:41:905 [140603715176192] 1 hub: TSESS-A-5-74 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:30:41:905 [140603715176192] 3 hub: TSESS-A-5-74 [TunnelB] is paired with [P-298-74] Mar 30 15:30:41:905 [140603715176192] 1 hub: TSESS-A-5-74 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:30:41:905 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33354 Mar 30 15:30:41:905 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33354' delegated to TSESS-A-10 [140603706783488] Mar 30 15:30:41:905 [140603706783488] 3 hub: TSESS-A-10-55 [TunnelB] restarted (for local) Mar 30 15:30:41:910 [140603715176192] 5 hub: TSESS-A-5-74 [TunnelB] local 100 '_close' Mar 30 15:30:41:910 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:41:910 [140603715176192] 1 hub: TSESS-A-5-74 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:30:41:910 [140603715176192] 3 hub: TSESS-A-5-74 [TunnelB] local session closed, suspending Mar 30 15:30:41:910 [140603706783488] 5 hub: TSESS-A-10-55 [TunnelB] local 100 '_status' Mar 30 15:30:41:910 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:41:910 [140603706783488] 1 hub: TSESS-A-10-55 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:30:41:921 [140603706783488] 1 hub: TSESS-A-10-55 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:30:41:921 [140603706783488] 3 hub: TSESS-A-10-55 [TunnelB] is paired with [P-300-55] Mar 30 15:30:41:921 [140603706783488] 1 hub: TSESS-A-10-55 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:30:41:926 [140603706783488] 5 hub: TSESS-A-10-55 [TunnelB] local 100 '_close' Mar 30 15:30:41:926 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:41:926 [140603706783488] 1 hub: TSESS-A-10-55 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:30:41:926 [140603706783488] 3 hub: TSESS-A-10-55 [TunnelB] local session closed, suspending Mar 30 15:30:42:111 [140603715176192] 5 hub: TSESS-A-5-74 [TunnelB] suspended Mar 30 15:30:42:126 [140603706783488] 5 hub: TSESS-A-10-55 [TunnelB] suspended Mar 30 15:30:43:722 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:45:525 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:46:271 [140605283890944] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:30:46:271 [140605283890944] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:30:46:271 [140605283890944] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:30:46:271 [140605283890944] 1 hub: PERF: hubup_check_interval: 0 sec hubhash_size: 19 Mar 30 15:30:46:271 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-12 100.64.1.22/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:271 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:271 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:271 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:271 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:271 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:271 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:271 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:30:46:271 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:271 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:271 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:271 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:271 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:271 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:271 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:271 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:271 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:30:46:271 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:30:46:271 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:30:46:272 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(37) 127.0.0.1/47728->127.0.0.1/48007 Mar 30 15:30:46:272 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/47728 Mar 30 15:30:46:272 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/47728' delegated to TSESS-A-6 [140603203516160] Mar 30 15:30:46:272 [140603203516160] 3 hub: TSESS-A-6-18 [Secondary] restarted (for local) Mar 30 15:30:46:277 [140603203516160] 5 hub: TSESS-A-6-18 [Secondary] local 100 'hubup' Mar 30 15:30:46:277 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:46:277 [140603203516160] 1 hub: TSESS-A-6-18 [Secondary] sent 100 'hubup' (687 bytes, 0 ms) Mar 30 15:30:46:287 [140603203516160] 1 hub: TSESS-A-6-18 [Secondary] ssl 200 '-' (455 bytes) Mar 30 15:30:46:287 [140603203516160] 3 hub: TSESS-A-6-18 [Secondary] is paired with [P-2120-18] Mar 30 15:30:46:287 [140603203516160] 1 hub: TSESS-A-6-18 [Secondary] received 200 '-' (463 bytes) Mar 30 15:30:46:287 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:30:46:287 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:30:46:287 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:30:46:287 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:30:46:287 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary same proximity=0 status=0 secver=217 Mar 30 15:30:46:287 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-12 100.64.1.22/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:287 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:287 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:287 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:287 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:287 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:287 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:287 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:30:46:287 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:287 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:287 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:287 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:287 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:287 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:287 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:287 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:287 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:30:46:287 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:30:46:287 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:30:46:288 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(37) 127.0.0.1/33360->127.0.0.1/48009 Mar 30 15:30:46:288 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33360 Mar 30 15:30:46:288 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33360' delegated to TSESS-A-5 [140603715176192] Mar 30 15:30:46:288 [140603715176192] 3 hub: TSESS-A-5-75 [TunnelB] restarted (for local) Mar 30 15:30:46:292 [140603203516160] 5 hub: TSESS-A-6-18 [Secondary] local 100 '_close' Mar 30 15:30:46:292 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:46:292 [140603203516160] 1 hub: TSESS-A-6-18 [Secondary] sent 100 '_close' (294 bytes, 0 ms) Mar 30 15:30:46:292 [140603203516160] 3 hub: TSESS-A-6-18 [Secondary] local session closed, suspending Mar 30 15:30:46:293 [140603715176192] 5 hub: TSESS-A-5-75 [TunnelB] local 100 'hubup' Mar 30 15:30:46:293 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:46:293 [140603715176192] 1 hub: TSESS-A-5-75 [TunnelB] sent 100 'hubup' (685 bytes, 0 ms) Mar 30 15:30:46:303 [140603715176192] 1 hub: TSESS-A-5-75 [TunnelB] ssl 200 '-' (443 bytes) Mar 30 15:30:46:303 [140603715176192] 3 hub: TSESS-A-5-75 [TunnelB] is paired with [P-298-75] Mar 30 15:30:46:303 [140603715176192] 1 hub: TSESS-A-5-75 [TunnelB] received 200 '-' (451 bytes) Mar 30 15:30:46:303 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:30:46:303 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:30:46:303 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:30:46:303 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:30:46:303 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB same proximity=0 status=0 secver=217 Mar 30 15:30:46:303 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-12 100.64.1.22/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:303 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:303 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:303 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:303 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:303 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:303 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:303 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:30:46:303 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:303 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:303 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:303 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:303 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:303 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:303 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:303 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:30:46:303 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:30:46:303 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:30:46:303 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:30:46:303 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(37) 127.0.0.1/40780->127.0.0.1/48008 Mar 30 15:30:46:304 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/40780 Mar 30 15:30:46:304 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/40780' delegated to TSESS-A-3 [140603195123456] Mar 30 15:30:46:304 [140603195123456] 3 hub: TSESS-A-3-18 [SecondaryHA] restarted (for local) Mar 30 15:30:46:308 [140603715176192] 5 hub: TSESS-A-5-75 [TunnelB] local 100 '_close' Mar 30 15:30:46:308 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:46:308 [140603715176192] 1 hub: TSESS-A-5-75 [TunnelB] sent 100 '_close' (292 bytes, 0 ms) Mar 30 15:30:46:308 [140603715176192] 3 hub: TSESS-A-5-75 [TunnelB] local session closed, suspending Mar 30 15:30:46:309 [140603195123456] 5 hub: TSESS-A-3-18 [SecondaryHA] local 100 'hubup' Mar 30 15:30:46:309 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:46:309 [140603195123456] 1 hub: TSESS-A-3-18 [SecondaryHA] sent 100 'hubup' (689 bytes, 0 ms) Mar 30 15:30:46:319 [140603195123456] 1 hub: TSESS-A-3-18 [SecondaryHA] ssl 200 '-' (459 bytes) Mar 30 15:30:46:319 [140603195123456] 3 hub: TSESS-A-3-18 [SecondaryHA] is paired with [P-1947-18] Mar 30 15:30:46:319 [140603195123456] 1 hub: TSESS-A-3-18 [SecondaryHA] received 200 '-' (467 bytes) Mar 30 15:30:46:319 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:30:46:319 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:30:46:319 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:30:46:319 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:30:46:319 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA same proximity=0 status=0 secver=217 Mar 30 15:30:46:319 [140605283890944] 5 hub: mark_hubup_list_updated Mar 30 15:30:46:319 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-12 100.64.1.22/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:46:319 [140605283890944] 5 hub: hublist changed: /DOMAIN/Primary 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:46:319 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:46:319 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:46:319 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:46:319 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:46:319 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:46:319 [140605283890944] 5 hub: hublist changed: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:30:46:319 [140605283890944] 5 hub: hublist changed: /DOMAIN/PrimaryHA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:46:319 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:46:319 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:46:319 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:46:319 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:46:319 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:46:319 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:46:319 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:30:46:319 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:30:46:319 [140605283890944] 5 hub: hublist changed: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:30:46:319 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelBHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:30:46:319 [140605283890944] 5 hub: mark_hubup_list_updated end Mar 30 15:30:46:319 [140605283890944] 1 hub: PERF: hubup_check_for_updates: 0 sec hubhash_size: 19 Mar 30 15:30:46:319 [140605283890944] 1 hub: PERF: hubi_run_command: 0 sec hubhash_size: 19 Mar 30 15:30:46:319 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:30:46:320 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:30:46:320 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:30:46:321 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:30:46:321 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:30:46:321 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:30:46:321 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:30:46:322 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:30:46:322 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:30:46:322 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:30:46:323 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:30:46:323 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:30:46:323 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:30:46:324 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:30:46:324 [140603195123456] 5 hub: TSESS-A-3-18 [SecondaryHA] local 100 '_close' Mar 30 15:30:46:324 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:46:324 [140603195123456] 1 hub: TSESS-A-3-18 [SecondaryHA] sent 100 '_close' (296 bytes, 0 ms) Mar 30 15:30:46:324 [140603195123456] 3 hub: TSESS-A-3-18 [SecondaryHA] local session closed, suspending Mar 30 15:30:46:325 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:30:46:325 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:30:46:326 [140605283890944] 1 hub: PERF: hubi_check_security: 0 sec hubhash_size: 19 Mar 30 15:30:46:493 [140603203516160] 5 hub: TSESS-A-6-18 [Secondary] suspended Mar 30 15:30:46:509 [140603715176192] 5 hub: TSESS-A-5-75 [TunnelB] suspended Mar 30 15:30:46:525 [140603195123456] 5 hub: TSESS-A-3-18 [SecondaryHA] suspended Mar 30 15:30:46:962 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:30:46:990 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:30:47:031 [140605547460672] 4 hub: medium timeout Mar 30 15:30:47:031 [140605547460672] 3 hub: next_check [1585596696] now [1585596647] Mar 30 15:30:47:031 [140605547460672] 2 hub: verify_login from ctrl 127.0.0.1/54090 Mar 30 15:30:47:031 [140605547460672] 1 hub: verify - [target] prid=controller cmd=probe_list ip=10.x.x.x perm=1 Mar 30 15:30:47:031 [140605547460672] 1 hub: verify - [source] id=ppm ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:47:273 [140603153159936] 5 hub: TSESS-P-8-15 [Secondary] suspended Mar 30 15:30:47:327 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:47:607 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:30:47:939 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:30:48:056 [140603161552640] 5 hub: TSESS-P-4-20 [SecondaryHA] suspended Mar 30 15:30:48:703 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:30:48:992 [140603169945344] 5 hub: TSESS-P-7-18 [TunnelB] suspended Mar 30 15:30:49:130 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:51:833 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:51:932 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:30:51:932 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:30:51:933 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33386 Mar 30 15:30:51:933 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33386' delegated to TSESS-A-5 [140603715176192] Mar 30 15:30:51:933 [140603715176192] 3 hub: TSESS-A-5-76 [TunnelB] restarted (for local) Mar 30 15:30:51:938 [140603715176192] 5 hub: TSESS-A-5-76 [TunnelB] local 100 'get_info' Mar 30 15:30:51:938 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:51:938 [140603715176192] 1 hub: TSESS-A-5-76 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:30:51:948 [140603715176192] 1 hub: TSESS-A-5-76 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:30:51:948 [140603715176192] 3 hub: TSESS-A-5-76 [TunnelB] is paired with [P-298-76] Mar 30 15:30:51:948 [140603715176192] 1 hub: TSESS-A-5-76 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:30:51:948 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33388 Mar 30 15:30:51:948 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33388' delegated to TSESS-A-10 [140603706783488] Mar 30 15:30:51:949 [140603706783488] 3 hub: TSESS-A-10-56 [TunnelB] restarted (for local) Mar 30 15:30:51:953 [140603715176192] 5 hub: TSESS-A-5-76 [TunnelB] local 100 '_close' Mar 30 15:30:51:953 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:51:953 [140603715176192] 1 hub: TSESS-A-5-76 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:30:51:953 [140603715176192] 3 hub: TSESS-A-5-76 [TunnelB] local session closed, suspending Mar 30 15:30:51:954 [140603706783488] 5 hub: TSESS-A-10-56 [TunnelB] local 100 '_status' Mar 30 15:30:51:954 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:51:954 [140603706783488] 1 hub: TSESS-A-10-56 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:30:51:964 [140603706783488] 1 hub: TSESS-A-10-56 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:30:51:964 [140603706783488] 3 hub: TSESS-A-10-56 [TunnelB] is paired with [P-300-56] Mar 30 15:30:51:964 [140603706783488] 1 hub: TSESS-A-10-56 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:30:51:969 [140603706783488] 5 hub: TSESS-A-10-56 [TunnelB] local 100 '_close' Mar 30 15:30:51:969 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:30:51:969 [140603706783488] 1 hub: TSESS-A-10-56 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:30:51:969 [140603706783488] 3 hub: TSESS-A-10-56 [TunnelB] local session closed, suspending Mar 30 15:30:52:154 [140603715176192] 5 hub: TSESS-A-5-76 [TunnelB] suspended Mar 30 15:30:52:170 [140603706783488] 5 hub: TSESS-A-10-56 [TunnelB] suspended Mar 30 15:30:53:636 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:55:438 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:56:493 [140603203516160] 5 hub: TSESS-A-6-18 [Secondary] suspended Mar 30 15:30:56:525 [140603195123456] 5 hub: TSESS-A-3-18 [SecondaryHA] suspended Mar 30 15:30:56:962 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:30:56:991 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:30:57:241 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:30:57:273 [140603153159936] 5 hub: TSESS-P-8-15 [Secondary] suspended Mar 30 15:30:57:607 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:30:57:938 [140605547460672] 4 hub: medium timeout Mar 30 15:30:57:938 [140605547460672] 3 hub: next_check [1585596696] now [1585596657] Mar 30 15:30:57:939 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:30:58:056 [140603161552640] 5 hub: TSESS-P-4-20 [SecondaryHA] suspended Mar 30 15:30:58:703 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:30:58:992 [140603169945344] 5 hub: TSESS-P-7-18 [TunnelB] suspended Mar 30 15:30:59:043 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:31:01:746 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:31:01:975 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:31:01:975 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:31:01:976 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33394 Mar 30 15:31:01:976 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33394' delegated to TSESS-A-5 [140603715176192] Mar 30 15:31:01:976 [140603715176192] 3 hub: TSESS-A-5-77 [TunnelB] restarted (for local) Mar 30 15:31:01:981 [140603715176192] 5 hub: TSESS-A-5-77 [TunnelB] local 100 'get_info' Mar 30 15:31:01:981 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:31:01:981 [140603715176192] 1 hub: TSESS-A-5-77 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:31:01:991 [140603715176192] 1 hub: TSESS-A-5-77 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:31:01:991 [140603715176192] 3 hub: TSESS-A-5-77 [TunnelB] is paired with [P-298-77] Mar 30 15:31:01:991 [140603715176192] 1 hub: TSESS-A-5-77 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:31:01:992 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33396 Mar 30 15:31:01:992 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33396' delegated to TSESS-A-10 [140603706783488] Mar 30 15:31:01:992 [140603706783488] 3 hub: TSESS-A-10-57 [TunnelB] restarted (for local) Mar 30 15:31:01:996 [140603715176192] 5 hub: TSESS-A-5-77 [TunnelB] local 100 '_close' Mar 30 15:31:01:997 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:31:01:997 [140603715176192] 1 hub: TSESS-A-5-77 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:31:01:997 [140603715176192] 3 hub: TSESS-A-5-77 [TunnelB] local session closed, suspending Mar 30 15:31:01:997 [140603706783488] 5 hub: TSESS-A-10-57 [TunnelB] local 100 '_status' Mar 30 15:31:01:997 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:31:01:997 [140603706783488] 1 hub: TSESS-A-10-57 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:31:02:007 [140603706783488] 1 hub: TSESS-A-10-57 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:31:02:007 [140603706783488] 3 hub: TSESS-A-10-57 [TunnelB] is paired with [P-300-57] Mar 30 15:31:02:007 [140603706783488] 1 hub: TSESS-A-10-57 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:31:02:012 [140603706783488] 5 hub: TSESS-A-10-57 [TunnelB] local 100 '_close' Mar 30 15:31:02:012 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:31:02:012 [140603706783488] 1 hub: TSESS-A-10-57 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:31:02:012 [140603706783488] 3 hub: TSESS-A-10-57 [TunnelB] local session closed, suspending Mar 30 15:31:02:197 [140603715176192] 5 hub: TSESS-A-5-77 [TunnelB] suspended Mar 30 15:31:02:213 [140603706783488] 5 hub: TSESS-A-10-57 [TunnelB] suspended Mar 30 15:31:03:549 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:31:05:351 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:31:05:980 [140605547460672] 3 hub: check_session - session 0x20f9050 (127.0.0.1/48002) is a server Mar 30 15:31:05:980 [140605547460672] 2 hub: Profile - clayk 38 bytes (OK) Mar 30 15:31:06:493 [140603203516160] 5 hub: TSESS-A-6-18 [Secondary] suspended Mar 30 15:31:06:525 [140603195123456] 5 hub: TSESS-A-3-18 [SecondaryHA] suspended Mar 30 15:31:06:963 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:31:06:991 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:31:07:154 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:31:07:164 [140603731961600] 5 hub: TSESS-P-4 not internal command - thread will handle. mtype=100; sfd:19 Mar 30 15:31:07:164 [140603731961600] 2 hub: CTRL SecondaryHA request 'hubup' passed to TSESS-P-4 [140603161552640] Mar 30 15:31:07:164 [140603161552640] 3 hub: TSESS-P-4-21 [SecondaryHA] restarted (for remote) Mar 30 15:31:07:164 [140603161552640] 2 hub: TSESS-P-4 SID issued to id=hub hub=SecondaryHA ip=10.x.x.x type=9 expire=2020-03-30 15:41:07 Mar 30 15:31:07:164 [140603161552640] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:31:07:164 [140603161552640] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:31:07:164 [140603161552640] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:31:07:164 [140603161552640] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:31:07:164 [140603161552640] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:31:07:164 [140603161552640] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:31:07:164 [140603161552640] 4 hub: CONNECT: 0x7fe09c000e60(37) 127.0.0.1/54110->127.0.0.1/48002 Mar 30 15:31:07:164 [140603161552640] 3 hub: TSESS-P-4-21 [SecondaryHA] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:31:07:164 [140605547460672] 4 hub: medium timeout Mar 30 15:31:07:164 [140603161552640] 1 hub: TSESS-P-4-21 [SecondaryHA] received 100 'hubup' (689 bytes) Mar 30 15:31:07:164 [140605547460672] 3 hub: next_check [1585596696] now [1585596667] Mar 30 15:31:07:164 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/54110 (s:392) Mar 30 15:31:07:164 [140605547460672] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [127.0.0.1/54110] Mar 30 15:31:07:164 [140605547460672] 4 hub: hubup source: SecondaryHA Mar 30 15:31:07:164 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:31:07:164 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from 127.0.0.1/54110 Mar 30 15:31:07:164 [140605547460672] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=127.0.0.1/54110 same proximity=0 status=0 secver=217 Mar 30 15:31:07:169 [140603161552640] 5 hub: TSESS-P-4-21 [SecondaryHA] local 200 'OK' Mar 30 15:31:07:169 [140603161552640] 1 hub: TSESS-P-4-21 [SecondaryHA] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:31:07:181 [140603161552640] 1 hub: TSESS-P-4-21 [SecondaryHA] ssl 100 '_close' (302 bytes) Mar 30 15:31:07:181 [140603161552640] 3 hub: TSESS-P-4-21 [SecondaryHA] is paired with [A-1948-21] Mar 30 15:31:07:181 [140603161552640] 1 hub: TSESS-P-4-21 [SecondaryHA] received 100 '_close' (298 bytes) Mar 30 15:31:07:186 [140603161552640] 1 hub: TSESS-P-4-21 [SecondaryHA] remote session closed Mar 30 15:31:07:186 [140603161552640] 3 hub: TSESS-P-4-21 [SecondaryHA] exiting msg loop (remote close) Mar 30 15:31:07:186 [140603161552640] 5 hub: TSESS-P-4-21 [SecondaryHA] suspended Mar 30 15:31:07:273 [140603153159936] 5 hub: TSESS-P-8-15 [Secondary] suspended Mar 30 15:31:07:383 [140603740354304] 5 hub: TSESS-P-8 not internal command - thread will handle. mtype=100; sfd:23 Mar 30 15:31:07:383 [140603740354304] 2 hub: CTRL Secondary request 'hubup' passed to TSESS-P-8 [140603153159936] Mar 30 15:31:07:383 [140603153159936] 3 hub: TSESS-P-8-16 [Secondary] restarted (for remote) Mar 30 15:31:07:383 [140603153159936] 2 hub: TSESS-P-8 SID issued to id=hub hub=Secondary ip=10.x.x.x type=9 expire=2020-03-30 15:41:07 Mar 30 15:31:07:383 [140603153159936] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:31:07:384 [140603153159936] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:31:07:384 [140603153159936] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:31:07:384 [140603153159936] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:31:07:384 [140603153159936] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:31:07:384 [140603153159936] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:31:07:384 [140603153159936] 4 hub: CONNECT: 0x7fe0a0000d90(37) 127.0.0.1/54112->127.0.0.1/48002 Mar 30 15:31:07:384 [140603153159936] 3 hub: TSESS-P-8-16 [Secondary] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:31:07:384 [140603153159936] 1 hub: TSESS-P-8-16 [Secondary] received 100 'hubup' (685 bytes) Mar 30 15:31:07:384 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/54112 (s:388) Mar 30 15:31:07:384 [140605547460672] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [127.0.0.1/54112] Mar 30 15:31:07:384 [140605547460672] 4 hub: hubup source: Secondary Mar 30 15:31:07:384 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:31:07:384 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from 127.0.0.1/54112 Mar 30 15:31:07:384 [140605547460672] 2 hub: hubinfo_update - DOMAIN/Secondary from=127.0.0.1/54112 same proximity=0 status=0 secver=217 Mar 30 15:31:07:389 [140603153159936] 5 hub: TSESS-P-8-16 [Secondary] local 200 'OK' Mar 30 15:31:07:389 [140603153159936] 1 hub: TSESS-P-8-16 [Secondary] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:31:07:399 [140603153159936] 1 hub: TSESS-P-8-16 [Secondary] ssl 100 '_close' (298 bytes) Mar 30 15:31:07:399 [140603153159936] 3 hub: TSESS-P-8-16 [Secondary] is paired with [A-2121-16] Mar 30 15:31:07:399 [140603153159936] 1 hub: TSESS-P-8-16 [Secondary] received 100 '_close' (298 bytes) Mar 30 15:31:07:404 [140603153159936] 1 hub: TSESS-P-8-16 [Secondary] remote session closed Mar 30 15:31:07:404 [140603153159936] 3 hub: TSESS-P-8-16 [Secondary] exiting msg loop (remote close) Mar 30 15:31:07:404 [140603153159936] 5 hub: TSESS-P-8-16 [Secondary] suspended Mar 30 15:31:07:608 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:31:07:939 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:31:08:053 [140603723568896] 5 hub: TSESS-P-7 not internal command - thread will handle. mtype=100; sfd:22 Mar 30 15:31:08:053 [140603723568896] 2 hub: CTRL TunnelB request 'hubup' passed to TSESS-P-7 [140603169945344] Mar 30 15:31:08:053 [140603169945344] 3 hub: TSESS-P-7-19 [TunnelB] restarted (for remote) Mar 30 15:31:08:054 [140603169945344] 2 hub: TSESS-P-7 SID issued to id=hub hub=TunnelB ip=10.x.x.x type=9 expire=2020-03-30 15:41:08 Mar 30 15:31:08:054 [140603169945344] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:31:08:054 [140603169945344] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:31:08:054 [140603169945344] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:31:08:054 [140603169945344] 5 hub: inametoip - lookup name /DOMAIN/TunnelBHA/ROBOT_NAME/hub Mar 30 15:31:08:054 [140603169945344] 2 hub: r_nns(+):/DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 Mar 30 15:31:08:054 [140603169945344] 2 hub: nametoip - /DOMAIN/TunnelBHA/ROBOT_NAME/hub 127.0.0.1/48002 - 0 ms (nns) Mar 30 15:31:08:054 [140603169945344] 4 hub: CONNECT: 0x7fe0a8000d20(37) 127.0.0.1/54114->127.0.0.1/48002 Mar 30 15:31:08:054 [140603169945344] 3 hub: TSESS-P-7-19 [TunnelB] locally connected to /DOMAIN/TunnelBHA/ROBOT_NAME/hub (127.0.0.1/48002) Mar 30 15:31:08:054 [140603169945344] 1 hub: TSESS-P-7-19 [TunnelB] received 100 'hubup' (673 bytes) Mar 30 15:31:08:054 [140605547460672] 5 hub: hubup_tcp - from 127.0.0.1/54114 (s:378) Mar 30 15:31:08:054 [140605547460672] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [127.0.0.1/54114] Mar 30 15:31:08:054 [140605547460672] 4 hub: hubup source: TunnelB Mar 30 15:31:08:054 [140605547460672] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:31:08:054 [140605547460672] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from 127.0.0.1/54114 Mar 30 15:31:08:054 [140605547460672] 2 hub: hubinfo_update - DOMAIN/TunnelB from=127.0.0.1/54114 same proximity=0 status=0 secver=217 Mar 30 15:31:08:059 [140603169945344] 5 hub: TSESS-P-7-19 [TunnelB] local 200 'OK' Mar 30 15:31:08:059 [140603169945344] 1 hub: TSESS-P-7-19 [TunnelB] sent 200 'OK' (457 bytes, 0 ms) Mar 30 15:31:08:069 [140603169945344] 1 hub: TSESS-P-7-19 [TunnelB] ssl 100 '_close' (284 bytes) Mar 30 15:31:08:069 [140603169945344] 3 hub: TSESS-P-7-19 [TunnelB] is paired with [A-299-19] Mar 30 15:31:08:069 [140603169945344] 1 hub: TSESS-P-7-19 [TunnelB] received 100 '_close' (296 bytes) Mar 30 15:31:08:075 [140603169945344] 1 hub: TSESS-P-7-19 [TunnelB] remote session closed Mar 30 15:31:08:075 [140603169945344] 3 hub: TSESS-P-7-19 [TunnelB] exiting msg loop (remote close) Mar 30 15:31:08:075 [140603169945344] 5 hub: TSESS-P-7-19 [TunnelB] suspended Mar 30 15:31:08:703 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:31:09:857 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:31:11:016 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:31:11:016 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:31:11:016 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33410 Mar 30 15:31:11:016 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33410' delegated to TSESS-A-5 [140603715176192] Mar 30 15:31:11:016 [140603715176192] 3 hub: TSESS-A-5-78 [TunnelB] restarted (for local) Mar 30 15:31:11:021 [140603715176192] 5 hub: TSESS-A-5-78 [TunnelB] local 100 'get_info' Mar 30 15:31:11:021 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:31:11:021 [140603715176192] 1 hub: TSESS-A-5-78 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:31:11:032 [140603715176192] 1 hub: TSESS-A-5-78 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:31:11:032 [140603715176192] 3 hub: TSESS-A-5-78 [TunnelB] is paired with [P-298-78] Mar 30 15:31:11:032 [140603715176192] 1 hub: TSESS-A-5-78 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:31:11:032 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33412 Mar 30 15:31:11:032 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33412' delegated to TSESS-A-10 [140603706783488] Mar 30 15:31:11:032 [140603706783488] 3 hub: TSESS-A-10-58 [TunnelB] restarted (for local) Mar 30 15:31:11:037 [140603715176192] 5 hub: TSESS-A-5-78 [TunnelB] local 100 '_close' Mar 30 15:31:11:037 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:31:11:037 [140603715176192] 1 hub: TSESS-A-5-78 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:31:11:037 [140603715176192] 3 hub: TSESS-A-5-78 [TunnelB] local session closed, suspending Mar 30 15:31:11:037 [140603706783488] 5 hub: TSESS-A-10-58 [TunnelB] local 100 '_status' Mar 30 15:31:11:037 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:31:11:037 [140603706783488] 1 hub: TSESS-A-10-58 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:31:11:047 [140603706783488] 1 hub: TSESS-A-10-58 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:31:11:047 [140603706783488] 3 hub: TSESS-A-10-58 [TunnelB] is paired with [P-300-58] Mar 30 15:31:11:047 [140603706783488] 1 hub: TSESS-A-10-58 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:31:11:053 [140603706783488] 5 hub: TSESS-A-10-58 [TunnelB] local 100 '_close' Mar 30 15:31:11:053 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:31:11:053 [140603706783488] 1 hub: TSESS-A-10-58 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:31:11:053 [140603706783488] 3 hub: TSESS-A-10-58 [TunnelB] local session closed, suspending Mar 30 15:31:11:237 [140603715176192] 5 hub: TSESS-A-5-78 [TunnelB] suspended Mar 30 15:31:11:253 [140603706783488] 5 hub: TSESS-A-10-58 [TunnelB] suspended Mar 30 15:31:11:660 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:31:13:462 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:31:15:264 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:31:16:356 [140605283890944] 4 hub: nimGetIpList called with buffer size 40960 Mar 30 15:31:16:356 [140605283890944] 4 hub: nimGetIpList - skipping loopback address 127.0.0.1 Mar 30 15:31:16:356 [140605283890944] 4 hub: nimGetIpList - found |||10.x.x.x| Mar 30 15:31:16:356 [140605283890944] 1 hub: PERF: hubup_check_interval: 0 sec hubhash_size: 19 Mar 30 15:31:16:356 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-12 100.64.1.22/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:356 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:356 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:356 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:357 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:357 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:357 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:357 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:31:16:357 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:357 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:357 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:357 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:357 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:357 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:357 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:357 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:357 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:31:16:357 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:31:16:357 [140605283890944] 2 hub: hubupdatelist->Secondary[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:31:16:357 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(37) 127.0.0.1/47786->127.0.0.1/48007 Mar 30 15:31:16:357 [140603740354304] 2 hub: CTRL Secondary new NIMSESS requested by 127.0.0.1/47786 Mar 30 15:31:16:357 [140603740354304] 2 hub: CTRL Secondary session from '127.0.0.1/47786' delegated to TSESS-A-6 [140603203516160] Mar 30 15:31:16:357 [140603203516160] 3 hub: TSESS-A-6-19 [Secondary] restarted (for local) Mar 30 15:31:16:362 [140603203516160] 5 hub: TSESS-A-6-19 [Secondary] local 100 'hubup' Mar 30 15:31:16:362 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:31:16:362 [140603203516160] 1 hub: TSESS-A-6-19 [Secondary] sent 100 'hubup' (687 bytes, 0 ms) Mar 30 15:31:16:372 [140603203516160] 1 hub: TSESS-A-6-19 [Secondary] ssl 200 '-' (455 bytes) Mar 30 15:31:16:372 [140603203516160] 3 hub: TSESS-A-6-19 [Secondary] is paired with [P-2120-19] Mar 30 15:31:16:372 [140603203516160] 1 hub: TSESS-A-6-19 [Secondary] received 200 '-' (463 bytes) Mar 30 15:31:16:373 [140605283890944] 5 hub: hubinfo_update - name [Secondary] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [Secondary] source [(null)] from [/DOMAIN/Secondary] Mar 30 15:31:16:373 [140605283890944] 4 hub: hubup source: Secondary Mar 30 15:31:16:373 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/Secondary Mar 30 15:31:16:373 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/Secondary/ROBOT_NAME/hub, from /DOMAIN/Secondary Mar 30 15:31:16:373 [140605283890944] 2 hub: hubinfo_update - DOMAIN/Secondary from=/DOMAIN/Secondary same proximity=0 status=0 secver=217 Mar 30 15:31:16:373 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-12 100.64.1.22/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:373 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:373 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:373 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:373 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:373 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:373 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:373 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:31:16:373 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:373 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:373 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:373 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:373 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:373 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:373 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:373 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:373 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:31:16:373 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:31:16:373 [140605283890944] 2 hub: hubupdatelist->TunnelB[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:31:16:373 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(37) 127.0.0.1/33418->127.0.0.1/48009 Mar 30 15:31:16:373 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33418 Mar 30 15:31:16:373 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33418' delegated to TSESS-A-5 [140603715176192] Mar 30 15:31:16:373 [140603715176192] 3 hub: TSESS-A-5-79 [TunnelB] restarted (for local) Mar 30 15:31:16:378 [140603203516160] 5 hub: TSESS-A-6-19 [Secondary] local 100 '_close' Mar 30 15:31:16:378 [140603203516160] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:31:16:378 [140603203516160] 1 hub: TSESS-A-6-19 [Secondary] sent 100 '_close' (294 bytes, 0 ms) Mar 30 15:31:16:378 [140603203516160] 3 hub: TSESS-A-6-19 [Secondary] local session closed, suspending Mar 30 15:31:16:378 [140603715176192] 5 hub: TSESS-A-5-79 [TunnelB] local 100 'hubup' Mar 30 15:31:16:378 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:31:16:378 [140603715176192] 1 hub: TSESS-A-5-79 [TunnelB] sent 100 'hubup' (685 bytes, 0 ms) Mar 30 15:31:16:388 [140603715176192] 1 hub: TSESS-A-5-79 [TunnelB] ssl 200 '-' (443 bytes) Mar 30 15:31:16:388 [140603715176192] 3 hub: TSESS-A-5-79 [TunnelB] is paired with [P-298-79] Mar 30 15:31:16:388 [140603715176192] 1 hub: TSESS-A-5-79 [TunnelB] received 200 '-' (451 bytes) Mar 30 15:31:16:388 [140605283890944] 5 hub: hubinfo_update - name [TunnelB] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [TunnelB] source [(null)] from [/DOMAIN/TunnelB] Mar 30 15:31:16:388 [140605283890944] 4 hub: hubup source: TunnelB Mar 30 15:31:16:388 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/TunnelB Mar 30 15:31:16:388 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/TunnelB/ROBOT_NAME/hub, from /DOMAIN/TunnelB Mar 30 15:31:16:388 [140605283890944] 2 hub: hubinfo_update - DOMAIN/TunnelB from=/DOMAIN/TunnelB same proximity=0 status=0 secver=217 Mar 30 15:31:16:388 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-12 100.64.1.22/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:388 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Primary 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:388 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:388 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:388 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:388 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:388 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:388 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/Secondary 10.x.x.x/48002 () Mar 30 15:31:16:388 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/PrimaryHA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:388 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelA 10.x.x.x/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:388 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:388 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:388 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:388 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:389 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:389 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 (/DOMAIN/SecondaryHA) Mar 30 15:31:16:389 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelB 10.x.x.x/48002 () Mar 30 15:31:16:389 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/SecondaryHA 10.x.x.x/48002 () Mar 30 15:31:16:389 [140605283890944] 2 hub: hubupdatelist->SecondaryHA[-]: /DOMAIN/TunnelBHA 10.x.x.x/48002 () Mar 30 15:31:16:389 [140605283890944] 4 hub: CONNECT: 0x7fe130002350(37) 127.0.0.1/40838->127.0.0.1/48008 Mar 30 15:31:16:389 [140603731961600] 2 hub: CTRL SecondaryHA new NIMSESS requested by 127.0.0.1/40838 Mar 30 15:31:16:389 [140603731961600] 2 hub: CTRL SecondaryHA session from '127.0.0.1/40838' delegated to TSESS-A-3 [140603195123456] Mar 30 15:31:16:389 [140603195123456] 3 hub: TSESS-A-3-19 [SecondaryHA] restarted (for local) Mar 30 15:31:16:393 [140603715176192] 5 hub: TSESS-A-5-79 [TunnelB] local 100 '_close' Mar 30 15:31:16:393 [140603715176192] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:31:16:394 [140603715176192] 1 hub: TSESS-A-5-79 [TunnelB] sent 100 '_close' (292 bytes, 0 ms) Mar 30 15:31:16:394 [140603715176192] 3 hub: TSESS-A-5-79 [TunnelB] local session closed, suspending Mar 30 15:31:16:394 [140603195123456] 5 hub: TSESS-A-3-19 [SecondaryHA] local 100 'hubup' Mar 30 15:31:16:394 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:31:16:394 [140603195123456] 1 hub: TSESS-A-3-19 [SecondaryHA] sent 100 'hubup' (689 bytes, 0 ms) Mar 30 15:31:16:404 [140603195123456] 1 hub: TSESS-A-3-19 [SecondaryHA] ssl 200 '-' (459 bytes) Mar 30 15:31:16:404 [140603195123456] 3 hub: TSESS-A-3-19 [SecondaryHA] is paired with [P-1947-19] Mar 30 15:31:16:404 [140603195123456] 1 hub: TSESS-A-3-19 [SecondaryHA] received 200 '-' (467 bytes) Mar 30 15:31:16:404 [140605283890944] 5 hub: hubinfo_update - name [SecondaryHA] domain [DOMAIN] addr [(null)] ip [10.x.x.x] port [48002] robotname [ROBOT_NAME] version [9.20SHF6 [Build 9.20SHF6.255, Oct 28 2019]] status [0] ssl_mode [0] origin [SecondaryHA] source [(null)] from [/DOMAIN/SecondaryHA] Mar 30 15:31:16:404 [140605283890944] 4 hub: hubup source: SecondaryHA Mar 30 15:31:16:404 [140605283890944] 5 hub: hubinfo_update - found hubinfo for /DOMAIN/SecondaryHA Mar 30 15:31:16:404 [140605283890944] 5 hub: hubinfo_update - topology - origin was found in update - addr=/DOMAIN/SecondaryHA/ROBOT_NAME/hub, from /DOMAIN/SecondaryHA Mar 30 15:31:16:404 [140605283890944] 2 hub: hubinfo_update - DOMAIN/SecondaryHA from=/DOMAIN/SecondaryHA same proximity=0 status=0 secver=217 Mar 30 15:31:16:404 [140605283890944] 5 hub: mark_hubup_list_updated Mar 30 15:31:16:404 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-12 100.64.1.22/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:31:16:404 [140605283890944] 5 hub: hublist changed: /DOMAIN/Primary 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:31:16:404 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-4 100.64.1.14/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:31:16:405 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-9 100.64.1.19/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:31:16:405 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-5 100.64.1.15/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:31:16:405 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-1 100.64.1.11/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:31:16:405 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-6 100.64.1.16/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:31:16:405 [140605283890944] 5 hub: hublist changed: /DOMAIN/Secondary 10.x.x.x/48002 st=0 (p=) Mar 30 15:31:16:405 [140605283890944] 5 hub: hublist changed: /DOMAIN/PrimaryHA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:31:16:405 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelA 10.x.x.x/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:31:16:405 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-2 100.64.1.12/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:31:16:405 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-7 100.64.1.17/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:31:16:405 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-10 100.64.1.20/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:31:16:405 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-3 100.64.1.13/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:31:16:405 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-8 100.64.1.18/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:31:16:405 [140605283890944] 5 hub: hublist changed: /DOMAIN/A0F8-USDA-FMMI-11 100.64.1.21/48002 st=0 (p=/DOMAIN/SecondaryHA) Mar 30 15:31:16:405 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelB 10.x.x.x/48002 st=0 (p=) Mar 30 15:31:16:405 [140605283890944] 5 hub: hublist changed: /DOMAIN/SecondaryHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:31:16:405 [140605283890944] 5 hub: hublist changed: /DOMAIN/TunnelBHA 10.x.x.x/48002 st=0 (p=) Mar 30 15:31:16:405 [140605283890944] 5 hub: mark_hubup_list_updated end Mar 30 15:31:16:405 [140605283890944] 1 hub: PERF: hubup_check_for_updates: 0 sec hubhash_size: 19 Mar 30 15:31:16:405 [140605283890944] 1 hub: PERF: hubi_run_command: 0 sec hubhash_size: 19 Mar 30 15:31:16:405 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:31:16:405 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:31:16:405 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:31:16:406 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:31:16:406 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:31:16:407 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:31:16:407 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:31:16:407 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:31:16:407 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:31:16:408 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:31:16:408 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:31:16:408 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:31:16:409 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:31:16:409 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:31:16:409 [140603195123456] 5 hub: TSESS-A-3-19 [SecondaryHA] local 100 '_close' Mar 30 15:31:16:409 [140603195123456] 1 hub: verify - [source] id=hub ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:31:16:410 [140603195123456] 1 hub: TSESS-A-3-19 [SecondaryHA] sent 100 '_close' (296 bytes, 0 ms) Mar 30 15:31:16:410 [140603195123456] 3 hub: TSESS-A-3-19 [SecondaryHA] local session closed, suspending Mar 30 15:31:16:410 [140605283890944] 4 hub: cfgReader file open: security.cfg Mar 30 15:31:16:410 [140605283890944] 4 hub: cfgReader file close: security.cfg Mar 30 15:31:16:411 [140605283890944] 1 hub: PERF: hubi_check_security: 0 sec hubhash_size: 19 Mar 30 15:31:16:578 [140603203516160] 5 hub: TSESS-A-6-19 [Secondary] suspended Mar 30 15:31:16:594 [140603715176192] 5 hub: TSESS-A-5-79 [TunnelB] suspended Mar 30 15:31:16:610 [140603195123456] 5 hub: TSESS-A-3-19 [SecondaryHA] suspended Mar 30 15:31:16:963 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:31:16:991 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:31:17:022 [140605547460672] 4 hub: medium timeout Mar 30 15:31:17:022 [140605547460672] 3 hub: next_check [1585596696] now [1585596677] Mar 30 15:31:17:067 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:31:17:186 [140603161552640] 5 hub: TSESS-P-4-21 [SecondaryHA] suspended Mar 30 15:31:17:405 [140603153159936] 5 hub: TSESS-P-8-16 [Secondary] suspended Mar 30 15:31:17:608 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:31:17:939 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:31:18:075 [140603169945344] 5 hub: TSESS-P-7-19 [TunnelB] suspended Mar 30 15:31:18:703 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:31:19:771 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:31:21:056 [140605547460672] 5 hub: inametoip - lookup name /DOMAIN/TunnelB/ROBOT_NAME/hub Mar 30 15:31:21:056 [140605547460672] 2 hub: nametoip - /DOMAIN/TunnelB/ROBOT_NAME/hub 127.0.0.1/48009 - 0 ms (tunnel) Mar 30 15:31:21:057 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33428 Mar 30 15:31:21:057 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33428' delegated to TSESS-A-5 [140603715176192] Mar 30 15:31:21:057 [140603715176192] 3 hub: TSESS-A-5-80 [TunnelB] restarted (for local) Mar 30 15:31:21:062 [140603715176192] 5 hub: TSESS-A-5-80 [TunnelB] local 100 'get_info' Mar 30 15:31:21:062 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:31:21:062 [140603715176192] 1 hub: TSESS-A-5-80 [TunnelB] sent 100 'get_info' (313 bytes, 0 ms) Mar 30 15:31:21:072 [140603715176192] 1 hub: TSESS-A-5-80 [TunnelB] ssl 200 '-' (1278 bytes) Mar 30 15:31:21:072 [140603715176192] 3 hub: TSESS-A-5-80 [TunnelB] is paired with [P-298-80] Mar 30 15:31:21:072 [140603715176192] 1 hub: TSESS-A-5-80 [TunnelB] received 200 '-' (1286 bytes) Mar 30 15:31:21:073 [140603723568896] 2 hub: CTRL TunnelB new NIMSESS requested by 127.0.0.1/33430 Mar 30 15:31:21:073 [140603723568896] 2 hub: CTRL TunnelB session from '127.0.0.1/33430' delegated to TSESS-A-10 [140603706783488] Mar 30 15:31:21:073 [140603706783488] 3 hub: TSESS-A-10-59 [TunnelB] restarted (for local) Mar 30 15:31:21:077 [140603715176192] 5 hub: TSESS-A-5-80 [TunnelB] local 100 '_close' Mar 30 15:31:21:077 [140603715176192] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:31:21:077 [140603715176192] 1 hub: TSESS-A-5-80 [TunnelB] sent 100 '_close' (311 bytes, 0 ms) Mar 30 15:31:21:077 [140603715176192] 3 hub: TSESS-A-5-80 [TunnelB] local session closed, suspending Mar 30 15:31:21:078 [140603706783488] 5 hub: TSESS-A-10-59 [TunnelB] local 100 '_status' Mar 30 15:31:21:078 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:31:21:078 [140603706783488] 1 hub: TSESS-A-10-59 [TunnelB] sent 100 '_status' (313 bytes, 0 ms) Mar 30 15:31:21:088 [140603706783488] 1 hub: TSESS-A-10-59 [TunnelB] ssl 200 '-' (280 bytes) Mar 30 15:31:21:088 [140603706783488] 3 hub: TSESS-A-10-59 [TunnelB] is paired with [P-300-59] Mar 30 15:31:21:088 [140603706783488] 1 hub: TSESS-A-10-59 [TunnelB] received 200 '-' (288 bytes) Mar 30 15:31:21:093 [140603706783488] 5 hub: TSESS-A-10-59 [TunnelB] local 100 '_close' Mar 30 15:31:21:093 [140603706783488] 1 hub: verify - [source] id=HA ip=10.x.x.x hub=TunnelBHA (OK) Mar 30 15:31:21:094 [140603706783488] 1 hub: TSESS-A-10-59 [TunnelB] sent 100 '_close' (312 bytes, 0 ms) Mar 30 15:31:21:094 [140603706783488] 3 hub: TSESS-A-10-59 [TunnelB] local session closed, suspending Mar 30 15:31:21:278 [140603715176192] 5 hub: TSESS-A-5-80 [TunnelB] suspended Mar 30 15:31:21:294 [140603706783488] 5 hub: TSESS-A-10-59 [TunnelB] suspended Mar 30 15:31:21:573 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:31:23:375 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:31:25:178 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:31:26:578 [140603203516160] 5 hub: TSESS-A-6-19 [Secondary] suspended Mar 30 15:31:26:610 [140603195123456] 5 hub: TSESS-A-3-19 [SecondaryHA] suspended Mar 30 15:31:26:963 [140603186730752] 5 hub: TSESS-A-12-0 [Secondary] suspended Mar 30 15:31:26:991 [140603178338048] 5 hub: TSESS-A-9-0 [SecondaryHA] suspended Mar 30 15:31:27:062 [140605547460672] 4 hub: medium timeout Mar 30 15:31:27:062 [140605547460672] 3 hub: next_check [1585596696] now [1585596687] Mar 30 15:31:27:186 [140603161552640] 5 hub: TSESS-P-4-21 [SecondaryHA] suspended Mar 30 15:31:27:405 [140603153159936] 5 hub: TSESS-P-8-16 [Secondary] suspended Mar 30 15:31:27:608 [140602666645248] 5 hub: TSESS-P-14-4 [Secondary] suspended Mar 30 15:31:27:881 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0' Mar 30 15:31:27:939 [140602658252544] 5 hub: TSESS-P-11-0 [SecondaryHA] suspended Mar 30 15:31:28:075 [140603169945344] 5 hub: TSESS-P-7-19 [TunnelB] suspended Mar 30 15:31:28:703 [140602641467136] 5 hub: TSESS-P-13-0 [TunnelB] suspended Mar 30 15:31:29:684 [140603689998080] 3 hub: Sent heartbeat on queue route 't_0'