Symantec Privileged Access Management

  • 1.  TIBCO error msgs

    Posted 11-19-2015 06:07 AM

    Hello

    we are running PIM 12.9

    I noticed some error msg in the ems.log and acrptmq.log as the following example:

     

    2015-11-19 12:58:56.868 ERROR: Transaction for non-existent consumer: 153462 connID=1640979 sessID=175229 {formatID=131075 gtrid_length=29 bqual_length=27 data=1-7f000001:9533:564d9ba5:72a67f000001:9533:564d9ba5:72ab}

     

    2015-11-19 12:58:56.868 ERROR: Error processing xa end - transaction marked ROLLBACKONLY, Exception. connID=1640979 sessID=175229 {formatID=131075 gtrid_length=29 bqual_length=27 data=1-7f000001:9533:564d9ba5:72a67f000001:9533:564d9ba5:72ab}

     

    2015-11-19 12:58:56.869 ERROR: Error processing xa prepare request, Transaction rollback. connID=1640979 {formatID=131075 gtrid_length=29 bqual_length=27 data=1-7f000001:9533:564d9ba5:72a67f000001:9533:564d9ba5:72ab}

     

    how can I identify these error , in order to fix them ?!

     

    thanks



  • 2.  Re: TIBCO error msgs
    Best Answer

    Broadcom Employee
    Posted 11-22-2015 07:22 PM

    Hi wael_abdelwahab,

     

    I started looking at this and thought it best to look in ems.log on one of my test servers and to see if I had the same errors. By coincidence I have come across what appears to be the same problem, but from looking in server.log rather than ems.log.

     

    The same errors in my ems.log:

     

    2015-10-30 09:16:51.630 ERROR: Transaction for non-existent consumer: 998 connID=2144 sessID=2580 {formatID=131075 gtrid_length=28 bqual_length=26 data=1-7f000101:c110:5632e4f3:23f7f000101:c110:5632e4f3:244} 
    2015-10-30 09:16:51.630 ERROR: Transaction for non-existent consumer: 999 connID=2144 sessID=2580 {formatID=131075 gtrid_length=28 bqual_length=26 data=1-7f000101:c110:5632e4f3:23f7f000101:c110:5632e4f3:244} 
    2015-10-30 09:16:51.630 ERROR: Transaction for non-existent consumer: 1000 connID=2144 sessID=2580 {formatID=131075 gtrid_length=28 bqual_length=26 data=1-7f000101:c110:5632e4f3:23f7f000101:c110:5632e4f3:244} 
    2015-10-30 09:16:51.631 ERROR: Error processing xa end - transaction marked ROLLBACKONLY, Exception. connID=2144 sessID=2580 {formatID=131075 gtrid_length=28 bqual_length=26 data=1-7f000101:c110:5632e4f3:23f7f000101:c110:5632e4f3:244} 
    2015-10-30 09:16:51.631 ERROR: Error processing xa prepare request, Transaction rollback. connID=2144 {formatID=131075 gtrid_length=28 bqual_length=26 data=1-7f000101:c110:5632e4f3:23f7f000101:c110:5632e4f3:244} 

     

    I then matched them to the following errors in server.log, which I had previously come across:

     

    2015-10-30 09:16:51,636 WARN  [com.arjuna.ats.jta.logging.loggerI18N] [com.arjuna.ats.internal.jta.resources.arjunacore.preparefailed] [com.arjuna.ats.internal.jta.resources.arjunacore.preparefailed] XAResourceRecord.prepare - prepare failed with exception XAException.XAER_RMERR  
    2015-10-30 09:16:51,638 WARN  [com.arjuna.ats.arjuna.logging.arjLoggerI18N] [com.arjuna.ats.arjuna.coordinator.BasicAction_50] - Prepare phase of action 7f000101:c110:5632e4f3:23f received heuristic decision: TwoPhaseOutcome.HEURISTIC_HAZARD 
    2015-10-30 09:16:51,638 WARN  [com.arjuna.ats.arjuna.logging.arjLoggerI18N] [com.arjuna.ats.arjuna.coordinator.BasicAction_36] - BasicAction.End() - prepare phase of action-id 7f000101:c110:5632e4f3:23f failed. 
    2015-10-30 09:16:51,639 WARN  [com.arjuna.ats.arjuna.logging.arjLoggerI18N] [com.arjuna.ats.arjuna.coordinator.BasicAction_37] - Received heuristic: TwoPhaseOutcome.HEURISTIC_HAZARD . 
    2015-10-30 09:16:51,639 WARN  [com.arjuna.ats.arjuna.logging.arjLoggerI18N] [com.arjuna.ats.arjuna.coordinator.BasicAction_38] - Action Aborting 
    2015-10-30 09:16:51,834 ERROR [org.jboss.jms.asf.StdServerSession] failed to commit/rollback 
    javax.transaction.HeuristicMixedException
              at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1397)
              at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:135)
              at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:87)
              at org.jboss.jms.asf.StdServerSession.onMessage(StdServerSession.java:351)
              at com.tibco.tibjms.TibjmsxSessionImp._submit(TibjmsxSessionImp.java:4001)
              at com.tibco.tibjms.TibjmsxSessionImp._dispatchAsyncMessage(TibjmsxSessionImp.java:2220)
              at com.tibco.tibjms.TibjmsxSessionImp._run(TibjmsxSessionImp.java:3477)
              at com.tibco.tibjms.TibjmsxSessionImp.run(TibjmsxSessionImp.java:4680)
              at org.jboss.jms.asf.StdServerSession.run(StdServerSession.java:194)
              at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:761)
              at java.lang.Thread.run(Thread.java:745)

     

    I suspect that if you have similar messages in your server.log.20015-11-19 around 2015-11-19 12:58:56 then this will be fixed using document TEC614398:

     

    Description:

     

    Automatic Password reset results in a

    "javax.transaction.HeuristicMixedException" exception while resetting

    passwords.

     

    Solution:

     

    Do the following:

     

    Navigate to the following directory: /JBOSS_HOME/server/default/deploy/jms

     

    Open the jms-ds.xml and tibco-jms-ds.xml for editing.

     

    Add the xml tag entries <track-connection-by-tx></track-connection-by-tx> in

    <tx-connection-factory> in both files.

     

    For example:

     

    .

    <tx-connection-factory>

    ....................................

    <track-connection-by-tx></track-connection-by-tx>

    .......................................

    </tx-connection-factory>

     

     

    Restart the JBoss service.

     

    http://www.ca.com/us/support/ca-support-online/product-content/knowledgebase-articles/tec614398.aspx

     

    Note: This problem manifested itself with me by very occasionally having duplicate entries in Privileged Accounts -> Accounts -> Password History. Both entries would have the same password. This does not seem to affect the actual password change on the account.



  • 3.  Re: TIBCO error msgs

    Posted 11-23-2015 04:02 AM

    Thanks Simon_Naunton , I found that specific event on that time ,

    anyway when I add this line the next line just after  the <tx-connection-factory>

    the jboss was unable to connect to the ssl://localhost:7243 !!

    but when I put that line just before the closing tag </tx-connection-factory> , thing went correctly .

    on the other hand I found lot of these WARN lines in the sever.log

     

     

    2015-11-23 10:54:01,716 WARN  [com.arjuna.ats.jta.logging.loggerI18N] [com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa] [com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa] Could not find new XAResource to use for recovering non-serializable XAResource < 131075, 31, 29, 1-7f000001:afaa:563f2782:1647fa7f000001:afaa:563f2782:1647ff >

     

    2015-11-23 10:54:01,726 WARN  [com.arjuna.ats.jta.logging.loggerI18N] [com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa] [com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa] Could not find new XAResource to use for recovering non-serializable XAResource < 131075, 30, 28, 1-7f000001:c379:562cc18e:c2c887f000001:c379:562cc18e:c2c8d >

     

    2015-11-23 10:54:01,734 WARN  [com.arjuna.ats.jta.logging.loggerI18N] [com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa] [com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa] Could not find new XAResource to use for recovering non-serializable XAResource < 131075, 30, 28, 1-7f000001:da3c:56239500:6dc297f000001:da3c:56239500:6dc2e >

     

    and these lines when I am opening and using the Enterprise Management console:

     

    2015-11-23 10:55:29,540 WARN  [org.apache.myfaces.shared_impl.renderkit.html.HtmlGridRendererBase] PanelGrid main:subviewHostnameTable:hostTable:49:lstManagedDevices:0:panelMDActionsMany has not enough children. Child count should be a multiple of the columns attribute.

     

    2015-11-23 10:55:29,542 WARN  [org.apache.myfaces.shared_impl.renderkit.html.HtmlGridRendererBase] PanelGrid main:subviewHostnameTable:hostTable:49:lstManagedDevices:1:panelMDActionsMany has not enough children. Child count should be a multiple of the columns attribute.

     

    these repeating lines make the server.log is huge at the end of the day (near 300 MB size)



  • 4.  Re: TIBCO error msgs

    Broadcom Employee
    Posted 11-23-2015 04:25 AM

    Hi Wael

    Based on what we have seen previously

    The WARN message "Could not find new XAResource to use for recovering non-serializable XAResource" means that after a failure, the transaction manager tries to find a connector to recover (normally rollback) a transaction with a known id.

    This is a warning and shouln't be any specific functionality.

    You can try to perform below steps
    1.Stop Jboss
    2.Go to <JBOSS_DIR>\server\default\conf folder and edit jbossjta-properties.xml file
    Original settings
    <property
    name="com.arjuna.ats.arjuna.recovery.expiryScanInterval" value="12"
    change to
    name="com.arjuna.ats.arjuna.recovery.expiryScanInterval" value="1"
    Original settings
    <property
    name="com.arjuna.ats.arjuna.recovery.transactionStatusManagerExpiryTime" value="12"/>
    change to
    name="com.arjuna.ats.arjuna.recovery.transactionStatusManagerExpiryTime" value="1"/>
    Original settings
    <property
    name="com.arjuna.ats.arjuna.recovery.periodicRecoveryPeriod" value="120"/>
    change to
    <property
    name="com.arjuna.ats.arjuna.recovery.periodicRecoveryPeriod" value="7200"/>
    Original settings
    <property
    name="com.arjuna.ats.arjuna.recovery.recoveryBackoffPeriod" value="10"/>
    change to
    <property
    name="com.arjuna.ats.arjuna.recovery.recoveryBackoffPeriod" value="7200"/>
    3.Start Jboss.

     

    Setting these properties will clean Jboss transaction queue before it start

    to process them, this doesn't affect Enterprise Manager's product

    functionality because we are not leaning on Jboss transactions.



  • 5.  Re: TIBCO error msgs

    Posted 11-23-2015 04:37 AM

    Thanks Miquel-Gilibert-i-Sunye, I will do it right now



  • 6.  Re: TIBCO error msgs

    Posted 11-25-2015 03:27 AM

    Hello

    these warning have return to appear once again. even after the modification I have made



  • 7.  Re: TIBCO error msgs

    Broadcom Employee
    Posted 11-23-2015 04:36 AM

    With respect to the second message, this means basically for what I have seen about this issue looking around, this means that the number of records returned is not a multiple of the number of columns

    See

    _http://www.coderanch.com/t/212401/JSF/java/Warning-children

     

    for instance. it is not a dangerous warning, but if it is annoying you may want to open a support case for this