Some Projects can not be saved in CA PPM 14.3 SP1 or in Open Workbench. Deadlocks in Task and Assignments

[Patrice Dumont]

Hi everyone,

 

We got some issues when we want to save some projects in CA PPM 14.3 SP1 or in Open Workbench since two weeks.

 

We have to kill Open Workbench session when it happens but updates are not saved, and in CA PPM the saving function loops without updating projects. The oracle SQL transaction is running during some hours after trying to save our updates.

 

We restart CA PPM services every night, sometimes projects can be saved the next day and sometimes not...

 

We found nothing in PRLOCK table, and we got no idea why it happens and how resolve this problem.

 

We got this sort of errors in app.log :

 

ERROR 2017-10-20 09:46:07,275 [http-nio-443-exec-95] xql2.rule (clarity:info0771:7809055__C6034FF8-D6CC-46BD-BEEF-C89C3754EEE2:npt.gridUpdate) InvInvestmentsRule.updateProxyTasks
java.lang.NullPointerException
at com.niku.xql2.pmd.rules.INVInvestmentsRule.updateProxyTasks(INVInvestmentsRule.java:686)
at com.niku.xql2.pmd.rules.INVInvestmentsRule.onPostUpdate(INVInvestmentsRule.java:593)
at com.niku.xql2.odf.ODFRecord.update(ODFRecord.java:267)
at com.niku.xql2.handlers.UpdateHandler.postProcess(UpdateHandler.java:103)
at com.niku.xql2.XQLVisitor.postProcess(XQLVisitor.java:1419)
at com.niku.union.xml.dom.DOMWalker.postProcess(DOMWalker.java:210)
at com.niku.union.xml.dom.DOMWalker.traverseIntern(DOMWalker.java:94)

 

and then

 

ERROR 2017-10-20 13:43:29,745 [http-nio-443-exec-22] niku.schedulers (clarity:info0776:7811048__20A50703-5A42-4333-B9EC-7BA8284146D9:schedulers.postProject) Rolling back transaction:
com.niku.union.persistence.PersistenceException:
SQL error code: 0
Error message: [CA Clarity][Oracle JDBC Driver]Operation cancelled by user request.
Executed:
select a.*
, r.resource_type
, tm.prid teamId
from prassignment a
, prteam tm
, srm_resources r
, prtask t
where t.prprojectid=?
and a.prtaskid=t.prid
and r.id=a.prresourceid
and tm.prid = a.team_id and r.resource_type=?
Derived from statement:

 

and yesterday :

 

ERROR 2017-11-06 11:44:32,271 [http-nio-443-exec-1689] niku.schedulers (clarity:info0653:7874055__FF91E59A-C310-4D43-8110-D0410FCFAB65:schedulers.postProject) Rolling back transaction:
com.niku.union.persistence.DBConnectionException: java.sql.SQLNonTransientConnectionException: [CA Clarity][Oracle JDBC Driver]Internal error: Net8 protocol error.
at com.niku.union.persistence.PersistenceController.handleSQLException(PersistenceController.java:2111)
at com.niku.union.persistence.PersistenceController.processSql(PersistenceController.java:2823)
at com.niku.union.persistence.PersistenceController.processStatement(PersistenceController.java:869)
at com.niku.union.persistence.PersistenceController.processStatements(PersistenceController.java:769)
at com.niku.union.persistence.PersistenceController.doProcessRequest(PersistenceController.java:577)
at com.niku.union.persistence.PersistenceController.processRequest(PersistenceController.java:307)
at com.niku.xql2.pmd.PMDRecordSet.executePMDStatement(PMDRecordSet.java:267)
at com.niku.xql2.pmd.PMDRecordSet.<init>(PMDRecordSet.java:87)
at com.niku.xql2.pmd.PMDDataSource.select(PMDDataSource.java:94)
at com.niku.xql2.pmd.rules.InvestmentTSVController.getInvestmentAssignments(InvestmentTSVController.java:948)

 

Did anyone got any idea for resolving our issue ? Help will be appreciated, thank you.

 

Regards,

 

Ebubekir

[Legacy User]

As per the description, I believe that the issue is project specific. If that is the case, try to find out the similarity between the projects, which might be the rootcause of the problem:

1. Are they having multiple Baselines?

2. Are they having multiple subprojects?

3. When you are opening one of the affected projects in OWB, are you seeing a lots of tasks?

 

You can check the "Only Export Current Baselines When Opening Investments in a Scheduler" option from Admin --> Project Management --> Settings page. That might also help. 

[Legacy User]

If the issue is happening due to the heavy size of the projects (which is mentioned in point 2 and 3), then probably you can try to increase the Java memory limit of the OWB and see if that solves the issue. 

 

This technical document will help you how to increase the Java memory limit for your Open Workbench scheduler: 
How do you set the maximum java memory usage for Open Workbench versions 

Hope that helps. 

[Patrice Dumont]

Hi Abhisek,

 

Thank you very much for your return.

 

When we want to save these projects in Open Workbench, we often have this problem but not every time, sometimes the project can be saved and sometimes not.

 

After the lock of PRTASK and PRASSIGNMENT for theses projects (timesheets wich impact them are also blocked, and some procedures like update_wbs or update_cmn_pac_projects too), the project is blocked and it can not be saved in OWB or in CA PPM. We have to wait the nightly services restart for updating the project in CA PPM the next day, so the problem seems to come from OWB.

 

These three projects don't have multiple baselines or subprojects, and no more of 100 tasks so they are not very big.

 

We have some core dumps problems with our CA PPM version, so it can be a problem with Java memory in OWB...I will try to test your solution and then I will make a return.

 

Thank you.

 

Regards, Ebubekir

[Legacy User]

Yeah that would be best for you at this moment. Will be waiting to hear back from you on this. Let us hope for the best.

[Patrice Dumont]

Hi,

 

We just fixed our issue after changing our JDBC Oracle driver.

 

We had some net8 protocol errors when we wanted to execute Time Slicing or Investment Allocation or saving projects in OWB.

 

The problem comes from our JDVC driver, it is described in this article :

https://knowledgebase.progress.com/articles/Article/Object-has-been-closed-or-Internal-error-Net8-protocol-error-error-with-Oracle-JDBC-driver-when-executing-multiple-statements-on-the-same-connection

 

CA support provide to us the c-oracle.jarfile from an CAPPM 15.1 environment and we replace our c-oracle.jar file of our 14.3 version with it, solution described here : https://support.ca.com/us/knowledge-base-articles.TEC1461201.html

 

Regards, Patrice DUMONT

[Suman Pramanik]

Glad to hear that problem is resolved. I would recommend now planning for upgrade as 14.3 is already end of life

[Patrice Dumont]

Yes we planned to migrate in 15.2 or 15.3 version in Mars or April, thank you.