Service Virtualization

 View Only
Expand all | Collapse all

Unable to run DevTest-10.3 registry pod on openshift

  • 1.  Unable to run DevTest-10.3 registry pod on openshift

    Posted Nov 22, 2019 08:10 AM

    Hi,

    Am trying to deploy the DevTest-10.3 Registry application by using customized docker image on openshift  and run it as a pod. When the container started, it is throwing an exception while creating DevTest Registry.

    Creating a DevTest Registry, name=tcp://devtest.registry.new.in.int-a.nce1.paas.amadeus.net:32016/AEK_ACS_Registry_new
    Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
    java.lang.RuntimeException: java.lang.NullPointerException
    at com.itko.lisa.utils.JpaUtil.<init>(JpaUtil.java:484)
    at com.itko.lisa.utils.JpaUtil.getDradisUtil(JpaUtil.java:406)
    at com.itko.lisa.dradis.comm.JpaUtilWrapper.getAnnotations(JpaUtilWrapper.java:24)
    at com.itko.lisa.dradis.comm.DradisCommContext.doEnterpriseDashboardHandshake(DradisCommContext.java:133)
    at com.itko.lisa.coordinator.TestRegistryImpl.initialize(TestRegistryImpl.java:287)
    at com.itko.lisa.coordinator.TestRegistryImpl.<init>(TestRegistryImpl.java:242)
    at com.itko.lisa.coordinator.TestRegistryImpl.main(TestRegistryImpl.java:2184)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.exe4j.runtime.LauncherEngine.launch(Unknown Source)
    at com.install4j.runtime.launcher.UnixLauncher.main(Unknown Source)
    Caused by: java.lang.NullPointerException
    at com.itko.lisa.utils.db.ConnectionPoolManager.getUrl(ConnectionPoolManager.java:197)
    at com.itko.lisa.utils.db.ConnectionPoolManager.getDradisCacheUrl(ConnectionPoolManager.java:219)
    at com.itko.lisa.utils.db.DatabaseDialectProvider.getDradisCacheEclipselinkPlatform(DatabaseDialectProvider.java:42)
    at com.itko.lisa.utils.JpaUtil.<init>(JpaUtil.java:473)
    ... 12 more

    com.mysql.jdbc.Driver is for mysql 5.x and DevTest-10.3 supports only MySQL 5.5, 5.6, and 5.7, please see this "Database Requirements " from broadcom official page
    com.mysql.cj.jdbc.Driver is for mysql 8 and DevTest-10.3 doesn't support this.

    Could anyone help me, why this exception.

    Regards,
    Aravind



    ------------------------------
    Aravind
    ------------------------------


  • 2.  RE: Unable to run DevTest-10.3 registry pod on openshift
    Best Answer

    Broadcom Employee
    Posted Nov 26, 2019 02:31 AM
    'Customized docker image'. Who customized it and what was customized on top of the existing images?

    ------------------------------
    Head of Product Management: Service Virtualization
    CA Technologies - A Broadcom Company
    ------------------------------



  • 3.  RE: Unable to run DevTest-10.3 registry pod on openshift

    Posted Dec 03, 2019 05:54 AM
    Edited by Aravind Tiptur Venugopal Dec 03, 2019 06:17 AM
    Hi @Koustubh Warty,

    Thanks for your response.
    We have internal team who will provide support on DevTest tool. We received DevTest 10.3 docker image from internal team which contain all the components of DevTest 10.3. We fetch the docker image and we wrote our own Docker file to include our patches and dph, so that the customized docker image should have my application specific devtest patch and dph.

    But after a WebEx with Broadcom support team, i got to know we should maintain same version of mysql connector/j across the docker images, when i maintained the same version of connector/j (mysql-connector-java-8.0.17) and tried to deploy on openshift am facing nullpointerexception. PFA logs

    Setting-up the secrets
    Starting the Dashboard
    Reading user properties from /opt/CA/DevTest/bin//..//lisa.user.properties
    Locale: en_US
    Setting System.out and System.err to: ANSI_X3.4-1968
    LISA_HOME set to /opt/CA/DevTest/
    DevTest temporary directory '/opt/CA/DevTest/tmp' set by system property lisa.tmpdir
    DevTest temporary directory is /opt/CA/DevTest/tmp
    Configuring logging from '/opt/CA/DevTest/logging.properties'
    DevTest Solutions, Copyright (c) 2002-2019 CA, All rights reserved.
    DevTest Registry
    DevTest Solutions Version :: 10.3.0 (10.3.0.297)
    Installed patches:
    .... patch_DE359109_DE249873_10.3.0_GA.jar: iteration 1 built for version 10.3.0 of DevTest
    .... DE249873: Improve thread locking logic when attempting to communicate with multiple DevTest components, some of which may be down/unreachable. Make sure that the Registry can access a VSE based on its service URL before registering it and adding it to a lab.
    .... patch_DE389374_10.3.0_GA.jar: iteration 1 built for version 10.3.0 of DevTest
    .... DE389374: Need DevTest 10.3.0 Patch for "Updated VSI in Learning Mode not getting custom DPH details"
    ACL enabled: true

    Creating a DevTest Registry, name=tcp://devtest.registry.new.in.int-a.nce1.paas.amadeus.net:32016/AEK_ACS_Registry_new
    java.lang.RuntimeException: java.lang.NullPointerException
    at com.itko.lisa.utils.JpaUtil.<init>(JpaUtil.java:484)
    at com.itko.lisa.utils.JpaUtil.getDradisUtil(JpaUtil.java:406)
    at com.itko.lisa.dradis.comm.JpaUtilWrapper.getAnnotations(JpaUtilWrapper.java:24)
    at com.itko.lisa.dradis.comm.DradisCommContext.doEnterpriseDashboardHandshake(DradisCommContext.java:133)
    at com.itko.lisa.coordinator.TestRegistryImpl.initialize(TestRegistryImpl.java:287)
    at com.itko.lisa.coordinator.TestRegistryImpl.<init>(TestRegistryImpl.java:242)
    at com.itko.lisa.coordinator.TestRegistryImpl.main(TestRegistryImpl.java:2184)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.exe4j.runtime.LauncherEngine.launch(Unknown Source)
    at com.install4j.runtime.launcher.UnixLauncher.main(Unknown Source)
    Caused by: java.lang.NullPointerException
    at com.itko.lisa.utils.db.ConnectionPoolManager.getUrl(ConnectionPoolManager.java:197)
    at com.itko.lisa.utils.db.ConnectionPoolManager.getDradisCacheUrl(ConnectionPoolManager.java:219)
    at com.itko.lisa.utils.db.DatabaseDialectProvider.getDradisCacheEclipselinkPlatform(DatabaseDialectProvider.java:42)
    at com.itko.lisa.utils.JpaUtil.<init>(JpaUtil.java:473)
    ... 12 more


    Please let me know if you need more information.

    Regards,
    Aravind
    ​​


  • 4.  RE: Unable to run DevTest-10.3 registry pod on openshift

    Broadcom Employee
    Posted Dec 05, 2019 02:14 PM

    Ok got it. First things first - I am assuming that you are running the Dashboard and that you are able to login successfully?
      Also have you followed the steps specifically for MySQL as per our documentation? 


    https://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-enterprise-software/continuous-testing/devtest-solutions/10-3/system-requirements.html#concept.dita_a96de309f3a80b3bfa64d6fc2fffcb36383f77a8_DatabaseRequirements


     checkout section on Database Requirements and MySQL paragraph. 



    ------------------------------
    Head of Product Management: Service Virtualization
    CA Technologies - A Broadcom Company
    ------------------------------



  • 5.  RE: Unable to run DevTest-10.3 registry pod on openshift

    Posted Dec 18, 2019 04:12 AM
    Edited by Aravind Tiptur Venugopal Dec 18, 2019 04:30 AM

    Hi @Koustubh Warty,

    Sorry for the late response.
    Yes am able to access Dashboard.

    All the database details am following as per the above documentation.

    Am able to bypass the above issue, but now i got into other issue as said below:

    Registry Logs:

    ---- ERROR http://ncedevtestapp01.nce.amadeus.net:1506/api/v1/dradis/registries/AEK_ACS_Registry_new%40devtest-registry-new-1-4qj2x%3A32016
    java.net.SocketTimeoutException: timeout
        at okio.Okio$3.newTimeoutException(Okio.java:207)
        at okio.AsyncTimeout.exit(AsyncTimeout.java:261)
        at okio.AsyncTimeout$2.read(AsyncTimeout.java:215)
        at okio.RealBufferedSource.indexOf(RealBufferedSource.java:306)
        at okio.RealBufferedSource.indexOf(RealBufferedSource.java:300)

    DB logs:

    2019-12-18  4:53:42 139713341024000 [Warning] Aborted connection 17 to db: 'commondb_10_3' user: 'iQ743XodhwtfAq5t' host: '10.225.67.12' (Got an error reading communication packets)

    Note: 1) Am able  to see my registry registered in Enterprise Dashboard
               2) Am able to see the tables created in commondb_10_3

    Regards,
    Aravind




  • 6.  RE: Unable to run DevTest-10.3 registry pod on openshift

    Posted Dec 18, 2019 04:30 AM
    PFA registry and DB logfiles

    Attachment(s)

    txt
    MariaDB.txt   17 KB 1 version
    txt
    Registry.txt   35 KB 1 version


  • 7.  RE: Unable to run DevTest-10.3 registry pod on openshift

    Broadcom Employee
    Posted Dec 21, 2019 09:40 AM
    @Aravind Tiptur Venugopal -first of all - lets break this down into multiple parts

    1. MariaDB is not supported out of the box as a database.
    2. You are using OpenShift which we have not certified as yet.

    So this is not a straightforward installation and configure.

    Questions:
    1. You mentioned you were able to bypass​ the earlier issue. Please provide details on what the problem was and how it was fixed?

    2. Where is your MariaDB as well as the Enterprise Dashboard located as compared to the Registry?

    3. Can you see if you are able to get the Registry to come up using the default Derby?

    Thanks!


    ------------------------------
    Head of Product Management: Service Virtualization
    CA Technologies - A Broadcom Company
    ------------------------------



  • 8.  RE: Unable to run DevTest-10.3 registry pod on openshift

    Posted Dec 22, 2019 06:40 AM
    Hi @Koustubh Warty,

    Thanks for your response.

    1. You mentioned you were able to bypass​ the earlier issue. Please provide details on what the problem was and how it was fixed?
    @Aravind Tiptur Venugopal
    There was a typo error in below said variables of site.properties file
    lisadb.reporting.poolName=common
    lisadb.dradiscache.poolName=common

    In place of lisadb.pool.common.password_enc=SQL_PASSWORD i used lisadb.pool.common.password=SQL_PASSWORD

    2. Where is your MariaDB as well as the Enterprise Dashboard located as compared to the Registry?
    @Aravind Tiptur Venugopal MariaDB we are running it as a different POD on the same openshift environment. Enterprise Dashboard is running as part of the Registry POD.

    3. Can you see if you are able to get the Registry to come up using the default Derby?
    ​​​​@Aravind Tiptur Venugopal We are not supposed to use default Derby DB

    As per me there is some problem with my configuration part which am unable to figure it out.
    Also as a part of debugging process, i analyzed the events of the POD and i see below error when i use kubectl -n devtest describe pod devtest-registry-new-2-rs8zg this command

    Warning Unhealthy 9s (x12 over 4m19s) kubelet, ose3-int-a-minion-a-a1.node.nce1-1a-eu-nce-2.acs Readiness probe failed: Get http://10.225.87.31:1505/: dial tcp 10.225.87.31:1505: getsockopt: connection refused

    so i removed the below snippet from my template file and i see my POD is not rolling back, but am unable to access the service (http://devtest.registry.new.in.int-a.nce1.paas.amadeus.net/)  from the browser



    "readinessProbe": {
    "timeoutSeconds": 1,
    "initialDelaySeconds": 60,
    "httpGet": {
    "port": 1505
    }

    Note: The log files which i have attached in my previous response are captured in DEBUG mode.

    Regards,
    Aravind​


  • 9.  RE: Unable to run DevTest-10.3 registry pod on openshift

    Posted Jan 03, 2020 01:07 AM
    Hi Aravind,

    I assume your POD name is "ncedevtestapp01.nce.amadeus.net", please validate if not this then you might have to change configuration in your docker.

    Also, make sure the ports 1505,1506, 1528 etc. are not having any processes running before initiating the installation process.

    Thanks.

    ------------------------------
    Regards,
    Vaibhav Jain
    Capgemini
    ------------------------------



  • 10.  RE: Unable to run DevTest-10.3 registry pod on openshift

    Posted Feb 12, 2020 02:24 AM

    Hi @Vaibhav Jain,

    Thanks for the response.

    ncedevtestapp01.nce.amadeus.net is not the POD name, it is enterprise dashboard host name. Could you please explain me how does POD name will cause this issue.

    Also the ports mentioned are all free. All these ports are user defined.

    Regards,
    Aravind




  • 11.  RE: Unable to run DevTest-10.3 registry pod on openshift

    Posted Feb 12, 2020 03:39 AM
    Hi Aravind,

    When i did the openshift based CA devtest configuration, the ED, registry everything was configured on the same pod. Hence, I assumed, ncedevtestapp01.nce.amadeus.net is the pod name. If this is the ED hostname (other than POD), then both POD and ED host should have the connectivity and ports should be enabled to communicate twoway from both side.

    Thanks.

    ------------------------------
    Regards,
    Vaibhav Jain
    Capgemini
    ------------------------------



  • 12.  RE: Unable to run DevTest-10.3 registry pod on openshift

    Posted Dec 18, 2019 04:19 AM
    PFA logfiles