Hello Rajdeep,
A lot of this looks like a communication error and not necessarily an upgrade issue. I would make sure that the server can see one another and there is no issues with packet loss. Then I would check to see if the DB information on your gateway is correct and make sure it was not somehow overwritten during the upgrade. But based on the error I would lean more towards the networking side of things.
------------------------------
Support Engineer
Broadcom
------------------------------
Original Message:
Sent: 05-19-2020 12:53 AM
From: Rajdeep Singh
Subject: Getting error while upgrading gateway from 9.3 to 9.4 versions
While upgrading an AWS CA gateway clustered environment from 9.3 to 9.4, we are running in an issue. Can you take a look at the logs and help us identify the problem.
2020-05-17T10:04:20.873+0000 WARNING 1 org.hibernate.util.JDBCExceptionReporter: SQL Error: 0, SQLState: 08S01
2020-05-17T10:04:20.874+0000 SEVERE 1 org.hibernate.util.JDBCExceptionReporter: Communications link failure
The last packet successfully received from the server was 120,093 milliseconds ago. The last packet sent successfully to the server was 120,093 milliseconds ago.
2020-05-17T10:04:20.878+0000 WARNING 1 com.l7tech.server.policy.assertion.ServerHttpRoutingAssertion: 4000: Could not initialize SSL Context
2020-05-17T10:04:20.879+0000 WARNING 1 com.l7tech.server.policy.assertion.ServerHttpRoutingAssertion: 4: Failed to initialize SSL context: could not execute query; nested exception is org.hibernate.exception.JDBCConnectionException: coul
d not execute query. Exception caught!
2020-05-17T10:04:20.930+0000 WARNING 1 org.hibernate.util.JDBCExceptionReporter: SQL Error: 0, SQLState: 08003
2020-05-17T10:04:20.931+0000 SEVERE 1 org.hibernate.util.JDBCExceptionReporter: No operations allowed after connection closed.
2020-05-17T10:04:20.931+0000 WARNING 1 com.l7tech.policy.Policy: Unable to prepare entity for assertion ordinal 70 of policy GUID eec5c295-fbe9-40b9-a6d4-a261edbdd180 (API Portal SSO SAML Validation Service Fragment): Couldn't check uniq
ueness
2020-05-17T10:04:20.932+0000 WARNING 1 org.hibernate.util.JDBCExceptionReporter: SQL Error: 0, SQLState: 08003
2020-05-17T10:04:20.932+0000 SEVERE 1 org.hibernate.util.JDBCExceptionReporter: No operations allowed after connection closed.
2020-05-17T10:04:20.941+0000 WARNING 1 com.l7tech.server.policy.PolicyCacheImpl: 3252: Error accessing policy (#{1})
com.l7tech.objectmodel.FindException: Error finding entities
at com.l7tech.server.HibernateEntityManager.findMatching(Unknown Source)
at com.l7tech.server.HibernateEntityManager.findMatching(Unknown Source)
at com.l7tech.server.policy.PolicyVersionManagerImpl.findActiveVersionForPolicy(Unknown Source)
at sun.reflect.GeneratedMethodAccessor304.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy125.findActiveVersionForPolicy(Unknown Source)
at com.l7tech.server.policy.PolicyCacheImpl.a(Unknown Source)
at com.l7tech.server.policy.PolicyCacheImpl.b(Unknown Source)
at com.l7tech.server.policy.PolicyCacheImpl.d(Unknown Source)
at com.l7tech.server.policy.PolicyCacheImpl.c(Unknown Source)
at com.l7tech.server.policy.PolicyCacheImpl.update(Unknown Source)
at com.l7tech.server.service.ServiceCache.a(Unknown Source)
at com.l7tech.server.service.ServiceCache.cache(Unknown Source)
at com.l7tech.server.service.ServiceCache.initializeServiceCache(Unknown Source)
at com.l7tech.server.service.aw.doInTransactionWithoutResult(Unknown Source)
at org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:33)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:130)
at com.l7tech.server.service.ServiceCache.onApplicationEvent(Unknown Source)
at com.l7tech.server.util.r.onApplicationEvent(Unknown Source)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:97)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:303)
at com.l7tech.server.BootProcess.start(Unknown Source)
at com.l7tech.server.boot.GatewayBoot.f(Unknown Source)
at com.l7tech.server.boot.GatewayBoot.start(Unknown Source)
at com.l7tech.server.boot.GatewayBoot.runUntilShutdown(Unknown Source)
at com.l7tech.server.boot.GatewayMain.main(Unknown Source)
Caused by: org.springframework.dao.DataAccessResourceFailureException: could not execute query; nested exception is org.hibernate.exception.JDBCConnectionException: could not execute query
at org.springframework.orm.hibernate3.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:625)
at org.springframework.orm.hibernate3.HibernateAccessor.convertHibernateAccessException(HibernateAccessor.java:412)
at org.springframework.orm.hibernate3.HibernateTemplate.doExecute(HibernateTemplate.java:411)
at org.springframework.orm.hibernate3.HibernateTemplate.execute(HibernateTemplate.java:339)
... 33 more
Caused by: org.hibernate.exception.JDBCConnectionException: could not execute query
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:99)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
at org.hibernate.loader.Loader.doList(Loader.java:2545)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2276)
at org.hibernate.loader.Loader.list(Loader.java:2271)
at org.hibernate.loader.criteria.CriteriaLoader.list(CriteriaLoader.java:119)
at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1716)
at org.hibernate.impl.CriteriaImpl.list(CriteriaImpl.java:347)
at com.l7tech.server.bt.doInHibernateReadOnly(Unknown Source)
at com.l7tech.server.bt.doInHibernateReadOnly(Unknown Source)
at com.l7tech.server.util.ReadOnlyHibernateCallback.doInHibernate(Unknown Source)
at org.springframework.orm.hibernate3.HibernateTemplate.doExecute(HibernateTemplate.java:406)
... 34 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.Util.getInstance(Util.java:386)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1013)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:987)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:982)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:927)
at com.mysql.jdbc.ConnectionImpl.throwConnectionClosedException(ConnectionImpl.java:1251)
at com.mysql.jdbc.ConnectionImpl.checkClosed(ConnectionImpl.java:1243)
at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4216)
at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4182)
at com.mchange.v2.c3p0.impl.NewProxyConnection.prepareStatement(NewProxyConnection.java:213)
at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy$LazyConnectionInvocationHandler.invoke(LazyConnectionDataSourceProxy.java:376)
at com.sun.proxy.$Proxy71.prepareStatement(Unknown Source)
at org.hibernate.jdbc.AbstractBatcher.getPreparedStatement(AbstractBatcher.java:534)
at org.hibernate.jdbc.AbstractBatcher.getPreparedStatement(AbstractBatcher.java:452)
at org.hibernate.jdbc.AbstractBatcher.prepareQueryStatement(AbstractBatcher.java:161)
at org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1700)
at org.hibernate.loader.Loader.doQuery(Loader.java:801)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:274)
at org.hibernate.loader.Loader.doList(Loader.java:2542)
... 43 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 120,093 milliseconds ago. The last packet sent successfully to the server was 120,093 milliseconds ago.