DevTest database migration error when starting IAM

[Gabor Roman]

Hi,

I've been trying to migrate from DevTest 10.3 to 10.6, and I've encountered a problem when I start up the Identity and Access Manager (IAM) component.
What I do in Docker is the following:

• I install DevTest 10.3 on (all the databases use utf8mb4 charset and collation):
  • container A with the Enterprise Dashboard (ED) + a separate container with a mysql 5.7.27 database for the ED.
  • container B with the Registry, Portal, and VSE + a separate container with a mysql 5.7.27 database for the Registry.
  • one additional machine with a mysql 5.7.27 database for the future IAM.
• I start up ED, then the Registry, Portal, and finally the VSE. I wait until they populate the databases. The dradis.properties on container A is configured for the ED DB, and the site.properties on container B is configured for the Registry DB.
• I stop all the 10.3 services.
• I install DevTest 10.6 without ED on container B in a different directory than the 10.3 installation, migrating the required files from the 10.3 installation directory into the 10.6 installation directory. I configure IdentityAccessManager/iam.properties on container B for the IAM DB.
• I start the IAM, I wait, then it stops.

As I check the IdentityAccessManager/standalone/log/server.log, I see that there are problems with some alter table statements. I've tried setting innodb_strict_mode=off, and also innodb_log_file_size=256M without any luck. Maybe there is some other problem that I cannot spot - sorry, I'm not an expert with DevTest. I've attached the logs from the server.log file.

Would you be so kind to help me resolving this problem? Thank you in advance for your response.

Sincerely,
Gábor Román

Attachment(s)

server.log   59 KB 1 version

[Gene Howard]

We can see the following in the logs provided:
2020-10-16 13:52:52,588 ERROR [org.keycloak.connections.jpa.updater.liquibase.conn.DefaultLiquibaseConnectionProvider] (ServerService Thread Pool -- 53) Change Set META-INF/jpa-changelog-1.9.1.xml::1.9.1::keycloak failed. Error: Row size too large. The maximum row size for the used table type, not counting BLOBs, is 65535. This includes storage overhead, check the manual. You have to change some columns to TEXT or BLOBs [Failed SQL: ALTER TABLE iamdb.REALM MODIFY CERTIFICATE VARCHAR(4000)]: liquibase.exception.DatabaseException: Row size too large. The maximum row size for the used table type, not counting BLOBs, is 65535. This includes storage overhead, check the manual. You have to change some columns to TEXT or BLOBs [Failed SQL: ALTER TABLE iamdb.REALM MODIFY CERTIFICATE VARCHAR(4000)]
at liquibase.executor.jvm.JdbcExecutor$ExecuteStatementCallback.doInStatement(JdbcExecutor.java:316)

this is a problem with the underlying database support
You are using MySQL 5.7 which is supported.
they only thing I can think that might be this issue is the  utf8mb4 charset and collation instead of utf8

https://techdocs.broadcom.com/us/en/ca-enterprise-software/devops/devtest-solutions/10-6/administering/database-administration/external-registry-database-configuration/configure-devtest-to-use-mysql.html

The MySQL database must provide collation and characters set supporting UTF-8; double-byte characters are stored in the ACL and reporting tables. The default code page for the database has to be UTF-8; it is not enough only to define your database as UTF-8. If the MySQL database is not UTF-8, then runtime errors occur.

Please redo the Database with the proper Character set and try again.

------------------------------
Gene Howard
Principal Support Engineer
Broadcom
------------------------------

[Gabor Roman]

Hi,

Thank you! I'm going to try your suggestion, and post the result.

Sincerely,
Gábor Román

[Gabor Roman]

Hi,

You were right!
After I've changed the character set to "utf8" and the collation to "utf8_general_ci", the "Row size too large..." errors are gone, and the IAM doesn't crash.
Thank you very much!

Sincerely,
Gábor Román