graphman import issue

[Michael Mueller]

Dear team.

I encountered the following issue, when trying to update an existing api throughh graphman.

Error:

{
  "errors": [
    {
      "message": "Exception while fetching data (/setWebApiServices) : could not execute statement; SQL [n/a]; constraint [published_service.PRIMARY]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement",
      "locations": [
        {
          "line": 53,
          "column": 9
        }
      ],
      "path": [
        "setWebApiServices"
      ],
      "extensions": {
        "classification": "DataFetchingException"
      }
    }
  ],
  "data": {}
}

Any obvious root cause, or hint , what I am doing wrong?

Thanks

...Michael

[Michael Mueller]

I found the following , obviously related messages in the ssg log

Update of first existing API
2024-02-08T09:26:19.030+0000 INFO    1141 com.l7tech.server.admin: PublishedService #5526a9fb68dc99b77dcac358120e9b37 (check input) updated
2024-02-08T09:26:19.031+0000 INFO    1141 com.l7tech.server.admin: Policy #579cb701bc5e248f153b2a2d18e914a3 (Policy for service #5526a9fb68dc99b77dcac358120e9b37, check input) updated (changed xml)

Obviously trying to create the second API, even though it exists.
2024-02-08T09:26:19.035+0000 INFO    1141 com.l7tech.server.admin: PublishedService #2cdeca1091319223f1ad2339c6ef1c68 (mm-new-api2) created
2024-02-08T09:26:19.036+0000 INFO    1141 com.l7tech.server.service.ServiceManagerImp: Saved service #2cdeca1091319223f1ad2339c6ef1c68
2024-02-08T09:26:19.038+0000 INFO    1141 com.l7tech.server.admin: Policy #8eed69b32d1ad8b7a03d51cbf4b7c37b (Policy for service #2cdeca1091319223f1ad2339c6ef1c68, mm-new-api2) created
2024-02-08T09:26:19.043+0000 INFO    1141 com.l7tech.server.admin: PolicyVersion #8eed69b32d1ad8b7a03d51cbf4b7c37c (null) created (activated v2 of policy 8eed69b32d1ad8b7a03d51cbf4b7c37b)
2024-02-08T09:26:19.046+0000 INFO    1141 com.l7tech.server.admin: PublishedService #2cdeca1091319223f1ad2339c6ef1c68 (mm-new-api2) updated (set policy)
2024-02-08T09:26:19.050+0000 WARN    1141 org.hibernate.engine.jdbc.spi.SqlExceptionHelper: SQL Error: 1062, SQLState: 23000
2024-02-08T09:26:19.050+0000 ERROR   1141 org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Duplicate entry ',\xDE\xCA\x10\x911\x92#\xF1\xAD#9\xC6\xEF\x1Ch' for key 'published_service.PRIMARY'
2024-02-08T09:26:19.051+0000 WARNING 1141 com.l7tech.external.assertions.gatewaygraphql.server.util.GatewayDataFetcherExceptionHandler: Exception while fetching data (/setWebApiServices) : could not execute statement; SQL [n/a]; constraint [published_service.PRIMARY]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement, could not execute statement

my json bundle to import consist of 2 webApiServices. both already existing in the target gw.

Hence I would expect two updates:

[Francois Lascelles]

Hi Michael,

If it's creating instead of updating, maybe it's because the target has mismatch goid/guid/targetUri. If there is nothing special with targetUri, you can try stripping out goid/guid from bundle before applying it just to see if that helps. If you send the bundle you are using and the query result for the same service on the target gateway, we can look into it with more details.

[Raju Gurram]

These sort of errors could be expected due to the violation of the DB constraints. As Francois mentioned, please try mutation without goid/guid fields. Or, assign them to new values. 

[Francois Lascelles]

Thanks for your question Michael,

We will look into more deeply but the first thing that comes to mind to me is this "Duplicate entry... 'published_service.PRIMARY'". If the service exists, but the /resolutionPath is different, or the bundle specifies a guid or goid that collides with the goid/guid on the target. If you remove goid/guid from the bundle, and apply, what happens?

To dig deeper, I would request the bundle you are applying (you can strip out the policies if that helps) and the query result for the same services on the target target gateway.

[Raju Gurram]

Looking at the posted error response from the graphman, I think you are with the older version of gateway. We've improved this with the latest available. May I know the gateway and graphman-client version?

[Michael Mueller]

Thanks you for comments, Francois and Raju.

@Francois Lascelles: What surprises me the most, that source and target goid were the same, but graphman decided to insert the 2nd resource, that of course lead to the constraint violation, as the "new" goid already exists.

Which leads me to the question, how the lookup logic looks like, to decide if a resource needs to be inserted or updated?

Would the removal of the source goid/guid be a general approach, to have a more reliable lookup processing?

I assume that then the  lookup will only be done on the unique "symbol" attribute. e.g. like resolutionPath for api's.

@Raju Gurram Yes you are right, we are not yet on V11 CR02 but still plain V11 and using graphman-client V1.0.01.

Thanks for the hint. Just another reason for being up-to-date ;)

I am looking into all this, as we are on the way to automate deployments, with  the help of GitLab and Jenkins.

A special scenario might be , that we are looking at "incremental" deployments only on VMware based gw's, not "all in one" ones.

Meaning , we are selecting  explicit api's & policies to deploy, not the whole gw config at once.

That leads to some more complex situations and questions, especially in regard to rollback and error handling.

Kind regards

...Michael

[Raju Gurram]

You assumption about the lookup is correct. Graphman checks the existence of a given entity by non goid unique property (i.e., resolutionPath for WebApi). For more info, please check this out: https://techdocs.broadcom.com/us/en/ca-enterprise-software/layer7-api-management/api-gateway/11-0/apis-and-toolkits/graphman-management-api.html#concept.dita_d3303fde-e786-4fd4-b0b6-e3a28fd60a82_entities

Table from the above page lists the identity fields for the supported entities.

Most likely, goid matches between source and target and the difference in their identity fields happen if we author the entities manually. Especially, when we duplicate the existing entity manually and changed the fields but not goid/guid's. 

[Michael Mueller]

Thanks a lot for all your input.

It helps to grow my understanding.

I am not able to reproduce the issue anymore, so , no further investigations possible and needed !

Again, I appreciate your support, always!

Kind Regards

...Michael