Plex 2E

View Only

Plex Size restriction for Plex models

Klaus Lichte posted Mar 05, 2021 11:56 AM

Hello Plexperts,

we have been working with our Plex models for more than 20 years. So far there have been no major problems or failures, but the models have steadily increased in size over time. Has anyone had experience with the fact that there is an upper limit on the size of Plex models. Maybe there is a limit to the number of triples or the number of surrogates is limited?

Thanks in advance
Klaus

AnetteNicole Jahn posted Mar 08, 2021 03:38 AM

Hi Klaus,

There is some information available in the Plex Online Help in the Group Model Integrity Checks chapter, topic High Surrogates Checks . The limit is in fact imposed by the number of possible surrogates. As the online help mentions “there have been isolated cases of CA Plex models approaching the limit of available surrogates due to bugs in the surrogate allocation process”.

In all those years, we had one customer who run into problems due to surrogate numbers no longer being available. He used the steps also described in the Online Help in chapter Alternate Group Model Surrogate Assignment. So the best would be to read the whole mentioned Group Model Integrity Checks chapter in the Online Help to get further information.

Viele Grüße/Best regards

Anette-Nicole

AnetteNicole Jahn posted Mar 08, 2021 03:40 AM

Hi Klaus,

There is some information available in the Plex Online Help in the Group Model Integrity Checks chapter, topic High Surrogates Checks . The limit is in fact imposed by the number of possible surrogates. As the online help mentions “there have been isolated cases of CA Plex models approaching the limit of available surrogates due to bugs in the surrogate allocation process”.

In all those years we had one customer who run into problems due to surrogate numbers no longer being available. He used the tips also described in the Online Help in chapter Alternate Group Model Surrogate Assignment. So the best would be to read the whole mentioned Group Model Integrity Checks chapter in the Online Help to get further information.

Viele Grüße/Best regards

Anette-Nicole

Jorge Ubeda posted Mar 08, 2021 03:44 AM

I can't respond your direct question, because there is no information on such limit, as long I know. However, I would say that, being Plex a 32 bit architecture, its technical limit would be a signed 2 GB limit. We have large models but hardly greater than 200 MB. Which is the size of yours? It is a unique model?

AnetteNicole Jahn posted Mar 08, 2021 03:46 AM

Hi Klaus,

Viele Grüße/Best regards

Anette-Nicole

AnetteNicole Jahn posted Mar 08, 2021 03:55 AM

Hi Klaus,

Viele Grüße/Best regards

Anette-Nicole

AnetteNicole Jahn posted Mar 08, 2021 04:07 AM

Hi Klaus,

In all those years we had one customer who run into problems due to surrogate numbers no longer being available. He used the steps also described in the Online Help in chapter Alternate Group Model Surrogate Assignment. So the best would be to read the whole mentioned Group Model Integrity Checks chapter in the Online Help to get further Information.

Viele Grüße/Best regards

Anette-Nicole

AnetteNicole Jahn posted Mar 08, 2021 04:31 AM

Hi Klaus,

There is some information available in the Plex Online Help in the Group Model Integrity Checks chapter, topic High Surrogates Checks . The limit is in fact imposed by the number of possible surrogates. As the online help mentions “there have been isolated cases of CA Plex models approaching the limit of available surrogates due to bugs in the surrogate allocation process”.

In all those years we had one customer who run into problems due to surrogate numbers no longer being available. He used the steps also described in the Online Help in chapter Alternate Group Model Surrogate Assignment. So the best would be to read the whole mentioned Group Model Integrity Checks chapter in the Online Help to get further information.

Viele Grüße/ Best regards

Anette-Nicole

AnetteNicole Jahn posted Mar 08, 2021 04:37 AM

Hi Klaus,

There is some information available in the Plex Online Help in the Group Model Integrity Checks chapter, topic High Surrogates Checks . The limit is in fact imposed by the number of possible surrogates. As the online help mentions “there have been isolated cases of CA Plex models approaching the limit of available surrogates due to bugs in the surrogate allocation process”.

Viele Grüße/Best regards

Anette-Nicole

AnetteNicole Jahn posted Mar 08, 2021 04:45 AM

Hi Klaus,

In all those years we had one customer who run into problems due to surrogate numbers no longer available. He used the steps also described in the Online Help in chapter Alternate Group Model Surrogate Assignment. So the best would be to read the whole mentioned Group Model Integrity Checks chapter in the Online Help to get further information.

Viele Grüße Best regards

Anette-Nicole

Klaus Lichte posted Mar 08, 2021 05:44 AM

Hello,

one of our largest local models is 560MB in size. If there is a maximum size of 2GB, we would still have some space left. It would be very helpful if Broadcom itself could make a clear statement on this.

In addition, I received the information that there is a limitation by the surrogates. This is described in the Plex Help under the topic "Group Model Integrity Checks". Here, however, the question arises as to how one can determine the current number of triples and objects.

Jorge Ubeda posted Mar 08, 2021 06:03 AM

Hi Klaus
It would be very helpful if Broadcom itself could make a clear statement on this
I agree with you

the question arises as to how one can determine the current number of triples and objects
Have you tried the model api?...there are triple methods: get first, get next...

Klaus Lichte posted Mar 09, 2021 04:39 AM

Hello all,

Anette-Nicole, thank you very much for your answer. i can see that you tried to post the answer quite a number of times on this thread.

Jorge, we've also thought about using the Plex Model APIs, but we don't think that they can be used to get the necessary information easily and to the point.

Maybe we're lucky and Broadcom will post some Information on this thread.

Best Regards
Klaus

John Rhodes posted Mar 09, 2021 04:16 PM

Hi Klaus

I think Anette has provided some good information, and I don't have much to add on limits.

However, we have had a few customers that have run into a serious surrogate problem, due to minor corruption causing issues in surrogate allocation. If you do experience this problem, please be advised it will get worse progressively and potentially quickly, so it is important to contact Broadcom support to see if they can help before the problem gets worse. If they can't help, because the problem has progressed too far, we at CM First have developed an automated process to export/import large models in a way that will reallocate the surrogates. However, this process requires some manual work so it is not ideal. Happy to discuss if you need to explore this.

Good luck, John Rhodes

Klaus Lichte posted Mar 10, 2021 05:38 AM

Hello John,

Thank you very much for your response. We currently have no problems with the surrogates and my request is purely informative but may also be important.

I also think that Anette-Nicole and Jorge mentioned the two possible and important aspects of a limitation.

1. Is there actually a 2GB limit for all files that are created by Plex?
Since our largest model has reached a size of 560 MB after 21 years, this shouldn't be a problem anytime soon.
2. To what percentage are our models occupied, taking into account the upper limit of 16,000,000 triples and objects mentioned in the Plex Help.

We had both questions addressed to Broadcom. We hope that we will receive an answer on this.

Best Regards
Klaus

HIFUMI ONISHI posted Mar 12, 2021 02:58 AM

Hello Klaus,

Our customer model encountered a High Surrogates Checks error, so they were doing an Alternate Group Model Surrogate Assignment.
(High Surrogates Checks error was resolved using the surrogate reassignment method developed by CM First.)

A surrogate is required to store objects and triples. I think the upper limit mentioned in Plex Help is the upper limit of the number of surrogate that can be used.
If you want to know the number of surrogate remaining that the model can use, you can check it by the following method.

Confirmation method:
(1) Check the value in the rightmost column of the last row of the UPDATID.DAT file in the group model folder.

(2) Convert the value of (1) to a hexadecimal number.

(3) Since the first 1 byte of the value of (2) is the group model ID, the remaining 3 bytes are converted to decimal numbers.

(4) From the maximum value FF FF FF = 16777215, subtract the value of (3).

The value in (4) is the remaining number of surrogate that can be used.

* Please consider the numerical value obtained by this calculation as a guide only.
This is not the case if you use renum.num or if you have surrogate gaps.

e.g.:
(1) The number in the rightmost column of the last row of the UPDATID.DAT file is "919189802".

(2) Convert "919189802" to hexadecimal
→ 36C9B92A

(3) Convert the group model ID of "36C9B92A" other than "36" to decimal.
→ 13220138

(4) From the maximum value FF FF FF = 16777215, minus the value "13220138" in (3).
→ 3557077

The remaining number of surrogate that can be used is 3,557,077.

Regards,
Hifumi

Klaus Lichte posted Mar 12, 2021 10:38 AM

Hello Hifumi,

Thank you very much for your response as well.

The process for determining the assigned surrogates looks very much like the fact that there is very good inside knowledge. I did the calculation for our sensitive model. So we still have space for more than 11 million additional surrogates.

Rightmost column of the last row of the UPDATID.DAT = 206575828

206575828   hex >   C5018D4
5018D4        dec >    5.249.236
                                  -------
16777215 - 5249236 = 11.527.979 remaining (quite a lot)

Thank you very much again. Best regards
Klaus

John Rhodes posted Mar 12, 2021 10:53 AM

Hi Klaus -

Sounds like you have enough surrogates for a while. Just want to be clear, if you get the "negative surrogate" error on group model update, you should take that seriously as that situation will consume your available surrogates at fast rate.

-John

Klaus Lichte posted Mar 12, 2021 12:09 PM

Hi John,

Thank you for bringing this to our attention again.

When we extract from one of our Group models we have sometimes a message about 2 or 10 negative surrogates and sometimes no message.

When this happend for the first time a few years ago, we contacted our plex distributor and they told us as long as the number does not increase significantly, that is not a Problem at all.

But it is very strange, however, that not all developers receive this message. I just extracted the model and got the above message. A colleague extracted the model 5 minutes later and did not get this error message. I then extracted again and received the message again. The buttons for displaying system errors, errors and warnings in the message log are active on both PC's. That's the way it is.

A nice Weekend for everyone

Klaus

Klaus Lichte posted Mar 22, 2021 12:08 PM

Hello to all,

We received also a statement from Broadcom regarding the filesize limitation:

Answer: "Plex IDE is a 32-bit application, and the application runtime memory footprint can go up to 4 GB of memory.Opening a file (.mdl, group model .dat files) has no limit but due to Windows 32 bit API (fseek, ftell) for file operations like saving, loading specific object from memory can go up to 2 GB file size, not beyond that.So within-group model, any single file exceeding the size of 2 GB has this limitation. Having multiple files below 2 GB will work fine. As of today, we didn’t hear from any user reaching a 2 GB file Limit."

So we should have all the limits of a Plex model.

Many thanks to everyone for the very good support

Klaus