Automic Workload Automation

Expand all | Collapse all

Maximum size of XML imports

  • 1.  Maximum size of XML imports

    Posted 03-19-2018 06:32 AM

    I just submitted a new idea:

    Increase maximum size of XML exports/imports from 30 MB to at least 100 MB 

    The maximum size of XML file imports is limited to 30 MB. (This can be further limited using the MAX_IMPORT_SIZE system setting.)
    The  maximum size should be increased, preferably to at least 100 MB.  Failing that, an automated mechanism for splitting XML exports into  multiple files should be provided.                    

    If you like the idea, please vote for it.



  • 2.  Maximum size of XML imports

    Posted 03-19-2018 07:24 AM
    seconded, but one remark:

    Maximum size of things that are achived with standard libraries should not be arbitarily limited by Automic. Maximum size should be dictated either by unavoidable limits in the library, or by available RAM, and at best lower limits should be configurable.

    Automic has a habit of limiting stuff arbitarily with hard limits (cough ... lines of JCL ... cough ... characters in print statements ... activity window cough) which each and every time ends in tears. Especially something as cleart cut as XML imports should be limited by available RAM, since it'll be quite easy to figure out why it takes long (and where to place plame) if someone actually attempts to import a yugetabyte *) of XML file.

    *) not an actual unit. but it's Donald-Trump-yuge.


  • 3.  Re: Maximum size of XML imports

    Posted 04-20-2018 05:36 AM

    As there will be always some limitations for RAM or file size, a splitting tool to create multiple XML files would be nice and doesn't need modifications on the current modules.

     

    Of course this tool should create files with coherent object definitions in each file, don't split simply physically the number of bytes !

     

    The size of the splitted files should also be a parameter, to enable quick load using multiple small files (multiple new application load i.e.) or a longer one for a few big files (mass update i.e.). It also can help to get file size usable by the AE Server regarding the availability of RAM (2 GB RAM is not 36 GB RAM when loading XML definitions !), workload on the server, etc ...

     

    This is something already managed by the compressing tools (Zip, 7Zip, ...) so technically this should not be a real issue to create the tool. Just need to include the concept of object definition to ensure that the splitted files are usable independently for loading.