Layer 7 API Management

Expand all | Collapse all

Message size limit

Jump to Best Answer
  • 1.  Message size limit

    Posted 12-07-2016 04:58 AM

    Hi,

     

    We're experiencing some troubles finding out how we can limit on a per API basis, a message size limit.

    We ran a few tests and you'll find below results.

    Is this what is expected and above all is this behavior likely to change in a future version ?

    We are implementing security policies for each project to use as per policy header, somewhat a mandatory policy, that will indeed include message size limitation based on project's inputs.

     

    Test case n°Listen port max sizeio.xmlPartMaxByteslimit message sizemessage acceptedReasonImportant point Comment
    1DisabledDisabledDisabledOK   
    22000KbOK   
    3500KbKOAssertion falsified  
    42.000.000 bDisabledOK   
    52000KbOK   
    6500KbKOAssertion falsified  
    7500.000 bDisabledKOUnable to read stream  
    82000KbKOUnable to read streamXwe expected that the assertion overrides the CWP
    9500KbKOUnable to read streamXwe expected that the assertion overrides the CWP
    102.000.000 bDisabledDisabledOK   
    112000KbOK   
    12500KbKOAssertion falsified  
    132.000.000 bDisabledOK   
    142000KbOK   
    15500KbKOAssertion falsified  
    16500.000 bDisabledOK Xio.xmlMaxPartBytes is ignored
    172000KbOK Xio.xmlMaxPartBytes is ignored
    18500KbKOAssertion falsified  
    19500.000 bDisabledDisabledKOUnable to read stream  
    202000KbKOUnable to read stream  
    21500KbKOUnable to read stream  
    222.000.000 bDisabledKOUnable to read stream  
    232000KbKOUnable to read stream  
    24500KbKOUnable to read stream  
    25500.000 bDisabledKOUnable to read stream  
    262000KbKOUnable to read stream  
    27500KbKOUnable to read stream  
    test message size1.038.762 bytes1.038 Kbytes


  • 2.  Re: Message size limit

    Posted 12-07-2016 09:24 PM

    Hello PhBrand,

    I'm not sure if I understand you question, but if the size limit is base on policy, you can use "Limit Message Size" assertion.

     

    Regards,

    Mark



  • 3.  Re: Message size limit

    Posted 12-08-2016 05:57 AM

    We found out "Limit Message Size" assertion can allow a limit greater than theone specified by CWP io.xmlPartMaxBytes *only* if Listen Port limit is set with "Override Maximum Message Size" *and* being greater than CWP and the assertion 's limit.



  • 4.  Re: Message size limit
    Best Answer

    Posted 12-08-2016 10:48 AM

    Hi @PhBrand,

     

    The difference in behavior is due to a setting in the Service Properties called "Perform WS-Security processing for this service". When that checkbox is checked, it results in the message being buffered before it reaches policy, and some processing of the message is performed before reaching the service policy as well. Therefore the message is checked against the cluster-wide property (io.xmlPartMaxBytes) first, before it reaches policy. If the message passes the cluster-wide property, then it makes it to the policy, and is checked against the Limit Message Size assertion (if it exists). If the message fails the check against the cluster-wide property to begin with, it never makes it to policy, regardless of the value of the "Limit Message Size" assertion.

     

    If the checkbox is unchecked, then the message goes to the service policy directly, and the message is compared against the Limit Message Size assertion if it exists, and if the assertion doesn't exist, will be compared to the cluster-wide property.

     

    The difference between REST vs SOAP in this instance is that SOAP services by default have this checkbox checked, whereas REST services have this checkbox unchecked. If you uncheck this value from a SOAP service, it will behave like a default REST service. If you check this value in a REST service, it will behave like a default SOAP service.

     

    So to simplify it, a request to a REST service or SOAP service with WS-Security processing enabled, is buffered and therefore checked against the Cluster-Wide Property first. If it exceeds the limit, you receive "Error: Unable to read stream: the specified maximum data size limit would be exceeded". If it is under the limit, the message continues to the policy, and processed accordingly.

     

    The REST service and SOAP service with WS-Security processing disabled, goes to the policy first, and ONLY adheres to the Limit Message Size assertion (if it exists) and ignores the cluster-wide property. If the Limit Message Size assertion doesn't exist in the policy, then it adheres to the cluster-wide property.

     

    I hope that clarifies things for you. Let me know if you have any additional questions on this matter.

    Regards,

    Azad



  • 5.  Re: Message size limit

    Posted 11-08-2017 03:23 AM

    Hi Azad,

    thank you very much for this very detailed explanation. I also have an issue related to the message size and was very happy to read this. But when I disable the WS-Security processing (for a SOAP service) and insert the "Limit Message Size" assertion to the policy, I'm still getting the error message "Error: Unable to read stream: the specified maximum data size limit would be exceeded". Do you have an explanation for this as well? Did I miss something?

    Thank you!

     

    Ciao Stefan



  • 6.  Re: Message size limit

    Posted 11-08-2017 04:16 PM

    Stefan,

     

    I just ran a few more tests in my lab and it looks like the behavior differs even further when the option of "Allow requests intended for operations not supported by the WSDL" is checked or not. This option is under the Service Properties, and under the WSDL tab. It seems that if this option is unchecked, that it will adhere to the cluster-wide property regardless of the WS-Security setting being checked or unchecked. But if the "Allow requests..." option is checked, then it will behave as I had originally outlined.

     

    The information posted previously is a bit generalized, there are other factors involved, and it revolves around whether the request gets buffered (for whatever action) prior to service resolution. The options discussed are just the most common scenarios and options that may or may not buffer a request.

     

    I hope this helps.

     

    Regards,

    Azad



  • 7.  Re: Message size limit

    Posted 01-17-2018 05:17 AM

    Hi all,

    can someone provide some usefull information, how a larger allowed message size (per policy, not as CWP) can/will impact the performance of the gateway? What will happen, if I allow e.g. 20MB for a policy instead of the default 2,6MB from the CWP? Does it require more RAM for processing these requests? Is it depending on the amount of such big request e.g. 1 request with more than 10MB per hour vs, 10 requests per second?
    Thank you for your help or any provided Information!

     

    Ciao Stefan



  • 8.  Re: Message size limit

    Posted 08-10-2018 09:01 PM

    Hi Stefan,

     

    My apologies for the very delayed response to your last question here from January.

     

    Unfortunately, it's difficult to say exactly how the Gateway will perform if the message size limit is higher, as there are many factors involved in the performance of the Gateway. But in general, if you are receiving messages with larger sizes, then yes more memory will be needed. However, simply setting the limit at 20 MB for example does not automatically reserve that memory, meaning if your actual messages are still just 1 MB, it won't need to use any more memory than it was already using if the usual size has been 1 MB. In other words, changing the limit should not automatically affect performance, unless the messages are actually utilizing that extra ceiling space.

     

    I hope that helps.