Message size limit

[Philippe Brand]

Hi,

 

We're experiencing some troubles finding out how we can limit on a per API basis, a message size limit.

We ran a few tests and you'll find below results.

Is this what is expected and above all is this behavior likely to change in a future version ?

We are implementing security policies for each project to use as per policy header, somewhat a mandatory policy, that will indeed include message size limitation based on project's inputs.

 

Test case n°	Listen port max size	io.xmlPartMaxBytes	limit message size	message accepted	Reason	Important point	Comment
1	Disabled	Disabled	Disabled	OK
2			2000Kb	OK
3			500Kb	KO	Assertion falsified
4		2.000.000 b	Disabled	OK
5			2000Kb	OK
6			500Kb	KO	Assertion falsified
7		500.000 b	Disabled	KO	Unable to read stream
8			2000Kb	KO	Unable to read stream	X	we expected that the assertion overrides the CWP
9			500Kb	KO	Unable to read stream	X	we expected that the assertion overrides the CWP
10	2.000.000 b	Disabled	Disabled	OK
11			2000Kb	OK
12			500Kb	KO	Assertion falsified
13		2.000.000 b	Disabled	OK
14			2000Kb	OK
15			500Kb	KO	Assertion falsified
16		500.000 b	Disabled	OK		X	io.xmlMaxPartBytes is ignored
17			2000Kb	OK		X	io.xmlMaxPartBytes is ignored
18			500Kb	KO	Assertion falsified
19	500.000 b	Disabled	Disabled	KO	Unable to read stream
20			2000Kb	KO	Unable to read stream
21			500Kb	KO	Unable to read stream
22		2.000.000 b	Disabled	KO	Unable to read stream
23			2000Kb	KO	Unable to read stream
24			500Kb	KO	Unable to read stream
25		500.000 b	Disabled	KO	Unable to read stream
26			2000Kb	KO	Unable to read stream
27			500Kb	KO	Unable to read stream

test message size

1.038.762 bytes

1.038 Kbytes

[Zhijun He]

Hello PhBrand,

I'm not sure if I understand you question, but if the size limit is base on policy, you can use "Limit Message Size" assertion.

 

Regards,

Mark

[Philippe Brand]

We found out "Limit Message Size" assertion can allow a limit greater than theone specified by CWP io.xmlPartMaxBytes *only* if Listen Port limit is set with "Override Maximum Message Size" *and* being greater than CWP and the assertion 's limit.

[Azad Ellini]

Hi @PhBrand,

 

The difference in behavior is due to a setting in the Service Properties called "Perform WS-Security processing for this service". When that checkbox is checked, it results in the message being buffered before it reaches policy, and some processing of the message is performed before reaching the service policy as well. Therefore the message is checked against the cluster-wide property (io.xmlPartMaxBytes) first, before it reaches policy. If the message passes the cluster-wide property, then it makes it to the policy, and is checked against the Limit Message Size assertion (if it exists). If the message fails the check against the cluster-wide property to begin with, it never makes it to policy, regardless of the value of the "Limit Message Size" assertion.

 

If the checkbox is unchecked, then the message goes to the service policy directly, and the message is compared against the Limit Message Size assertion if it exists, and if the assertion doesn't exist, will be compared to the cluster-wide property.

 

The difference between REST vs SOAP in this instance is that SOAP services by default have this checkbox checked, whereas REST services have this checkbox unchecked. If you uncheck this value from a SOAP service, it will behave like a default REST service. If you check this value in a REST service, it will behave like a default SOAP service.

 

So to simplify it, a request to a REST service or SOAP service with WS-Security processing enabled, is buffered and therefore checked against the Cluster-Wide Property first. If it exceeds the limit, you receive "Error: Unable to read stream: the specified maximum data size limit would be exceeded". If it is under the limit, the message continues to the policy, and processed accordingly.

 

The REST service and SOAP service with WS-Security processing disabled, goes to the policy first, and ONLY adheres to the Limit Message Size assertion (if it exists) and ignores the cluster-wide property. If the Limit Message Size assertion doesn't exist in the policy, then it adheres to the cluster-wide property.

 

I hope that clarifies things for you. Let me know if you have any additional questions on this matter.

Regards,

Azad

[Stefan Klotz]

Hi Azad,

thank you very much for this very detailed explanation. I also have an issue related to the message size and was very happy to read this. But when I disable the WS-Security processing (for a SOAP service) and insert the "Limit Message Size" assertion to the policy, I'm still getting the error message "Error: Unable to read stream: the specified maximum data size limit would be exceeded". Do you have an explanation for this as well? Did I miss something?

Thank you!

 

Ciao Stefan

[Azad Ellini]

Stefan,

 

I just ran a few more tests in my lab and it looks like the behavior differs even further when the option of "Allow requests intended for operations not supported by the WSDL" is checked or not. This option is under the Service Properties, and under the WSDL tab. It seems that if this option is unchecked, that it will adhere to the cluster-wide property regardless of the WS-Security setting being checked or unchecked. But if the "Allow requests..." option is checked, then it will behave as I had originally outlined.

 

The information posted previously is a bit generalized, there are other factors involved, and it revolves around whether the request gets buffered (for whatever action) prior to service resolution. The options discussed are just the most common scenarios and options that may or may not buffer a request.

 

I hope this helps.

 

Regards,

Azad

[Stefan Klotz]

Hi all,

can someone provide some usefull information, how a larger allowed message size (per policy, not as CWP) can/will impact the performance of the gateway? What will happen, if I allow e.g. 20MB for a policy instead of the default 2,6MB from the CWP? Does it require more RAM for processing these requests? Is it depending on the amount of such big request e.g. 1 request with more than 10MB per hour vs, 10 requests per second?
Thank you for your help or any provided Information!

 

Ciao Stefan

[DustinDauncey]

Hi Stefan,

 

My apologies for the very delayed response to your last question here from January.

 

Unfortunately, it's difficult to say exactly how the Gateway will perform if the message size limit is higher, as there are many factors involved in the performance of the Gateway. But in general, if you are receiving messages with larger sizes, then yes more memory will be needed. However, simply setting the limit at 20 MB for example does not automatically reserve that memory, meaning if your actual messages are still just 1 MB, it won't need to use any more memory than it was already using if the usual size has been 1 MB. In other words, changing the limit should not automatically affect performance, unless the messages are actually utilizing that extra ceiling space.

 

I hope that helps.