Layer 7 API Management

Expand all | Collapse all

Streaming behavior of API Gateway

  • 1.  Streaming behavior of API Gateway

    Posted 02-06-2019 08:27 AM

    During troubleshooting and research the forum for different topics, I found out the last days that my knowledge in regards to streaming and the way the API Gateway handles requests and responses by default or depending on different settings or used assertions is not optimal. Also the existing information on docops as well as here in the forum is not perfect. Therefor I'd like to summarize my current understanding and verify if this is correct and I also have some open questions, which can hopefully be answered.

    Current understanding

    • Requests will be loaded into memory by default
    • Responses will be streamed by default
    • For both requests and responses, in case the payload needs to be analyzed, adjusted or whatever, it will/must be loaded into memory
    • In regards to message size limit, this is only affective if the request or response will be loaded into memory
    • Streamed requests or responses will not be affected by any size limits

     

    Open questions

    • Assuming I don't need to touch the payload, how can I guarantee that also the request will be streamed? I made some local tests with the following result:
      • Using just a "Return Template Response"-assertion (so NO route assertion in use at all), the request seems to be streamed, because I don't get any message size limit error with a request-size of 3MB.
      • Using the "Copy Request Message to Response"-assertion will fail with a request-size of 3MB, but it is successful when adding the "Configure Message Streaming (enable Streaming)"-assertion.
      • Using the "Route via HTTP(S)"-assertion will always fail, doesn't matter if I use the "Configure Message Streaming"-assertion or not.
    • Based on these tests:
      • Is it important where the "Configure Message Streaming"-assertion is placed in the policy?
      • Is it depending how the backend in the "Route via HTTP(S)"-assertion handles the transfer of the large payload?
    • I found the CWP "attachment.diskThreshold" (default 1MB), but when is this affective?
      • Also in streaming mode or only when the request/response will be loaded into memory?
      • Only for real attachments?
      • Or for the size of the full request/response (even without attachments)?
    • Am I right that the main difference between SOAP and REST policies is the verification of the request against the configured WSDL-file and therefor always need to be loaded into memory? So assuming I don't need to offload these checks from the backend, I can handle a SOAP API via a "REST"-policy without being buffered, right?

     

    Ok, should be enough for the moment. I think there are maybe still some questions left, especially based on your answers, but these can be discussed later.

    So first I would be happy to get most of my above mentioned points being confirmed and answered.

    Thank you!

     

    Ciao Stefan



  • 2.  Re: Streaming behavior of API Gateway

    Posted 02-08-2019 09:38 PM

    Stefan,

     

    Definitely a few items to look through. I will see what I can do to answer your questions.

    Current understanding

    • Requests will be loaded into memory by default
    • Responses will be streamed by default
    • For both requests and responses, in case the payload needs to be analyzed, adjusted or whatever, it will/must be loaded into memory
    • In regards to message size limit, this is only affective if the request or response will be loaded into memory
    • Streamed requests or responses will not be affected by any size limits

    Response: The response payload by default will steam only if an assertion after the HTTP Routing assertion does not touch the body of the message including Audit with save request/response enabled. The other point appear to be on target.

     

    Open questions

    • Assuming I don't need to touch the payload, how can I guarantee that also the request will be streamed? I made some local tests with the following result:
      • Using just a "Return Template Response"-assertion (so NO route assertion in use at all), the request seems to be streamed, because I don't get any message size limit error with a request-size of 3MB.
      • Using the "Copy Request Message to Response"-assertion will fail with a request-size of 3MB, but it is successful when adding the "Configure Message Streaming (enable Streaming)"-assertion.
      • Using the "Route via HTTP(S)"-assertion will always fail, doesn't matter if I use the "Configure Message Streaming"-assertion or not.
    • Based on these tests:
      • Is it important where the "Configure Message Streaming"-assertion is placed in the policy? 

    Response: To stream inbound the Message Streaming assertion should be at the beginning of the policy and for response message it is not needed if no assertion touches the payload. 

     

    • Is it depending how the backend in the "Route via HTTP(S)"-assertion handles the transfer of the large payload?

    Response: If it is chunked encoded that analyzing the payload would be the only way to get the size. Otherwise the size of the payload or the type is unimportant to stream it back only the modification or review of the payload will affect this.

    • I found the CWP "attachment.diskThreshold" (default 1MB), but when is this affective?
      • Also in streaming mode or only when the request/response will be loaded into memory?

    Response: Only loaded into memory

    • Only for real attachments?

    Response: The whole message will be sent to disk if it exceeds the size outlined in the diskthreshold CWP. (Configure Message Streaming Assertion - CA API Gateway - 9.3 - CA Technologies Documentation )

    • Or for the size of the full request/response (even without attachments)?

    Response: Same as above

    • Am I right that the main difference between SOAP and REST policies is the verification of the request against the configured WSDL-file and therefor always need to be loaded into memory? So assuming I don't need to offload these checks from the backend, I can handle a SOAP API via a "REST"-policy without being buffered, right?

    Response: There is the validation against the WSDL and also WS-Security processing as outlined below from the Service Properties. you can covert any SOAP API to a Web API and just lose some of the SOAP based assertions. It may be worthwhile noting that response streaming works the same for both SOAP and REST services. 

     

    Sincerely,

     

    Stephen Hughes

    Broadcom Support



  • 3.  Re: Streaming behavior of API Gateway

    Posted 02-15-2019 11:02 AM

    Hi Stephen,

    thank you for your answer, which at least proves most of my understanding.

    But I still have a question/issue with streaming the request. I tested the "Configure Message Streaming"-Assertion in a test-policy, but still getting the message size limit error. Is this maybe related to the routing and/or behavior of the backend itself? I send a POST-request with a 3MB payload to a public API designed for GET-request, but answers with a error message for POST-requests. Is this error message related to this? And if yes, can I somehow guarantee that using this assertion don't break anything else? We are thinking about to use request streaming as some kind of standard for most of our policies.

     

    And one last question: why are requests loaded into memory by default? What's the technical reason behind that?

    Thank you!

     

    Ciao Stefan



  • 4.  Re: Streaming behavior of API Gateway

    Posted 02-15-2019 02:21 PM

    Stefan,

     

    As we are not a pass through proxy, we have always cached the inbound requests by default and added in streaming for inbound as time went on. Designed decision from the beginning of the product development. 

     

    For the issue that you outlined, I've tested with a simple policy that has just a Streaming for Inbound requests and the HTTP Routing assertion. If I force the back end to not support POST then I get the following error:

     

    <l7:assertionResult
    assertion="l7p:HttpRoutingAssertion" status="Error in Assertion Processing">
    <l7:detailMessage id="4042">Problem routing to http://hugst05-ssg94.ca.com:8080/echo. Error msg: Unable to obtain HTTP response from http://hugst05-ssg94.ca.com:8080/echo: Cannot retry request with a non-repeatable request entity. The cause lists the reason the original request failed. Caused by: Broken pipe (Write failed). </l7:detailMessage>
    </l7:assertionResult>

     

    Sincerely,

     

    Stephen Hughes

    Broadcom Support



  • 5.  Re: Streaming behavior of API Gateway

    Posted 02-15-2019 03:00 PM

    Hi Stephen,

    yes, such a similar error message I'm getting as well.

    Cannot retry request with a non-repeatable request entity. The cause lists the reason the original request failed. Caused by: Unable to read stream: the specified maximum data size limit would be exceeded.

    So what does this mean exactly?

    Thank you!

     

    Ciao Stefan



  • 6.  Re: Streaming behavior of API Gateway

    Posted 02-19-2019 02:19 PM

    Stefan,

     

    The meaning of this message is that the back end does not support the method in question so it can not be streamed resulting in the message being put into memory for a retry. If you change the Maximum Retries on the Connection tab of the HTTP Routing assertion then the same error will not appear.

     

    Sincerely,

     

    Stephen Hughes

    Broadcom Support