Question about Content-Encoding always gzip

[Jon Schendt]

I'm seeing some strange behavior that a customer recently brought to my attention.

When browsing to a file not going through the Gateway, I get the file without any encoding (no "Content-Encoding" header in the response).  When I navigate to that same file going through the Gateway, I get "Content-Encoding: gzip").  I've confirmed that the back-end server isn't compressing that output, and I've modified the "Accept-Encoding" request header to omit gzip right before the request gets sent out.  My customer doesn't want any additional compression, and I'm a little stumped.  Any ideas?

[Joseph Fry]

Jon,

I may be way off base here, but content-encoding is largely controlled by the requester using a Accept-Encoding header.  I suspect that the backend system does not support encoding and thus when you go direct you don't get gzip data back.

However presumably, the gateway does support gzip and therefore provides the response using the encoding that your client accepts.  Useful if the backend system isn't HTTP for example, but perhaps not what you would expect when it is HTTP.

There is a cluster property response.compress.gzip.allow that allows you to configure the gateway to not compress responses... but its not a  don't-compress-the-response-if-the-response-to-a-route-assertion-wasnt-compressed kinda switch.

I suspect that there is a way to selectively send an uncompressed response to a requester that has it in their Accept-Encoding header, but I don't know it off hand.  Might be as simple as stripping the Accept-Encoding header off of the request  before the response is sent?

But ultimately, I believe that the right answer is to have your customer not send an Accept-Encoding header in the first place?

[Jon Schendt]

Joe, I saw this after I responded a couple of minutes ago.  I did modify the response.compress.gzip.allow header to disable gzip compression, and that did seem to work (Content-Length isn't getting calculated or transferred now, but that might be something else.)

Do you know if there's a way to only disable gzip compression on certain service endpoints?  I want to keep the performance of gzip compression on large datasets, but this specific endpoint is already sending compressed data, so there's no need to compress it again.

Thanks!

[Joseph Fry]

Of course, I hit send and look at something I had just read and realize that I misunderstood... if you leave off the Accept-Encoding header, than you are saying that any encoding is acceptable.  So your customer needs to include an "Accept-Encoding: Identity" to tell the server (the gateway in this case) not to encode the data.

Per RFC: https://datatracker.ietf.org/doc/html/rfc7231#section-5.3.4

 A request without an Accept-Encoding header field implies that the
   user agent has no preferences regarding content-codings.  Although
   this allows the server to use any content-coding in a response, it
   does not imply that the user agent will be able to correctly process
   all encodings. 

So it appears that if the client (user agent) doesn't specify, the server (gateway) is free to choose.

Different browsers have different defaults... I found this in the comments section of a site from 2018, so its probably out of date but you get the point:

• Chrome: Accept-Encoding: identity;q=1, *;q=0.
  Safari: Accept-Encoding: identity.
  Firefox: No Accept-Encoding header.
  Edge: Accept-Encoding: gzip, deflate, br.

Interesting rabbit hole you sent me down.  But I am still 99% sure that the RIGHT answer is to have the client include a "Accept-Encoding: identity" header.

[Jon Schendt]

Joe, thank you for the responses! I didn't mention it, but I did also set the Accept-Encoding to identity, and got the same output. I'll see if I can replicate it on a different system and upload some screenshots.

Get Outlook for Android<https: aka.ms aab9ysg>

[Jon Schendt]

Joe, I was able to recreate the issue I'm seeing on my home network.  Please see attached screenshot.

If I navigate through the Gateway to the same webserver, I get a Content-Length of 2886bytes, and Content-Encoding of gzip. If I navigate directly to the webserver, I get a Content-Length of 11130 and no Content-Encoding.  My policy explicitly sets accept-encoding to identity and doesn't compress output.

[Ben Urbanski]

Hello, Jon. If you like, you may open a support case for this, and we'll have a look.

------------------------------
Ben Urbanski
Product Manager, API Gateway
Layer7 API Management
------------------------------