Is it necessary to write a code for compressing and decompressing data which needs to be sent over a webservice?

Question

A Web Service or A Web API accepts some payload and returns response both in XML or JSON format.

Content-Encoding: gzip, deflate, br is used to Compress the notify the Web Server that how request and response is encoded.

Does a client App required to write a code for these type of compressions or decompression based on the content encoding specified?

After using fiddler API test client it is found that:

1. Fiddler encodes the request before sending it to the server if content-encoding header is specified.
2. Fiddler decodes the response after receiving it.

So Questions:

1. Does client App needs to encode / compress the request as request can be big size.
2. Does client App needs to decode / decompress the response after receiving it? or it can receive decoded response?
3. Does Compression / Decompression handled at server side?
4. If writing code is necessary to decompress and read the compressed encoded request-response then how to handle situations where multiple comma separated encodings are used, for ex: Content-Encoding: gzip, deflate, br; how to handle this in Client App as well as in Server Web App?
5. Does all existing API testing tools like postman or fiddler encodes or decodes the request and response contents if content-Encoding header is specified?
6. Is it possible to avoid writing code for this in our own client app and server web app?

Answer 1

Hi @Piyush Meshram ,

If the request is large, the client needs to be compressed. This can reduce the transmission time of the request between the client and the server.

Correspondingly, the client needs to decompress the compressed response after receiving it. But this does not require writing code, the browser can handle it automatically.

So in IIS, the simplest compression method is to use the compression module. After the client sends a request, IIS compresses the response and returns it to the client.

---

If the answer is helpful, please click "Accept Answer" and upvote it.

Note: Please follow the steps in our  documentation  to enable e-mail notifications if you want to receive the related email notification for this thread.

Best regards,
Bruce Zhang

Answer 2

You mixing two issues

>Does client App needs to encode / compress the request as request can be big size.

It needs to encode. Most (even asp.net) will provide automatic XML or jason, depending on the client side settings for "context" type of request.

But encoding has ZERO to do with compression!!!

You have to introduce your own compression code and library - and it rather pain full. So, server side, you have a number of compression routines you could use from the .net library. but client side?

No, nothing built in. That would mean you take your jason (or maybe xml), compress it, and then do your ajax post (web service call). On the other end, you would have to use and have a compatible compression routine. You are in effect ON YOUR own here.

I mean, if you going to pull a picture - then if it already png, or jpeg, then it is already HIGH compressed. So it don't matter.

Now I suppose for a large serialized object? Sure, it will use jason, and it not auto-matic compressed for you. The obvious solution is to try and keep such data transfers small. But the compression is not built in for you.

I had a case in which I was using a desktop program - and thus un-like JavaScript (client side) I did in fact zip up the data and on client side un-zipped the serlizedd ojbect. (xml, and even jason was really bloating up the size of the object).

So, compression <> encoding - two VERY different things and concepts.

>Does client App needs to decode

Yes, it needs to decode. jQuery seems able to decode jason, or xml. So yes this is done for you. Or in some cases you get a json string, and then in your code you can use JSON.Parse to create the json object.

>decompress the response

see above - decode/encode are not the same as compress/de-compress. No compression is not built in. If you were/are to introduce compression, then you need a compatible compression system on each end. So, maybe gzip, pkzip or some such - but you on your own in this area. (its not automatic, and not done for you).

>then how to handle situations where multiple comma separated encodings are used,

I not ware that multiple and different encodings are common or even ever used in one web service calls. I assume that the object in question (server side) is some object, it is serialized to json, and then on the other end (client side) you are happy with that json.

but, as noted, you do set the encoding used on the post request (the ajax call to the web site). The data returned is automatics encoded and converted for you into a serialized json object (or, as noted, the jQuery library seems to be happy with xml or jason. And in fact any public method on the asp.net web site marked as a web service? It works without any extra code to use xml, or json.

> decodes the request and response contents if content-Encoding header is specified?

Well, in most cases - it just plain text anyway. So, I am not 100% sure, but it really never mattered. It might, but the text is really just plain text in most cases anyway. So tossing in say utf-8 or some such tends to result in text that is still readable say in notepad or as plain text anyway.

What probably matters more is are you using say XML or json for the transfer. At one time XML + soap was really common, but now days a REST call, and using JSON is quite much used.

However, as stated, in asp.net? You mark a routine as webmethod, and it actually out of the box supports REST calls, json, and even supports SOAP + xml calls without any additional efforts on your part (but you have to tell the server which format to use with the correct headers). If you use some network looer tool, you see the raw formats - json or xml - so there not a lot of decoding when looking at the raw data - it still quite much plain text for the most part.

Edit:
But to answer your question? No, compression is not built in, nor is some standard avaible here. Browsers could in theory adopt a standard, but then all existing web servers would have to be updated and support as such.

I would be nice to have a standard. A noted, for things like pictures etc., they are already compressed, but for larger objects being sent back and forth? yes, it would be nice to have but we don't have this option right now. So, if you wanted some compression for that web service, you have to roll your own and adopt compression software on both server side, and client side.

Edit and follow up:

Now, as per the follow up and suggest by Bruce?

IIS server (internet services) DOES in fact have a compression system built in. And it seems that most browsers support this standard. You do have to enable this compression on the server (in the IIS configuration menus). So, it looks like a "general" or "built-in" compression option is avaible.

In fact, I am testing this right now (and I had to install + add the compression features to IIS on that server).

So, while as a general rule what I stated above? it looks like a compression option for the browser can be turned on - I don't know how wide the support for this is in browsers - but I am trying this as I write this!!

And this compression option would in general not require you to write client side code for this compression. I don't know, and have to check if the compression applies to web service calls - but I don't see why it would not.

Regards,
Albert D. Kallal (Access MVP 2003-2017)
Edmonton, Alberta Canada