Websocket Client for a mobile game in Godot. Creating a protocol to understand byte array being passed.

Currently I have to implement a Websocket client for a Godot mobile game.

What was really fun about it was deciding on the data transfer specifications. We want the data to be encoded and compressed on server side. We also want the client to decompress and decode the received data.

This sounds fairly simple but..there’s a small issue (it always is 😀 ).

The data is passed as a byte array (in Godot a PoolByteArray). To decompress it on the client side we need one piece of vital info: the size of the uncompressed original data.

So when the game received this compressed/encoded data, it scratches its head. It can’t do anything with it, it needs the size of the uncompressed data to decode it.

We can’t put this information in the original data sent from the server. That data is going to be already compressed – to decompress it we need the data inside the compressed data.
It’s like having the password for a .zip file inside the .zip file.

What we need to do is have the password outside the .zip file. In our case, we need the size of the uncompressed data outside of the compressed data, in an uncompressed format.

So if the size of the uncompressed size is 42.

Converted to string we get “42”. “4” converted to byte is 0x52. “2” converted to byte is 0x50. The byte array describing the size is thus: [0x52,0x50]. When the client gets this data, it will convert the byte array to a string representation first: “42”. Then it will try and parse it into an int, getting 42.

Where to put it? How to send it?

Here’s how I ended up doing this:

On server side I do the usual thing with the data.
1. Encode it with utf8.

2. Compress it.

So far so good. I end up with a byte array.
Now I need to attach to this byte array the information about the size of the uncompressed data.

I could just append it to the end. But how will the client know what is the compressed data and what is the information about the size of the uncompressed data? All it sees is a byte array, it doesn’t know where one ends and another begins.

One solution would be to decide on a “header” byte sub-array of fixed size. If the maximum size of the data is going to be 4096, I’ll need at least a header of 4 bytes in size (each byte will hold one number).

But what if the size is going to be smaller than 4096? Say 5. Then I’ll have to pad the remaining bytes with either “0” char or the nullbyte.
This presents 2 issues: one, it requires more code&work on the client side. Two – it wastes bandwith since all we need is just 1 byte of info (and we’ll send 3 extra “empty” bytes).

I chose another more elegant solution that also saved bandwith.

By convention, we’ll mark the end of compressed data with a nullbyte (0x00). After this nullbyte we put our uncompressed data size information in byte array [0x52,0x50]. Assuming the compressed data looks like: [0x01,0x05,0x99], by appending this nullbyte marker and the uncompressed size we’ll get:
[0x01,0x05,0x99,0x00,0x52,0x50]

The client will then start looking for a nullbyte from the end of the byte array (because nullbytes might be present in the compressed data itself). Once it finds than nullbyte, it will extract the sub array from the nullbyte location to the end of the byte array. In our case it sees 0x00 at index 3. It knows that from index 4 (inclusive) to the end of the array is the info about the uncompressed size.

Now the client has [0x52,0x50] byte array. Converted to string it gets “42”. Converted to int it gets 42. Now it can use this info to decompress the compressed data [0x01,0x05,0x99] (dummy example, not actual compressed data).

Of course, we perform all kinds of checks on the client side to avoid out of range errors and other data issues.

There might be smarter ways of passing along this information. But for my use case it works great!

Thank you for reading!

[EDIT] Some friends told me there might be better solutions for this issue. Mainly about prefixing the data, using htonl and possibly a header. I’ll keep you updated with how things ended up (I’m creating this backend server as a mock server so I can finish the client code. The actual backend server will be implemented by a backend developer).

Leave a Reply

Your email address will not be published.