Defer Unmarshalling in Go

In this blog, let us understand a unique problem of dynamic unmarshalling of JSON in golang. We will also look at where one might encounter it and how json.RawMessage helps us navigate it.

A common use-case for json unmarshalling is request/response parsing in HTTP. The client's deserialisation logic is straightforward since we have separate endpoints for separate json resources.

But in WebSocket world, we send and receive all sorts of different messages on the same WebSocket connection. And then the unique problem of intelligently deserialising these messages arises. You don't want to end up deserialising all the messages entirely every time !!! 🤯

That's messy! 🤢

Let's say as a WebSocket server we want to send 2 types of messages:

type MessageType_A struct {
    Name  string `json:"name"`
    Place string `json:"place"`
}

type MessageType_B struct {
    Animal string `json:"animal"`
    Thing  string `json:"thing"`
}

To send these messages we need to marshal these structs and write those bytes on the WebSocket connection.
The code will look something like this:

websocket_server_v1.go

    messageA := MessageTypeA { Name:  "Pankhudi", Place: "India"}
    // OR
    messageB := MessageTypeB { Animal: "dog", Thing:  "cap"}

    bytes, err := json.Marshal(messageA) // OR messageB 
    err = websocketConn.WriteMessage(websocket.TextMessage, bytes)

Now, the WebSocket client's deserializing logic is not aware beforehand of what type of message it is going to receive.

websocket_client_v1.go

    bytes, _, err := websocketConn.Read()
    if err != nil {...}

    messageA := MessageTypeA{}
    // OR
    messageB := MessageTypeB{}

    json.Unmarshal(dataBytes, &messageA)
    OR
    json.Unmarshal(dataBytes, &messageA)

Only if we knew which struct to initialize and unmarshal the bytes into. 🤔

json.RawMessage to the rescue ! 😎

On the server, let's create a wrapper struct and introduce an additional attribute called MessageType and the actual message as something generic like "content"

websocket_server_v2.go

    type MessageWrapper struct {
        MessageType   string `json:"message_type"`
        MessageType_A `json:"content"`
    }

Create an instance and populate it. Marshal it and write it on the WebSocket connection:

websocket_server_v2.go

    messageA := MessageWrapper{ 
                MessageType:   "A",
                MessageType_A: MessageType_A{Name: "Pankhudi", Place: "India"}
                }

    bytes, err := json.Marshal(messageA)

    err = websocketConn.WriteMessage(websocket.TextMessage, bytes)
    if err != nil {...}

And finally! Let us use json.RawMessage on the client side.

If you look inside the package it's essentially a type alias for byte array. Because that's the most basic form of representing data! Refer the library code: https://github.com/golang/go/blob/master/src/encoding/json/stream.go#L255

websocket_client_v2.go

    type MessageWrapper struct {
        MessageType string          `json:"message_type"`
        Content     json.RawMessage `json:"content"`
    }

Now we can unmarshal the received bytes into the wrapper struct :

    dataBytes, _, err := wsutil.ReadServerData(conn)
    if err != nil {...}

    messageWrapper := MessageWrapper{}
    err = json.Unmarshal(dataBytes, &messageWrapper)
    if err != nil {...}

By doing this, the unmarshaler populates only the MessageType field and ignores the rest. You don't believe me ? 🤨

Let's have a look at the demo: 💻

WebSocket Server:

WebSocket Client:

WebSocket Server receives the below request from client and responds with the message:

On the client end, post unmarshalling - you get to see the message type but the actual content is still in []byte and raw !

And you can happily defer unmarshalling the rest of the message based on message type! 🤗

So, In a nutshell when to use json.RawMessage?

1. To ignore certain parts of the JSON payload

   • Why would one even need that? - you ask. Well let's say in the above WebSocket connection, the server needed to send Keep-Alive packets. In that case knowing only the message type should serve the purpose.

2. To take explicit control of unmarshalling the data and thus defer that until you've branched out based on some condition

   • We saw in the above use-case.

Refer the Github gists here :

https://gist.github.com/PankhudiB/e06c56bcd65c329e6996b611d118f7ed

Hope this blog helped you understand the awesomeness of json.RawMessage! Let me know in the comments if you have any questions or feedbacks! Happy Coding! 👩‍💻