Encoders and Decoders in TypeScript

Encoding

Encoding in our context means converting a type to a string so we can save it to the web browser’s local storage. In soundly typed languages, this operation will never fail, but that is not true in JavaScript which TypeScript compiles to.

Caveat: Much of the code below is psuedo types and code. Please refer to the 3 “Final Full Examples” at the bottom of each section to see _actual_ working code you can use and safely reference.

Caveat: You’ll see if(error instanceof Error) example code. The instanceof keyword isn’t great for runtime type checking in all cases, so in the future use Error.isError(error) where applicable. At the time of this writing, the proposal is Stage 3, which means browsers intend to implement, and TypeScript will start to implement as well.

JSON.stringify

The JSON.stringify function can fail for a variety of reasons (circular references, BigInt), and has various nuances where some data types are converted to null, or are omitted entirely.

That means, encoding is not a safe operation. While not the true type, you can think of JSON.stringify having the following type:

stringify = (data:uknown) => string | never

That means, it can either return a string, or throw an exception and never return anything. Exceptions aren’t good, thus we must wrap the unsafe operation JSON.stringify and return some other type indicating either a success with the encoding data, or error indicating why the encoding failed.

EncodeData Function Type

In TypeScript, that means the return type must be defined as some type of Result such as a Promise or Observable, because the encoding operation can possibly fail. Below, the encoder’s type gets some data, and returns a string if it works, else an error if it fails.

type encodeData = (data:Record<string, unknown>) => Promise<string>

Success example:

john = { name: 'John', age: 17 }
result = encodeData(john)
console.log(result.data) // '{ "name": "John", "age": 17 }'

Failure example:

azathoth = { name: 'A̸̰̘͖̔z̵̛̝̀̊͠a̷͖̓̊ṱ̴̯̾̍̓̕ḣ̵̖̘̺̙̇ǫ̸̲̐̓̉t̴̢̜̊͊͂ẖ̵͆͊̾͛', age: BigInt(9007199254740991) }
result = encodeData(azathoth)
console.log(result.error) // TypeError: Do not know how to serialize a BigInt

EncodeData Function Contents

To ensure the types above remain correct, we can simply wrap the JSON.stringify above in a try/catch.

try {
  const encodedString = JSON.stringify(data)
  return Ok(string)
} catch(error) {
  return Err(error.message)
}

EncodeData Can Return a String

If you’ve seen soundly typed languages like Elm, ReScript, Haskell, Scala, etc, you may have noticed many of their encoders do not fail. This is because the the types are sound, meaning it cannot go wrong going from one type to another. The developer is required to write the encoders by hand, however. There is no one size fits all function like JSON.stringify to magically convert your type to a string. This means our encodeData would have the changed type of:

type encodeData = (data:Record<string, unknown>) => string

JavaScript has many mechanisms to encode more complex data types. The BigInt above that broke our original encoding, you can monkey patch BigInt.prototype.toJSON. For undefined and null, you can choose to manually include the property with a null value, or choose to omit it entirely, treating undefined and null as a missing value.

However, the correctness is hard to get correct with a Record<string, unknown> without library support given JavaScript’s numerous data types, and various options in encoding the data. Often data is encoded to be _decoded_, which means the developer will encode knowing they’ll want to decode the data a specific way later.

Narrowed Encoder

This means, we’ll often narrow our types to be more specific than Record, and then create custom encoders and decoders for them which is easier to create, and easier to verify they are correct in most cases.

Say we have an enum we want to encode:

enum CowRace {
  Cow = 'cow',
  NotACow = 'not a cow'
}

The type for the encoder would be:

type encodeCowRace = (cowRace:CowRace) => string

The function implementation would look something like the following:

if(cowRace === CowRace.Cow) {
  return JSON.stringify('cow')
} else {
  return JSON.stringify('not a cow')
}

Much easier to unit test, verify it is correct with compiler support, and decode back from local storage.

Specific Encoders as Function Parameters

Now that we’ve established using type narrowing results in easier to encode types, and that those types will have an associated encode function, let’s look at ways to use them in saving data.

Let’s encode our CowRace enum above to localstorage. We have 3 types, 2 of which we’ve already covered; our CowRace enum:

enum CowRace {
  Cow = 'cow',
  NotACow = 'not a cow'
}

Our encoderCowRace function type:

type encodeCowRace = (cowRace:CowRace) => string

And our new type, the saveCowRaceToLocalStorage function:

type saveCowRaceToLocalStorage =
  (cowRace:CowRace, encodeCowRace:EncodeCowRace) =>
 string

The type takes a cowRace enum, and your encoder function, and returns a string. The string is just whatever it was encoded too. The function may look something this:

encodedString = encodeCowRace(cowRace)
localStorage.setItem('cowrace', encodedString)
return encodedString

You’d invoke the saveCowRaceToLocalStorage like so:

result = saveCowRaceToLocalStorage(CowRace.Cow, encodeCowRace)
// result is "'cow'"

Generic Encoders as Function Parameters

The above uses a specific type and associated encoder function. What if you want a way to save to local storage that supports any type? In that case, you use a generic type parameter. Just like functions accept parameters, types accept parameters as well.

Let’s change our saveCowRaceToLocalStorage in 3 steps: update to accept a generic type, add a second parameter to accept any encoder, and finally add a return type. The generic implies “You can pass any type you want” which also means the developer passing the type must also create an encoder for it and pass it to us.

Step 1: Accept Generic Type Parameter

The first step is to change the name and 1st parameter type so we can save _anything_ to localstorage:

type saveAnythingLocalStorage<Type> = (data:Type) ...

That means now you can pass the cowRace parameter like before, but also the CowEnum type. Notice the cowRace is lowercase to be our enum value, and the CowRace type is uppercase to visually indicate a type:

saveAnythingLocalStorage<CowRace>(cowRace)

This also supports our Record<string, unknown>:

saveAnythingLocalStorage<Record<string, unknown>>(john)

Step 2: Accept Generic Encoder

The type parameter is a type. The generic encoder is also a type, more specifically a function type. We have to narrow down what the encoder actually returns, though. We’ll stick to string for now since most encoders will be converting their types to strings for use as JSON strings, in our case saving to local storage so we can read out later.

type saveAnythingLocalStorage<Type> =
  (
    data:Type,
    encoder:(convertMe:Type) => string
  ) ...

The function reads “Call saveAnythingLocalStorage and pass it the data you want to save, and the encoder function which converts it to a string.”

Using our existing CowRace encoder above, encodeCowRace, we can call that new function like so:

saveAnythingLocalStorage<CowRace>(cowRace, encodeCowRace)

We _could_ also do a generic one for JSON.stringify and our Record<string, unknown>, but that’s not safe given TypeScript thinks JSON.stringify has a return value of string, but we know it’s actually string | never. However, I’ve put here anyway so you know how to do it. TypeScript _is_ a gradually typed language after all, so good to get something work first, then make the types strong as you refactor.

saveAnythingLocalStorage<Record<string, unknown>>(john, JSON.stringify)

Step 3: Return Value

The last step is the return value. Since all of the encoders we’ve written cannot fail, we’ll simply return their value, a string which is their encoded representation.

type saveAnythingLocalStorage<Type> =
  (
    data:Type,
    encoder:(convertMe:Type) => string
  ) => string

The function implementation, regardless of inputs, looks like:

const encodedString = encoder(data)
return encodedString

Final Non-Failing Encoder Result

Putting it all together, our final happy path code looks like:

enum CowRace {
  Cow = 'cow',
  NotACow = 'not a cow'
}

type Encoder = (convertMe:Type) => string

const encodeCowRace = (cowRace:CowRace):string => {
  if(cowRace === CowRace.Cow) {
    return JSON.stringify('cow')
  } else {
   return JSON.stringify('not a cow')
  }
}

const saveAnythingLocalStorage = <Type,>(data:Type, encoder:Encoder):string => {
  const encodedString = encoder(data)
  return encodedString
}

const cowRace = CowRace.Cow
const result = saveAnythingLocalStorage<CowRace>(cowRace, encodeCowRace)
// result is: 'cow'

Generic Encoders That Can Fail

In soundly typed languages, encoders cannot fail. In TypeScript, using JSON.stringify under the hood means they can fail. To make our encodeCowRace safer and the types more accurate, we can change the return value to some type of Either; a type indicating something can fail. The most common in TypeScript regardless of Browser or Node.js server is Promise, and for Angular an Observable. However, both don’t treat errors as values as well, so we’ll just make our own for now.

If it works, return the encoded string. If it fails, return the Error explaining why it failed:

type EncodeResult = string | Error

We’ll change the encoder that returns a string:

type EncodeCowRace = (cowRace:CowRace) => string

To instead return our Result type:

type EncodeCowRace = (cowRace:CowRace) => EncodeResult

That means, the first part of our encodeCowRace function implementation is just wrapped with a try/catch:

try {
  if(cowRace === CowRace.Cow) {
    return JSON.stringify('cow')
  } else {
    return JSON.stringify('not a cow')
  }
...

The 2nd part, error is typed as unknown, so if it’s an Error, we’ll return that, else make a new Error and attempt to convert whatever to error was to a readable string inside it:

catch(error:unknown) {
  if(error instanceof Error) {
    return error
  } else {
    return new Error(`unknown encoding error: ${String(error)}`)
  }
}

That means our saveAnythingLocalStorage no longer “always succeeds”. So we’ll change it’s return type from a string…:

type saveAnythingLocalStorage<Type> =
  (
    data:Type,
    encoder:(convertMe:Type) => string
  ) => string

To the EncodeResult instead:

type saveAnythingLocalStorage<Type> =
  (
    data:Type,
    encoder:(convertMe:Type) => string
  ) => EncodeResult

Now the types are correct, the function is safe, and the developer can pass in any types they want to safely encode.

Final Can-Fail Decoding

Our final encoding example where the encoding can fail below:

enum CowRace {
  Cow = 'cow',
  NotACow = 'not a cow'
}

type EncodeResult = string | Error

type EncoderCanFail = <Type>(convertMe:Type) => EncodeResult

const encodeCowRace = <CowRace,>(cowRace:CowRace):EncodeResult => {
  try {
    if(cowRace === CowRace.Cow) {
      return JSON.stringify('cow')
    } else {
      return JSON.stringify('not a cow')
    }
  } catch(error:unknown) {
    if(error instanceof Error) {
      return error
    } else {
      return new Error(`unknown encoding error: ${String(error)}`)
    }
  }
}

const saveAnythingLocalStorage2 = <Type,>(data:Type, encoder:EncoderCanFail ):EncodeResult => {
  const encodedStringOrError = encoder(data)
  return encodedStringOrError
}

// success is string
const cowRace = CowRace.Cow
const result = saveAnythingLocalStorage2<CowRace>(cowRace, encodeCowRace)
// result is: 'cow'

// failure is Error
type CowIsCool = { race: CowRace, age: BigInt }
const encodeCowIsCool = <CowIsCool,>(cool:CowIsCool):EncodeResult => {
  try {
    return JSON.stringify(cool)
  } catch(error) {
    if(error instanceof Error) {
      return error
    } else {
      return new Error(`unknown encoding error: ${String(error)}`)
    }
  }
}
const failCow = { race: CowRace.Cow, age: BigInt(9007199254740991) }
const resultBad = saveAnythingLocalStorage2<CowIsCool>(failCow, encodeCowIsCool)
console.log("resultBad:", resultBad)
// BigInt value can't be serialized in JSON 

Decoding

Decoding works the same way as encoding, just in reverse. We give our decoder function a type we want to decode to, and a decoder to parse the string to our type.

Why not simply use JSON.parse and then cast the parsed result using as? A few reasons this is incorrect and dangerous:

• JSON.parse can also throw an error
• You can get an unknown return value, so you’ll have to type narrow to your type. (We’ll avoid doing type narrowing in this post and assume you’ll use something like Zod or ArkType heavily in your decoders).
• as turns TypeScript type checking off, which is unsafe

Let’s first create a specific decoder, then we’ll make it generic just like we did for our encoder.

Specific Decoder

Our enum before is CowRace, so our decoder needs to convert a string to a CowRace. However, what if someone passes a string that is not "cow" or "not a cow" such as "bunny" or empty string? We have 2 choices. We can either assume anything that’s not "cow" is CowRace.NotACow, OR we can return an error.

It may be tempting to just use a default, but this makes it much harder to debug later when many downstream components and UI’s are getting default data, and you didn’t expect it to. We want to parse, don’t validate; meaning we want to parse our data, and if it doesn’t look correct, it should fail vs. “make an assumption that bites us later which it turns out the data is invalid and we have to backtrack to figure out where our code went wrong”.

So let’s type it correctly as a Result: either we got our data and it’s good, or we got something that is not an encoded CowRace enum.

type DecodeResult = CowRace | Error

Next up is to pass in our decoder, which takes a JSON string, parses it, and attempts to convert it to a CowRace enum.

type CowRaceDecoder = (jsonString:string) => DecodeResult

The internals look something like this:

const result = JSON.parse(jsonString)
if(result === 'cow') {
  return CowRace.Cow
} else if(result === 'not a cow') {
  return CowRace.NotACow
} else {
  return new Error(`Cannot decode ${result} to a CowRace.`)}
}

That’s the happy path assuming JSON.parse works. If that fails, such as when localStorage.getItem returns a null value or a malformed JSON string, then we’ll need to handle that unhappy path as well:

} catch(error:unknown) {
  if(error instanceof Error) {
    return error
  } else {
    return new Error(`Unknown decoding error: ${String(error)}`)
  }
}

Finally, our CowRace function to read out of local storage and decode it in a type safe way looks something like:

type readCowRaceFromLocalStorage = (decoder:CowRaceDecoder) => DecodeResult

The internals look something like this:

const readString:string | null = localStorage.getItem('cowrace')
if(readString !== null) {
  const decodeResult = decoder(readString)
  return decodeResult
} else {
  return new Error('No CowRace encoded data found in localstorage.')
}

Generic Decoder

The above is specific to decoding our CowRace enum, but what if we wanted our local storage decoder to be generic? There are 3 things to make dynamic:

1. the key of where we read in local storage
2. the decoder type has to be generic
3. the decoder function needs to be updated

Let’s handle those 3 in order. The latter 2 should look familiar from the encoder exercise. We’ll cover the types first, then we’ll work on function implementation.

Step 1: Key

The key is just a string, so that type includes it as the first parameter:

type readAnythingFromLocalStorage = (key:string) => ...

That’d make your internals something like:

const readString:string | null = localStorage.getItem(key)

Step 2: Decoder

Next up is to pass our decoder. However, the return result is _too_ specific:

type DecodeResult = CowRace | Error

We need that result to return _any_ type. So let’s change that first via a type parameter:

// incorrect
type DecodeResult<Type> = Type | Error

HOWEVER, that may look right, but sadly, TypeScript unions “don’t know the difference” between the types they’re unifying if they’re objects like this. What’s to say you’re not passing an Error? Then what is type DecodeResult<Error> = Error | Error saying exactly? Yeah, I don’t know either.

So let’s whip out a discriminant so TypeScript knows the difference between _our_ generic type, and an Error.

// not quite correct
type DecodeResult<Type> = { tag: 'success', value: Type } | { tag: 'error', error: Error }

However, TypeScript may give you a compiler error when attempting to create those types like “Type ‘{ tag: “success”; value: CowRace; }’ is not assignable to type ‘DecodeResult’.” or “can’t assign Type ‘{ tag: “success”; value: CowRace; }’ is not assignable to type ‘DecodeResult’. to an Error”. TypeScript needs help, even with discriminants (our tag) to identify a type. You can’t always just make a local variable and type it. Creating the types through functions really helps TypeScript:

// almost correct
type DecodeResult<Type> = { tag: 'success', value: Type } | { tag: 'error', error: Error }

const DecodeSuccess = <Type,>(value:Type):{ tag: 'success', value: Type } =>
    ({ tag: 'success', value })

const DecodeError = (error:Error):{ tag: 'error', error: Error } =>
    ({ tag: 'error', error })

… however, those copy pasta’d anonymous types everywhere are hard to read. You can DRY types just like you can DRY code. You do this by naming your types, just like how you name your variables:

// correct
type DecodeResult<Type> = DecodeSuccess<Type> | DecodeError
type DecodeSuccess<Type> = { tag: 'success', value: Type }
type DecodeError = { tag: 'error', error: Error }

const DecodeSuccess = <Type,>(value:Type):DecodeSuccess<Type> =>
    ({ tag: 'success', value })

const DecodeError = (error:Error):DecodeError =>
    ({ tag: 'error', error })

We’ll now make the decoder more generic by including that new generic type parameter:

type Decoder<Type> = (jsonString:string) => DecodeResult<Type>

You can read that as “If I give you a JSON string, you’ll either give me the type I’m expecting back, or an Error”.

Step 3: Return Value

Now that we’ve got our return type setup, and the decoder type is now generic as well, let’s do the same for the read from local storage function’s 2nd parameter:

type readAnythingFromLocalStorage = <Type,>(
key:string,
decoder:Decoder<Type,>
) => ...

And the return value:

type readAnythingFromLocalStorage = <Type,>(
key:string,
decoder:Decoder<Type,>
):DecodeResult<Type>

Step 4: Function Implementations

We have 2 functions to write: our CowRace decoder, and our generic “read anything from localStorage”.

While our CowRace decoder is returning a specific type, it’s still using the generic type we defined above. The signature looks something like:

const decodeCowRace = (json:string):DecodeResult<CowRace> => {

Notice how we specify CowRace in the DecodeResult‘s first type parameter; it can be generic, so we’re like “Cool, we’ll return a DecodeResult with a CowRace inside it”.

Now let’s parse our Enum from a string. Since we cannot gurentee the string read from an external localStorage is _our_ only 2 available enum strings, and JSON.parse are both dangerous operations, we’ll wrap in a try/catch:

try {
    const value = JSON.parse(json)
    if(value === 'cow') {
        return DecodeSuccess<CowRace>(CowRace.Cow)
    } else if(value === 'not a cow') {
        return DecodeSuccess<CowRace>(CowRace.NotACow)
    } else {
        return DecodeError(new Error(`Cannot decode ${result} to a CowRace.`))
    }
}

That’ll handle getting our data,but if the JSON.parse throws, let’s handle the error:

} catch(error) {
  if(error instanceof Error) {
    return DecodeError(error)
  } else {
    return DecodeError(new Error(`Unknown decode error: ${String(error)}`))
  }
}

That handles our decoder. Now let’s create the generic readAnythingFromLocalStorage:

const encodedString = localStorage.getItem(key)
if(encodedString !== null) {
  return decoder(encodedString)
} else {
  return DecodeError(new Error('Failed to find key in local storage'))

Final Decoder Code

The final code for decoder is as follows:

enum CowRace {
  Cow = 'cow',
  NotACow = 'not a cow'
}

type DecodeResult<Type> = DecodeSuccess<Type> | DecodeError
type DecodeSuccess<Type> = { tag: 'success', value: Type }
type DecodeError = { tag: 'error', error: Error }

const DecodeSuccess = <Type,>(value:Type):DecodeSuccess<Type> =>
    ({ tag: 'success', value })

const DecodeError = (error:Error):DecodeError =>
    ({ tag: 'error', error })

type Decoder<Type> = (jsonString:string) => DecodeResult<Type>

const decodeCowRace = (json:string):DecodeResult<CowRace> => {
  try {
    const value = JSON.parse(json)
    if(value === 'cow') {
        return DecodeSuccess<CowRace>(CowRace.Cow)
    } else if(value === 'not a cow') {
        return DecodeSuccess<CowRace>(CowRace.NotACow)
    } else {
        return DecodeError(new Error(`Cannot decode ${result} to a CowRace.`))
    }
  } catch(error) {
    if(error instanceof Error) {
      return DecodeError(error)
    } else {
      return DecodeError(new Error(`Unknown decode error: ${String(error)}`))
    }
  }
}

const readAnythingFromLocalStorage = <Type,>(
key:string,
decoder:Decoder<Type>
):DecodeResult<Type> => {
  const encodedString = localStorage.getItem(key)
  if(encodedString !== null) {
    return decoder(encodedString)
  } else {
    return DecodeError(new Error('Failed to find key in local storage'))
  }
}

localStorage.setItem('cowrace', JSON.stringify('cow'))
const result = readAnythingFromLocalStorage<CowRace>('cowrace',decodeCowRace)
// 'Cow'

localStorage.clear()
const result2 = readAnythingFromLocalStorage<CowRace>('cowrace',decodeCowRace)
// Error: Failed to find key in local storage

Note: < Type, > vs < Type >

You have noticed in some of the types above, we used a <Type,> instead of a <Type>. For old school JavaScript functions, you can use the type parameters like so:

function<Type> nameOfFunction(...)

However, for Arrow functions, that syntax currently fails to parse in TypeScript.

type func<Type> = () => void // works
function <Type>() {} // works
func = <Type>() => undefined // fails
func = <Type,>() => undefined // works

The parameters and return value are fine, it’s just the initial type parameter part in the front. There are a few options such as having the 1st parameter extends unknown, but mixing inheritance and type parameters doesn’t make much since and is a lot more to read. Types are hard enough to read, and TypeScript types are quite verbose, so anything you can do to shrink it helps.

Conclusions

As you can see, encoding and decoding in TypeScript can make your code safer, reduce the amount of type narrowing you need to do, especially if you use Zod. For consumers of your code and api’s, it gives them the flexibility of utilizing your API’s while providing their own types which includes their own encoders and decoders in a type-safe way. TypeScript can safely ensure all erros are typesafe, and those error scenarios are handled in developers who use your code.