TSON

TSON is an extensible JSON parser that covers more data types and is a strict superset of JSON.

Initially developed as part of Milkio, TSON was later separated and released as a standalone npm package to make it available for frontend and other projects.

What is TSON used for?

Serialization and deserialization are inevitable steps in data transmission. JSON is the most common data serialization protocol on the internet, but it lacks some of the newer types needed today, such as bigint and Date. On the other hand, Protocol seems like a better choice, but it is too heavy and not specifically designed for the JavaScript ecosystem. TSON fills this gap as an extensible JSON parser that provides more data types, making data transmission more convenient and flexible. This allows us to handle richer data structures in the JavaScript environment, beyond just the basic data types defined by JSON.

Installation

TSON is built into Milkio, and all the parameters you send are actually processed and returned to the frontend using TSON. If you are using Milkio, there is no need to install it separately. You can simply use it as follows:

import { TSON } from "milkio";

For frontend developers, you can choose to treat Milkio’s responses as regular JSON and handle them in the usual way. Alternatively, you can install TSON and leverage its functionality to automatically convert some useful types back into their original objects for easier manipulation of the data. This allows you to handle responses that contain more complex data types more easily, without manual conversion and processing.

# Using bun
bun add @southern-aurora/tson

# Using npm
npm install @southern-aurora/tson

Usage

TSON is compatible with JSON, and in most cases, you can simply replace all instances of JSON in your code with TSON.

Serialization:

TSON.stringify({ hello: "world", date: new Date(0), url: new URL("https://example.com") });
// Output: '{"hello":"world","date":"t!Date:1970-01-01T00:00:00.000Z","url":"t!URL:https://example.com/"}'

Deserialization:

TSON.parse(`{"hello":"world","date":"t!Date:1970-01-01T00:00:00.000Z","url":"t!URL:https://example.com/"}`);
// Output: { hello: "world", date: 1970-01-01T00:00:00.000Z, url: URL {...} }

How It Works

TSON converts types that are not supported by JSON into string representations, for example: "t!Date:1970-01-01T00:00:00.000Z".

This string consists of the TSON prefix t!, the name of the original object, and the content of the original object. The content of the original object is serialized as much as possible so that it can be directly placed in the object constructor for recovery. You can determine if a string starts with t! to identify if it is a TSON prefix.

Default Supported Types

Type	JSON	TSON
string	✅	✅
number	✅	✅
boolean	✅	✅
null	✅	✅
Array	✅	✅
Object	✅	✅
bigint	❌	✅
Date	❌	✅
RegExp	❌	✅
URL	❌	✅
Uint8Array	❌	✅
ArrayBuffer	❌	✅

Note: While TSON supports Uint8Array and ArrayBuffer types for transmitting binary files over the network, it is not always a good idea to do so. You also need to consider factors such as chunked transmission, network fluctuations, and bandwidth costs. Perhaps what you actually need is a service similar to AWS S3 or TencentCloud COS?

Extending with More Types

TSON only includes commonly used types by default, but you can extend it with more types. Let’s take the popular date library Day.js as an example.

import * as dayjs from "dayjs";

TSON.rules.stringify.push({
  match: (v) => v.$d instanceof Date,
  handler: (v: bigint) => `t!dayjs:${v.$d.toISOString()}`,
});

TSON.rules.parse.push({
  match: (v) => v.startsWith("t!dayjs:"),
  handler: (v: string) => dayjs(v.slice("t!dayjs:".length)),
});

We check if an object has a $d property with a value of type Date to determine if it is a Day.js date object. If it is, we serialize it as a string starting with t!dayjs:, so that it can be stored legally in JSON. Note that the code for adding TSON rules needs to be placed before the code that uses TSON.

More Built-in Types?

TSON is cautious about adding more built-in types. It only includes commonly used types that are likely to be used for network transmission. The following types, although common, will never be built-in:

undefined

JavaScript implicitly converts non-existent values to undefined, so in most cases, you don’t need TSON to support undefined to obtain a value that is undefined. Supporting undefined would increase the amount of transmitted data.

Set & Map

In most cases, Set and Map can be replaced with Array and Object, and they can easily be converted to and from Array and Object. Moreover, the semantics of Set and Map are independent of JSON. Some teams may prefer to use Set and Map as alternatives to Array and Object in an attempt to gain slight performance advantages in certain scenarios. However, doing so would not only increase the cognitive burden on developers but also result in lower performance when serializing them in TSON, as they would be serialized twice. Therefore, TSON does not provide built-in support for Set and Map.

Symbols

We cannot guarantee their equality in network transmission. Are you looking for something like cuid2 or similar solutions?

Function

Serializing functions involves many issues, such as variables in the context and potential security risks. Therefore, TSON will never provide built-in support for functions. If you want to share certain functions with the frontend, consider whether extracting them into an npm package is a better solution.