1. Preface
I had made a TypeScript RPC (Remote Procedure Call) framework for WebSocket
and Worker
protocols 8 years ago, but have forgotten it for a long time. It's because I'd no chance to develop the websocket protocol based service for last 8 years.
By the way, I suddenly had to develop a websocket based service in nowadays. Furthermore, the websocket based service was a little bit huge (about 120K LOC), so that required many complicated and deep structured messages and procedures. What's even worse is that I had to create the service only myself.
In the project, I'd decided to re-use my TypeScript RPC framework I had long forgotten about, and soon I felt it. Ah, it would be competitive even in todayβs era. In such reason, I'd decided to revive and maintain the project again. And today, I've finally completed the example projects' developments and detailed documentations.
Now, I am here to introduce you my open source project TGrid
and its guidances.
- π¦ Github Repository: https://github.com/samchon/tgrid
- π Guide Documents: https://tgrid.com/docs
- π API Documents: https://tgrid.com/api
- π» Playground Websites:
2. Remote Procedure Call
TGrid
supports RPC (Remote Procedure Call).
With the RPC feature, you can call remote functions as if they were of your local.
If you're developing websocket client, you can call the remote function of websocket server providing. Otherwise you're developing websocket server, you also can utilize the remote functino of websocket client providing, too.
Let's see what the RPC means.
2.1. Client Program
import { Driver, WebSocketConnector } from "tgrid";
export const webSocketClientMain = async () => {
// CONNECT TO WEBSOCKET SERVER
const connector: WebSocketConnector<null, null, ICalculator> =
new WebSocketConnector(
null, // header
null, // provider for remote server
);
await connector.connect("ws://127.0.0.1:37000");
// CALL REMOTE FUNCTIONS
const remote: Driver<ICalculator> = connector.getDriver();
console.log(
await remote.plus(10, 20), // returns 30
await remote.minus(7, 3), // returns 4
await remote.multiply(3, 4), // returns 12
await remote.divide(5, 2), // returns 2.5
);
await connector.close();
};
interface ICalculator {
plus(a: number, b: number): number
minus(a: number, b: number): number
multiply(a: number, b: number): number
divide(a: number, b: number): number
}
$ npm start 30 4 12 2.5
You can run it on a Playground Website.
Looking at the example websocket client program, it is connecting to the websocket server of "ws://127.0.0.1:37000"
URL, without any header and provider.
After the connection, client program is getting Driver<ICalculator>
instance from the connector
instance to the remote websocket server. Also, client program is calling functions on the Driver<ICalculator>
instance like await remote.multiply(3, 4)
statement.
Looking the below 2.2. Server Program, you may understand that the ICalculator
is an interface type corresponding to the Calculator
class, provided by the websocket server. Driver<ICalculator>
. In actually, Driver
is a proxy instance hooking function call expression, so that intermediates them with the remote server (or client) through network communication.
Such remote function call way with TypeScript type guarding, this is the RPC (Remote Procedure Call) of TGrid
.
2.2. Server Program
import { WebSocketServer } from "tgrid";
export const webSocketServerMain = async () => {
const server: WebSocketServer<
null, // header
Calculator, // provider for remote client
null // provider from remote client
> = new WebSocketServer();
await server.open(37_000, async (acceptor) => {
const provider: Calculator = new Calculator();
await acceptor.accept(provider);
});
return server;
};
class Calculator {
public plus(x: number, y: number): number {
return x + y;
}
public minus(x: number, y: number): number {
return x - y;
}
public multiply(x: number, y: number): number {
return x * y;
}
public divide(x: number, y: number): number {
return x / y;
}
}
Websocket server is providing Calculator
class.
As you can see, every remote functions called to the Driver<ICalculator>
from the 2.1. Client Program are defined in the Calculator
class, and current websocket server program is providing the Calculator
instance to the client system for RPC (Remote Procedure Call).
2.3. Components
Describing RPC (Remote Procedure Call) concept of TGrid
with above example codes, some significant words were repated like provider
and driver
. Those words are categorized in the Components section in the TGrid
, and here is the summarized descriptions of them.
-
Communicator
: network communication with remote system -
Header
: header value directly delivered after the connection -
Provider
: object provided for remote system -
Driver
: proxy instance for calling functions of the remote system'sProvider
If you want to know more, please visit the TGrid guide documents:
- Features > Components
- Learn from Examples > Remote Function Call
- Learn from Examples > Remote Object Call
- Learn from Examples > Object Oriented Netwoprk
3. Protocols
3.1. WebSocket
TGrid
supports websocket protocol.
With TGrid
, you can easily develop WebSocket system under the RPC (Remote Procedure Call) concept.
By the way, if you're planning to develop websocket application, I recommend integrate TGrid
with NestJS
instead of utilzing native classes like WebSocketServer
and WebSocketConnector
directly.
It's because you can manage WebSocket API endpoints much effectively and easily by NestJS
controller patterns. Also, you can make your server to support both HTTP and WebSocket protocols at the same time. NestJS
controllers are compatible with both HTTP and WebSocket operations.
import { Driver, WebSocketServer } from "tgrid";
import { ICalcConfig } from "./interfaces/ICalcConfig";
import { ICalcEventListener } from "./interfaces/ICalcEventListener";
import { CompositeCalculator } from "./providers/CompositeCalculator";
import { ScientificCalculator } from "./providers/ScientificCalculator";
import { SimpleCalculator } from "./providers/SimpleCalculator";
import { StatisticsCalculator } from "./providers/StatisticsCalculator";
export const webSocketServerMain = async () => {
const server: WebSocketServer<
ICalcConfig,
| CompositeCalculator
| SimpleCalculator
| StatisticsCalculator
| ScientificCalculator,
ICalcEventListener
> = new WebSocketServer();
await server.open(37_000, async (acceptor) => {
// LIST UP PROPERTIES
const config: ICalcConfig = acceptor.header;
const listener: Driver<ICalcEventListener> = acceptor.getDriver();
// ACCEPT OR REJECT
if (acceptor.path === "/composite")
await acceptor.accept(new CompositeCalculator(config, listener));
else if (acceptor.path === "/simple")
await acceptor.accept(new SimpleCalculator(config, listener));
else if (acceptor.path === "/statistics")
await acceptor.accept(new StatisticsCalculator(config, listener));
else if (acceptor.path === "/scientific")
await acceptor.accept(new ScientificCalculator(config, listener));
else await acceptor.reject(1002, `WebSocket API endpoint not found.`);
});
return server;
};
A websocket server example using native websocket classes of
TGrid
directly. If you integrate withNestJS
, you can manage operation paths much effectively.
3.2. Worker
TGrid
supports Worker
protocol(?).
TGrid
considers Worker
as a 1: 1 dedicated server, as well as a little bit special network protocol. Therefore, you don't no more need to manually send and parse raw level message (Worker.postMessage()
). Just utilize RPC (Remote Procedure Call) even when developing the Worker
based multi-processing application.
Here is an example code interacting with Worker
with RPC through TGrid
. With the example code, you may understand why TGrid
is considering the Worker
as a network system, even if Worker
is never a type of network system in the real world.
import { Driver, WorkerConnector } from "tgrid";
import { ICalcConfig } from "./interfaces/ICalcConfig";
import { ICalcEvent } from "./interfaces/ICalcEvent";
import { ICalcEventListener } from "./interfaces/ICalcEventListener";
import { ICompositeCalculator } from "./interfaces/ICompositeCalculator";
const EXTENSION = __filename.endsWith(".ts") ? "ts" : "js";
export const workerClientMain = async () => {
const stack: ICalcEvent[] = [];
const listener: ICalcEventListener = {
on: (evt: ICalcEvent) => stack.push(evt),
};
const connector: WorkerConnector<
ICalcConfig,
ICalcEventListener,
ICompositeCalculator
> = new WorkerConnector(
{ precision: 2 }, // header
listener, // provider for remote server
"process",
);
await connector.connect(`${__dirname}/server.${EXTENSION}`);
const remote: Driver<ICompositeCalculator> = connector.getDriver();
console.log(
await remote.plus(10, 20), // returns 30
await remote.multiplies(3, 4), // returns 12
await remote.divides(5, 3), // returns 1.67
await remote.scientific.sqrt(2), // returns 1.41
await remote.statistics.mean(1, 3, 9), // returns 4.33
);
await connector.close();
console.log(stack);
};
$ npm start 30 12 1.67 1.41 4.33 [ { type: 'plus', input: [ 10, 20 ], output: 30 }, { type: 'multiplies', input: [ 3, 4 ], output: 12 }, { type: 'divides', input: [ 5, 3 ], output: 1.67 }, { type: 'sqrt', input: [ 2 ], output: 1.41 }, { type: 'mean', input: [ 1, 3, 9 ], output: 4.33 } ]
You can run it on a Playground Website.
3.3. SharedWorker
As well as Worker
case, TGrid
also supports SharedWorker
protocol.
Also, as SharedWorker
can accept multiple clients, TGrid
considers it a s a local server running on the web browser, so that its interfaces are almost similar with the websocket protocol case.
import { Driver, SharedWorkerServer } from "tgrid";
import { ICalcConfig } from "./interfaces/ICalcConfig";
import { ICalcEventListener } from "./interfaces/ICalcEventListener";
import { CompositeCalculator } from "./providers/CompositeCalculator";
const main = async () => {
let pool: number = 0;
const server: SharedWorkerServer<
ICalcConfig,
CompositeCalculator,
ICalcEventListener
> = new SharedWorkerServer();
await server.open(async (acceptor) => {
// LIST UP PROPERTIES
const config: ICalcConfig = acceptor.header;
const listener: Driver<ICalcEventListener> = acceptor.getDriver();
// ACCEPT OR REJECT THE CONNECTION
if (pool >= 8) {
await acceptor.reject("Too much connections.");
} else {
await acceptor.accept(new CompositeCalculator(config, listener));
++pool;
await acceptor.join();
--pool;
}
});
};
main().catch(console.error)
You can run it on your local machine.
git clone https://github.com/samchon/tgrid.example.shared-worker npm install npm run build npm start
4. NestJS Integration
4.1. Outline
NestJS integration is the only one feature newly added to TGrid
, reving 8 years have slept project. Also, I've told in the previous section (3.1. WebSocket) that integrating with NestJS
is better than just utilizing native classes, when developing websocket protocol based application.
Let's see how NestJS integration makes websocket development efficiently.
You can run it on a Playground Website
4.2. Server Program
4.2.1. Bootstrap
import { WebSocketAdaptor } from "@nestia/core";
import { INestApplication } from "@nestjs/common";
import { NestFactory } from "@nestjs/core";
import { CalculateModule } from "./calculate.module";
export const bootstrap = async (): Promise<INestApplication> => {
const app: INestApplication = await NestFactory.create(CalculateModule);
await WebSocketAdaptor.upgrade(app);
await app.listen(37_000, "0.0.0.0");
return app;
};
To integrate TGrid
with NestJS
, you have to upgrade the NestJS application like above.
Just call the WebSocketAdaptor.upgrade()
, then you can utilize TGrid
in the NestJS
server.
4.2.2. Controller
import { TypedRoute, WebSocketRoute } from "@nestia/core";
import { Controller } from "@nestjs/common";
import { Driver, WebSocketAcceptor } from "tgrid";
import { ICalcConfig } from "./api/interfaces/ICalcConfig";
import { ICalcEventListener } from "./api/interfaces/ICalcEventListener";
import { ICompositeCalculator } from "./api/interfaces/ICompositeCalculator";
import { IScientificCalculator } from "./api/interfaces/IScientificCalculator";
import { ISimpleCalculator } from "./api/interfaces/ISimpleCalculator";
import { IStatisticsCalculator } from "./api/interfaces/IStatisticsCalculator";
import { CompositeCalculator } from "./providers/CompositeCalculator";
import { ScientificCalculator } from "./providers/ScientificCalculator";
import { SimpleCalculator } from "./providers/SimpleCalculator";
import { StatisticsCalculator } from "./providers/StatisticsCalculator";
@Controller("calculate")
export class CalculateController {
/**
* Health check API (HTTP GET).
*/
@TypedRoute.Get("health")
public health(): string {
return "Health check OK";
}
/**
* Prepare a composite calculator.
*/
@WebSocketRoute("composite")
public async composite(
@WebSocketRoute.Acceptor()
acceptor: WebSocketAcceptor<
ICalcConfig,
ICompositeCalculator,
ICalcEventListener
>,
@WebSocketRoute.Header() header: ICalcConfig,
@WebSocketRoute.Driver() listener: Driver<ICalcEventListener>
): Promise<void> {
const provider: CompositeCalculator = new CompositeCalculator(
header,
listener
);
await acceptor.accept(provider);
}
/**
* Prepare a simple calculator.
*/
@WebSocketRoute("simple")
public async simple(
@WebSocketRoute.Acceptor()
acceptor: WebSocketAcceptor<
ICalcConfig, // header
ISimpleCalculator, // provider for remote client
ICalcEventListener // provider from remote client
>
): Promise<void> {
const header: ICalcConfig = acceptor.header;
const listener: Driver<ICalcEventListener> = acceptor.getDriver();
const provider: SimpleCalculator = new SimpleCalculator(header, listener);
await acceptor.accept(provider);
}
/**
* Prepare a scientific calculator.
*/
@WebSocketRoute("scientific")
public async scientific(
@WebSocketRoute.Acceptor()
acceptor: WebSocketAcceptor<
ICalcConfig,
IScientificCalculator,
ICalcEventListener
>
): Promise<void> {
const header: ICalcConfig = acceptor.header;
const listener: Driver<ICalcEventListener> = acceptor.getDriver();
const provider: ScientificCalculator = new ScientificCalculator(
header,
listener
);
await acceptor.accept(provider);
}
/**
* Prepare a statistics calculator.
*/
@WebSocketRoute("statistics")
public async statistics(
@WebSocketRoute.Acceptor()
acceptor: WebSocketAcceptor<
ICalcConfig,
IStatisticsCalculator,
ICalcEventListener
>
): Promise<void> {
const header: ICalcConfig = acceptor.header;
const listener: Driver<ICalcEventListener> = acceptor.getDriver();
const provider: IStatisticsCalculator = new StatisticsCalculator(
header,
listener
);
await acceptor.accept(provider);
}
}
As you can see from the above code, CalculateController
has many API operations, including both HTTP and WebSocket protocols. The CalculatorController.health()
is an HTTP Get method operation, and the others are all WebSocket operations.
When defining WebSocket operation, attach @WebSocketRoute()
decorator to the target controller method with path specification. Also, the controller method must have the @WebSocketRoute.Acceptor()
decorated parameter with WebSocketAcceptor type, because you have to determine whether to WebSocketAcceptor.accept()
the client's connection or WebSocketAcceptor.reject()
it.
With such controller patterned WebSocket operation, you can manage WebSocket API endpoints much effectively and easily. Also, you can generate SDK (Software Development Kit) library for your client application through Nestia
. Let's see how to generate SDK library, and how it would be looked like in the next section.
4.2.3. Software Development Kit
/**
* @packageDocumentation
* @module api.functional.calculate
* @nestia Generated by Nestia - https://github.com/samchon/nestia
*/
//================================================================
import type { IConnection, Primitive } from "@nestia/fetcher";
import { PlainFetcher } from "@nestia/fetcher/lib/PlainFetcher";
import { WebSocketConnector } from "tgrid";
import type { Driver } from "tgrid";
import type { ICalcConfig } from "../../interfaces/ICalcConfig";
import type { ICalcEventListener } from "../../interfaces/ICalcEventListener";
import type { ICompositeCalculator } from "../../interfaces/ICompositeCalculator";
import type { IScientificCalculator } from "../../interfaces/IScientificCalculator";
import type { ISimpleCalculator } from "../../interfaces/ISimpleCalculator";
import type { IStatisticsCalculator } from "../../interfaces/IStatisticsCalculator";
/**
* Health check API (HTTP GET).
*
* @controller CalculateController.health
* @path GET /calculate/health
* @nestia Generated by Nestia - https://github.com/samchon/nestia
*/
export async function health(connection: IConnection): Promise<health.Output> {
return PlainFetcher.fetch(connection, {
...health.METADATA,
path: health.path(),
});
}
export namespace health {
export type Output = Primitive<string>;
export const METADATA = {
method: "GET",
path: "/calculate/health",
request: null,
response: {
type: "application/json",
encrypted: false,
},
status: null,
} as const;
export const path = () => "/calculate/health";
}
/**
* Prepare a composite calculator.
*
* @controller CalculateController.composite
* @path /calculate/composite
* @nestia Generated by Nestia - https://github.com/samchon/nestia
*/
export async function composite(
connection: IConnection<composite.Header>,
provider: composite.Provider,
): Promise<composite.Output> {
const connector: WebSocketConnector<
composite.Header,
composite.Provider,
composite.Listener
> = new WebSocketConnector(connection.headers ?? ({} as any), provider);
await connector.connect(
`${connection.host.endsWith("/") ? connection.host.substring(0, connection.host.length - 1) : connection.host}${composite.path()}`,
);
const driver: Driver<composite.Listener> = connector.getDriver();
return {
connector,
driver,
};
}
export namespace composite {
export type Output = {
connector: WebSocketConnector<Header, Provider, Listener>;
driver: Driver<Listener>;
};
export type Header = ICalcConfig;
export type Provider = ICalcEventListener;
export type Listener = ICompositeCalculator;
export const path = () => "/calculate/composite";
}
/**
* Prepare a simple calculator.
*
* @controller CalculateController.simple
* @path /calculate/simple
* @nestia Generated by Nestia - https://github.com/samchon/nestia
*/
export async function simple(
connection: IConnection<simple.Header>,
provider: simple.Provider,
): Promise<simple.Output> {
const connector: WebSocketConnector<
simple.Header,
simple.Provider,
simple.Listener
> = new WebSocketConnector(connection.headers ?? ({} as any), provider);
await connector.connect(
`${connection.host.endsWith("/") ? connection.host.substring(0, connection.host.length - 1) : connection.host}${simple.path()}`,
);
const driver: Driver<simple.Listener> = connector.getDriver();
return {
connector,
driver,
};
}
export namespace simple {
export type Output = {
connector: WebSocketConnector<Header, Provider, Listener>;
driver: Driver<Listener>;
};
export type Header = ICalcConfig;
export type Provider = ICalcEventListener;
export type Listener = ISimpleCalculator;
export const path = () => "/calculate/simple";
}
/**
* Prepare a scientific calculator.
*
* @controller CalculateController.scientific
* @path /calculate/scientific
* @nestia Generated by Nestia - https://github.com/samchon/nestia
*/
export async function scientific(
connection: IConnection<scientific.Header>,
provider: scientific.Provider,
): Promise<scientific.Output> {
const connector: WebSocketConnector<
scientific.Header,
scientific.Provider,
scientific.Listener
> = new WebSocketConnector(connection.headers ?? ({} as any), provider);
await connector.connect(
`${connection.host.endsWith("/") ? connection.host.substring(0, connection.host.length - 1) : connection.host}${scientific.path()}`,
);
const driver: Driver<scientific.Listener> = connector.getDriver();
return {
connector,
driver,
};
}
export namespace scientific {
export type Output = {
connector: WebSocketConnector<Header, Provider, Listener>;
driver: Driver<Listener>;
};
export type Header = ICalcConfig;
export type Provider = ICalcEventListener;
export type Listener = IScientificCalculator;
export const path = () => "/calculate/scientific";
}
/**
* Prepare a statistics calculator.
*
* @controller CalculateController.statistics
* @path /calculate/statistics
* @nestia Generated by Nestia - https://github.com/samchon/nestia
*/
export async function statistics(
connection: IConnection<statistics.Header>,
provider: statistics.Provider,
): Promise<statistics.Output> {
const connector: WebSocketConnector<
statistics.Header,
statistics.Provider,
statistics.Listener
> = new WebSocketConnector(connection.headers ?? ({} as any), provider);
await connector.connect(
`${connection.host.endsWith("/") ? connection.host.substring(0, connection.host.length - 1) : connection.host}${statistics.path()}`,
);
const driver: Driver<statistics.Listener> = connector.getDriver();
return {
connector,
driver,
};
}
export namespace statistics {
export type Output = {
connector: WebSocketConnector<Header, Provider, Listener>;
driver: Driver<Listener>;
};
export type Header = ICalcConfig;
export type Provider = ICalcEventListener;
export type Listener = IStatisticsCalculator;
export const path = () => "/calculate/statistics";
}
When you run npx nestia sdk command, SDK (Software Development Kit) library be generated.
Above file is one of the SDK library corresponding to the CalculateController
class we've seen in the previous 4.2. Controller section. Client developers can utilize the automatically generated SDK functions to connect to the WebSocket server, and interact it type safely. Also, HTTP operation is compatible with the WebSocket operation.
Let's see how client developer utilizes the SDK library in the next section.
4.3. Client Program
import api from "./api";
import { ICalcEvent } from "./api/interfaces/ICalcEvent";
import { ICalcEventListener } from "./api/interfaces/ICalcEventListener";
export const testCalculateSdk = async () => {
//----
// HTTP PROTOCOL
//---
// CALL HEALTH CHECK API
console.log(
await api.functional.calculate.health({
host: "http://127.0.0.1:37000",
})
);
//----
// WEBSOCKET PROTOCOL
//---
// PROVIDER FOR WEBSOCKET SERVER
const stack: ICalcEvent[] = [];
const listener: ICalcEventListener = {
on: (evt: ICalcEvent) => stack.push(evt),
};
// DO CONNECT
const { connector, driver } = await api.functional.calculate.composite(
{
host: "ws://127.0.0.1:37000",
headers: {
precision: 2,
},
},
listener
);
// CALL FUNCTIONS OF REMOTE SERVER
console.log(
await driver.plus(10, 20), // returns 30
await driver.multiplies(3, 4), // returns 12
await driver.divides(5, 3), // returns 1.67
await driver.scientific.sqrt(2), // returns 1.41
await driver.statistics.mean(1, 3, 9) // returns 4.33
);
// TERMINATE
await connector.close();
console.log(stack);
};
$ npm start [Nest] 4328 - 05/15/2024, 3:19:50 AM LOG [NestFactory] Starting Nest application... [Nest] 4328 - 05/15/2024, 3:19:50 AM LOG [InstanceLoader] CalculateModule dependencies initialized +5ms [Nest] 4328 - 05/15/2024, 3:19:50 AM LOG [RoutesResolver] CalculateController {/calculate}: +5ms [Nest] 4328 - 05/15/2024, 3:19:50 AM LOG [NestApplication] Nest application successfully started +2ms Health check OK 30 12 1.67 1.41 4.33 [ { type: 'plus', input: [ 10, 20 ], output: 30 }, { type: 'multiplies', input: [ 3, 4 ], output: 12 }, { type: 'divides', input: [ 5, 3 ], output: 1.67 }, { type: 'sqrt', input: [ 2 ], output: 1.41 }, { type: 'mean', input: [ 1, 3, 9 ], output: 4.33 } ]
Do import the SDK, and enjoy the type-safe and easy-to-use RPC (Remote Procedure Call).
Looking at the above code, the client application is calling a function of the automatically generated SDK (Software Development Kit) library, so that connecting to the websocket server, and starting interaction through RPC (Remote Procedure Call) concept with Driver<ICompositeCalculator>
instance.
Doesn't the "SDK based development" seems much easier and safer than native websocket classes case? This is the reason why I've recommended to combine with the NestJS when using websocket protocol based network system.
This is the integration of TGrid with NestJS.
5. Conclusion
Let's develop websocket (NestJS
) and worker much easily through RPC (Remote Procedure Call).
TGrid
wakes up after 8 years and will help you.
Top comments (2)
Pretty nice, interesting project. especially communication with websockets.
Personally, I have a question for you: Do you have a plan to integrate the React 19 Server Component feature into NestJS?
Are you meaning that composing React Server Components in the
NestJS
, and the react server component is calling its own server API functions through the SDK generated byNestia
? If that, I have not tried it yet, but will not be any problem considering it in the theoretical level.