Synchronizing Collaborative Text Editing with Yjs and WebSockets

In this article, I almost do a copy-paste of what's found on the internet.
It's a "reminder" for my personal project
where I analyzed the use of Yjs.

SYNCHRONIZING DOCUMENTS VIA SOCKET

I foolishly use the y-websocket library.

NODEJS SERVER

Install:

npm install ws yjs y-websocket

create:
start.ts

import WebSocket from 'ws';
const utils = require("y-websocket/bin/utils")

/**
 * create a simple websocket server
 */
const wss = new WebSocket.Server({ port: 1234 })

wss.on('connection', (ws, req) => {

    // CONNECT the CLIENT to YJS documents
    // use the "room" passed in `req.url` (for example "/yjs-ws-demo")
    utils.setupWSConnection(ws, req)

    // and that's it.... these following are just logs
    console.log('CLIENT::CONNECTED')
    ws.on('message', message => console.log('CLIENT::MESSAGE', message))
    ws.on('close', () => console.log('CLIENT::DISCONNECTED'))
})

run:

npx ts-node start.ts

So the y-websocket library takes care of everything,
upon client connection, you just need to pass the websocket to the setupWSConnection method.

STORAGE

But if we shut down the server, all Yjs documents are lost.
To store them, we need to use y-leveldb.

Install:

npm install y-leveldb

add to
start.ts

import { LeveldbPersistence } from 'y-leveldb';
import * as Y from 'yjs';

const persistence = new LeveldbPersistence('./storage')

// apply DB updates to the Yjs document
persistence.getYDoc('yjs-ws-demo').then((persistedYdoc) => {
    Y.applyUpdate(ydoc, Y.encodeStateAsUpdate(persistedYdoc))
    ydoc.on('update', (update: Uint8Array) => {
        persistence.storeUpdate('yjs-ws-demo', update)
    })
})

With each update of the Yjs document, I store it in the DB.
It can be optimized by updating after a certain interval of time.
However, with the "update" event, we will always have a binary file "not in clear text"!
This interesting discussion: https://discuss.yjs.dev/t/decoding-yjs-messages/264/2
Given that Yjs is a CRDT system:
It allows distributing replicas of a document
these can be updated autonomously without a central server.
I realized that
implementing a custom storage and managing the clear text data of the "update"
could be a problem.

CLIENT SLATE

I use SLATE because these reflections are due to my current project.
Of course, SLATE has nothing to do with Yjs, it's just to give an example.
Later, I'll give an example with a textarea.

Create a ViteJs project for React and TypeScript

npm create vite@latest my-project --template react-ts

Install:

npm install yjs y-websocket @slate-yjs/core slate slate-react 

Replace
App.ts

import { withYjs, withYHistory, YjsEditor } from '@slate-yjs/core';
import { useEffect, useMemo, useState } from 'react';
import { createEditor, Editor, Transforms } from 'slate';
import { Editable, Slate, withReact } from 'slate-react';
import { WebsocketProvider } from 'y-websocket';
import * as Y from 'yjs';

function App() {

    const [isOnline, setIsOnline] = useState(false)
    // get the Yjs document
    const ydoc = useMemo(() => new Y.Doc(), [])
    // get the shared data type
    const sharedType = useMemo(() => ydoc.get('content', Y.XmlText), [ydoc])
    // create the Slate editor connected to Yjs
    const editor = useMemo(() => {
        const e = withReact(withYHistory(withYjs(createEditor(), sharedType)))
        // Extend normalizeNode to avoid the document being empty
        // (at least one node must be there otherwise SLATE gets very angry)
        const { normalizeNode } = e
        e.normalizeNode = (entry) => {
            const [node] = entry
            if (!Editor.isEditor(node) || node.children.length > 0) {
                return normalizeNode(entry)
            }
            Transforms.insertNodes(editor, [{ children: [{ text: '' }] }], { at: [0] })
        }
        return e
    }, [sharedType])

    // when I have all the nice things I connect to the server with the Y-websocket provider
    useEffect(() => {
        const provider = new WebsocketProvider('ws://localhost:1234', 'yjs-ws-demo', ydoc);
        provider.on('status', ({ status }: { status: string }) => 
            setIsOnline(status == 'connected')
        )

        YjsEditor.connect(editor);

        return () => {
            YjsEditor.disconnect(editor);
            provider.destroy();
        };
    }, [editor, ydoc]);

    // and here is a bit of UI
    return <div>

        <div style={{ color: isOnline ? 'green' : 'red' }}>
            {isOnline ? 'Connected' : 'Disconnected'}
        </div>

        <Slate
            editor={editor}
            initialValue={[{ children: [{ text: '' }] }]}>
            <Editable
                renderElement={({ attributes, children, element }) => <p {...attributes}>{children}</p>}
                renderLeaf={({ attributes, children, leaf }) => <span {...attributes}>{children}</span>}
                placeholder="Enter some text..."
            />
        </Slate>
    </div>
}

export default App

WebsocketProvider connects the local Yjs doc to the websocket server with the room "yjs-ws-demo":

new WebsocketProvider('ws://localhost:1234', 'yjs-ws-demo', ydoc)

So from that moment on, the Yjs document is synchronized with the WS server.
Finally, with withYjs and withYHistory, I connect the editor to Yjs.

CLIENT TEXTAREA

And if I wanted a simple textarea?
I found the solution thanks to raine. Great professional.

Install

npm install fast-diff

Replace

App.ts

import diff from 'fast-diff';
import { useEffect, useMemo, useState } from 'react';
import { WebsocketProvider } from 'y-websocket';
import * as Y from 'yjs';

function App() {

    // get the Yjs document
    const ydoc = useMemo(() => new Y.Doc(), [])
    // get the shared data type
    const sharedType = useMemo(() => ydoc.get('content', Y.Text), [ydoc])
    // connect the shared data to a "reactive" string
    const [text, setText] = useText(sharedType)

    // on component creation, connect via websocket
    useEffect(() => {
        const provider = new WebsocketProvider('ws://localhost:1234', 'yjs-ws-demo', ydoc);
        return () => {
            provider.destroy();
        };
    }, [sharedType]);

    // and here is a bit of UI
    return <div>
        <textarea style={{ width: '100%' }} rows={5}
            value={text}
            onChange={(e) => setText(e.target.value)}
        />
    </div>
}

export default App

Next, create the custom hook useText:

/** A hook to read and set a YText value. */
function useText(ytext: Y.Text) : [string, (text: string) => void] {
    // the "reactive" string
    const [text, setText] = useState(ytext.toString())
    // every time the shared data changes, update the "reactive" string
    ytext.observe(() => setText(ytext.toString()))
    // when I change the "reactive" string, update the Yjs shared data (only the differences)
    const setYText = (textNew: string) => {
        const delta = diffToDelta(diff(text, textNew));
        ytext.applyDelta(delta);
    }
    return [text, setYText]
}
/** Convert a fast-diff result to a YJS delta. */
function diffToDelta(diffResult: [number, string][]) {
    return diffResult.map(([op, value]) => ({
        [diff.INSERT]: { insert: value },
        [diff.EQUAL]: { retain: value.length },
        [diff.DELETE]: { delete: value.length },
    }[op])).filter(Boolean);
}

We could send all the text with each change, but that would be inefficient.
So we use fast-diff to calculate the differences between the current text and the previous one.
And then we transform the differences into a delta object that Yjs understands.
The sending to the server is done by y-websocket without us having to worry about it.

CONCLUSIONS

My impression is that Yjs is fantastic!
But for my personal project, I will implement a server-coordinated system for document management.
In this way, I can manage permissions and data persistence more easily.

Also, I have the feeling that the implementation for SLATE is a bit neglected.
I plan to publish some other reminders of my project!

Bye!

ivano