SLATE Code editor with highlight

INTRODUCTION

SLATE is an excellent library for creating WYSIWYG editors in REACT, I find it superior to QUILL.

However, I had difficulties inserting editable BLOCKS with syntax highlighting for code.

Yes, there is an official example, but at least for me, it's not very clear.

Let's cut to the chase! Let's see the CODE!!!

Let's say you have an empty React project with typescript.

Install the dependencies:

npm install slate slate-react slate-history prismjs

in App.tsx

function App() {

  const editor = useMemo(() => withHistory(withReact(createEditor())), []);

  return (
    <Slate
      editor={editor}
      initialValue={[{ children: [{ text: '' }] }]}
    >
      <Editable style={{ backgroundColor: "lightgray" }}
        renderElement={({ attributes, element, children }) =>
          <div {...attributes}>{children}</div>
        }
        renderLeaf={({ attributes, children, leaf }) =>
          <span {...attributes}>{children}</span>
        }
      />
    </Slate>
  )
}

On initialization of the "App" component

I create the editor controller

and apply it to the Slate component.

Let's create the tokens for highlighting with PRISMJS

in App.tsx

... 

type BaseRangeCustom = BaseRange & { className: string }

function decorateCode([node, path]: NodeEntry) {
    const ranges: BaseRangeCustom[] = []

    // make sure it is an Slate Element
    if (!Element.isElement(node)) return ranges
    // transform the Element into a string 
    const text = Node.string(node)

    // create "tokens" with "prismjs" and put them in "ranges"
    const tokens = Prism.tokenize(text, Prism.languages.javascript);
    let start = 0;
    for (const token of tokens) {
        const length = token.length;
        const end = start + length;
        if (typeof token !== 'string') {
        ranges.push({
            anchor: { path, offset: start },
            focus: { path, offset: end },
            className: `token ${token.type}`,
        });
        }
        start = end;
    }

    // these will be found in "renderLeaf" in "leaf" and their "className" will be applied
    return ranges;
}

This function receives a SLATE Node.

I get the text of the "Node"

With the text, I create the "tokens" with PRISMJS.

I transform the "tokens" into Range.

The "Ranges" have the className property with the information for the highlight.

Finally, I apply the "Ranges" to the Slate component

I assign the function to the decorate property which is rendered with renderLeaf

still in App.tsx

...
<Editable style={{ backgroundColor: "lightgray" }}
    decorate={decorateCode}
    renderElement={({ attributes, element, children }) =>
        <div {...attributes}>{children}</div>
    }
    renderLeaf={({ attributes, children, leaf }) =>
        // here I apply the className that I calculated in "decorateCode"
        <span {...attributes} className={leaf.className}>{children}</span>
    }
/>
...

The code is here!

End.

Optimize the code

You will notice that the "decorateCode" function is called with every interaction.

Every time you press a key, it creates the tokens for all the lines!

To optimize, we use a cache.

Let's move the "decorateCode" function inside the "App" component

function App() {
    ...

    const cacheMem = useRef<{ text: string, ranges: BaseRange[] }[]>([])

    function decorateCode([node, path]: NodeEntry) {

        // CACHE **************
        const ranges: BaseRangeCustom[] = []

        // make sure it is an Slate Element
        if (!Element.isElement(node)) return ranges
        // transform the Element into a string 
        const text = Node.string(node)

        // CACHE **************
        const index = path[0]
        const cache = cacheMem.current[index]
        if (!!cache && cache.text == text) return cache.ranges
        // CACHE **************

        // create "tokens" with "prismjs" and put them in "ranges"
        const tokens = Prism.tokenize(text, Prism.languages.javascript);
        let start = 0;
        for (const token of tokens) {
        const length = token.length;
        const end = start + length;
        if (typeof token !== 'string') {
            ranges.push({
            anchor: { path, offset: start },
            focus: { path, offset: end },
            className: `token ${token.type}`,
            });
        }
        start = end;
        }

        // CACHE **************
        cacheMem.current[index] = { text, ranges }
        // CACHE **************

        // these will be found in "renderLeaf" in "leaf" and their "className" will be applied
        return ranges;
    }
}

You can find the code here!

Basically, if the Path of the Node (which is an index)

is present in the cache and the text is the same

it immediately returns the "ranges" from the cache without creating the "tokens".

Comments

[priolo]

It is interesting to note that dev.to does not use WYSIWYG.
It could be a good choice (like visual studio code)
But in my opinion it should put a preview always visible

[Boopathi]

This is a really helpful breakdown of how to implement syntax highlighting in a SLATE editor. The caching optimization is particularly valuable, as it prevents unnecessary re-calculations. 👍

[priolo]

Thanks!
It was something I needed for a project of mine and I found it very difficult. Instead in the end it is quite simple.
Yes, optimization, for a very long text is necessary!