We can persist and rehydrate DOM objects with simple vanilla JS.
5-6 minutes, 1332 words, 4th grade
Here we will create two simple but powerful JavaScript functions.
The first, jsToDom, will take a JavaScript (or JSON) object and turn it into a DOM object that we can insert into our page. The second, domToJs, does the reverse. It takes a DOM object and converts it to JS.
Now we can stringify it and persist it in our database. And rehydrate it at will.
As you may know, JSON does not recognize functions. So we will need a way to deal with our event listeners. Donʼt worry! Weʼve got it covered.
Check out our example of DOM to JSON and back in action. We will explain it below.
A key axiom of Craft Code is less is more. This influences several of the Craft Code methods, such as code just in time and keep it simple.
This means that we donʼt rush to load up dozens of frameworks, libraries, and other dependencies. Instead, we start with nothing: zero dependencies.
We build the site structure with semantically-correct and accessible HTML. Then we add just enough CSS to make it attractive, user-friendly, and responsive.
Finally, we add JavaScript to progressively enhance the user experience. But only if we need it.
This vanilla approach works very well. One goal of the Craft Code effort is to see how far we can go before we have to add a dependency on someone elseʼs code.
The code
For the impatient, letʼs take a look at the final code. Then weʼll explain it. The next two functions are the totality of the module. The rest are specific to this example.
Note: this is about concepts, not production-ready code. This is a first pass and might could use some refactoring. YMMV.
// ./modules/js-to-dom.js
export default async function jsToDom (js) {
const { attributes, children, events, tagName } = js
const elem = document.createElement(tagName)
for (const attr in attributes) {
elem.setAttribute(attr, attributes[attr])
}
if (Array.isArray(children)) {
for (const child of children) {
typeof child === "object"
? elem.appendChild(await jsToDom(child))
: elem.appendChild(document.createTextNode(child))
}
}
if (events) {
for (const key in events) {
if (!events[key]) {
break
}
const handler =
typeof events[key] === "function"
? events[key]
: (await import(`./${events[key]}.js`)).default
handler && elem.addEventListener(key, handler)
}
setDataEvents(elem, js.events)
}
return elem
}
function setDataEvents (elem, obj = {}) {
const eventString = Object.keys(obj)
.reduce((out, key) => {
if (typeof obj[key] === "string") {
out.push(`${key}:${obj[key]}`)
}
return out
}, [])
.join(",")
if (eventString) {
elem.setAttribute("data-events", eventString)
}
}
We grab the tagName from the JSON and create a DOM element of that type. Then we add the attributes to that element. Then we apply jsToDom recursively on the children, appending them to the element.
The events object is pretty clever, IOHO. The keys are the names of the events (e.g., click) and the values are the names of the handler functions. We will import those functions only when needed. See an example below.
We also create a string representation of the events object. We could have simply stringified it, but we wanted to make it human readable, so we wrote our own (lines #39 to #52).
// ./modules/dom-to-js.js
export default function domToJs (dom) {
const { attributes, childNodes, tagName } = dom
const eventList = dom.getAttribute("data-events")
const events = eventList?.split(",").reduce((out, evt) => {
const [key, value] = evt.split(":")
if (key) {
out[key] = value
}
return out
}, {})
const attrs = Object.values(attributes)
.map((v) => v.localName)
.filter((name) => name !== "data-events")
return {
tagName,
attributes: attrs.reduce((out, attr) => {
out[attr] = dom.getAttribute(attr)
return out
}, {}),
events,
children: Array.from(childNodes).map((_, idx) => {
const child = childNodes[idx]
return child.nodeType === Node.TEXT_NODE
? child.nodeValue
: domToJs(child)
}),
}
}
This is a bit tricky as nothing in the DOM is simple. Go figure. We explain below.
This extracts the attributes, the childNodes, and tagName from the passed DOM element. Then we use these to create a simple JS/JSON object, recursing through the child nodes.
We pull the data-events attribute out and treat it separately. We parse the value back into an actual object and add it at the events key.
The output is JS, but we can stringify it to JSON as required. The JSON shown below is a typical example. It creates our test form.
{
"tagName": "FORM",
"attributes": {
"action": "#",
"method": "POST",
"name": "form"
},
"events": {
"focusin": "log",
"submit": "parse-submission"
},
"children": [
{
"tagName": "TEXTAREA",
"attributes": {
"data-type": "json",
"name": "json"
},
"children": [
"{\"tagName\":\"DIV\",\"attributes\":{\"class\":\"sb-test\",\"data-type\":\"string\",\"id\":\"sb-test-id\"},\"events\":{\"click\":\"log\"},\"children\":[{\"tagName\":\"STRONG\",\"children\":[\"Bob's yer uncle.\"]}]}"
]
},
{
"tagName": "BUTTON",
"attributes": {
"aria-label": "Run this baby",
"type": "submit"
},
"children": [
"Run"
]
}
]
}
We import this JSON and pass it to jsToDom in our index.js file below.
// index.js
import jsToDom from "./modules/js-to-dom.js"
import formJson from "./modules/form-json.js"
import outJson from "./modules/out-json.js"
export async function injectForm () {
const main = document.querySelector("main")
main.appendChild(
await jsToDom(outJson),
)
main.appendChild(
await jsToDom(formJson),
)
}
globalThis.addEventListener("DOMContentLoaded", injectForm)
Pretty self-explanatory. Now, how do we handle our events?
Simple. We take our events object from the passed JSON/JS. Then we loop through the keys, which are the event types. The values are the names of the handler functions.
We add an event listener for each type and assign it the default function from the module with that name. For example, our output div gets a click handler called “log”. This function is in ./modules/log.js.
We import the handler: (await import("./log.js")).default. We assign it to handler.
Then we add it like this: addEventListener("click", handler).
Drop dead simple. And we only import the modules that we need. See the actual log handler below.
// ./modules/log.js
export default function log ({ target }) {
console.log(
target?.tagName,
target?.innerText || target?.value
)
}
Kinda dumb, but it is merely an example. We add this log function as a click handler on our output strong element and as a focusin handler on our form.
The submit handler for the form is a bit more exciting:
// ./modules/parse-submission.js
import domToJs from "./dom-to-js.js"
import jsToDom from "./js-to-dom.js"
export default async function (event) {
event.preventDefault()
const form = event.target
const textarea = form.querySelector("textarea")
const out = document.querySelector(".out")
const js = JSON.parse(textarea.value)
out.appendChild(await jsToDom(js))
const newForm = domToJs(form)
document.querySelector("main").appendChild(
await jsToDom(newForm)
)
}
We canʼt store functions in JSON, so we put them into modules. Then we will import them as needed when we rehydrate the DOM elements.
We attach parseSubmission as the submit handler for our form element.
What can we do with this?
Ooo. All sorts of cool things.
Easy element creation
Instead of messing around with createElement, setAttribute, etc., we can use jsToDom. We pass it a JS or JSON object representing the DOM elements we want.
We create handler functions ahead of time in modules. When we need an event listener, jsToDom imports it just in time and assigns it to the element.
This works like Reactʼs createElement function. Or a library such as hyperscript. Sure, weʼd prefer JSX for its much reduced cognitive load. But our alternative here is the DOM methods such as createElement. Unless we want to load up a bulky library such as React, that is.
We donʼt.
Suppose I wanted to inject a password field with a show/hide button. First, we create a toggle handler such as this:
// ./modules/toggle-visibility.js
export default function (event) {
const button = event.target
const div = button.closest(".form-field")
const input = div?.querySelector("input")
if (input) {
if (input.type === "password") {
input.type = "text"
button.innerText = "hide"
button.setAttribute("aria-label", "Hide password.")
return
}
input.type = "password"
button.innerText = "show"
button.setAttribute("aria-label", "Show password.")
}
}
Now I can call jsToDom with the following JSON and it will create my password input. Try pasting it into the example form. Remember that the click event handler is already available at ./modules/toggle-visibility.js.
{
"tagName": "DIV",
"attributes": {
"class": "form-field"
},
"events": {
"click": "log"
},
"children": [
{
"tagName": "INPUT",
"attributes": {
"type": "password"
}
},
{
"tagName": "BUTTON",
"attributes": {
"aria-label": "Show password.",
"class": "xx-toggle-password",
"type": "button"
},
"events": {
"click": "toggle-visibility"
},
"children": ["show"]
}
]
}
We hope that it is straightforward how all this works.
Easy element persistance
What if I want to save current UI state? We can use domToJs to do just that.
We took the example page (linked above) and passed the html element to domToJs. Then we stringified it. Now we have preserved both the head and body elements.
So we can take a blank HTML document like this:
<html lang="en">
<head>
</head>
<body>
<script src="./modules/make-page.js" type="module"></script>
</body>
</html>
And we can use our persisted JSON head and body elements to create the page on the fly. We can store the JSON in a database or load it from an API.
Below is the code minus the JSON. Or view the actual code.
Then see it in action. View the source on that page to see what we mean.
import jsToDom from "./js-to-dom.js"
globalThis.addEventListener("DOMContentLoaded", async () => {
const h = document.documentElement.querySelector("head")
const b = document.documentElement.querySelector("body")
h.replaceWith(await jsToDom(/* head JSON here */))
b.replaceWith(await jsToDom(/* body JSON here */))
})
A whole page generated from simple JSON!
Data-driven user interfaces
One idea that we have been promoting for several years now is that of a data-driven interface. Weʼve got an article in the pipeline on that coming soon. But we can give a quick overview here.
The idea is quite simple. The easiest example to visualize is automated form rendering.
Forms collect data. Typically, that data is then persisted in a database.
Databases have schemas. That means that the database already knows the types it expects. If we have defined our schema well, then it knows those types precisely.
Particular data use particular interface widgets. For example, an email address would use an input element of type “email”. An integer might use an input element of type “number” with its step attribute set to “1”.
An enum might use a select or radio buttons or a checkbox group.
From this schema we should be able to determine how to both display and validate that data. After all, there is only a small number of widgets.
So what if the response to our database query for the data included the schema? Some GraphQL queries already make this possible. From that schema, we can generate validation functions. And we know which widgets to display.
So we can generate our form and our validators automatically.
Best of all, we have a single source of truth: our database schema.
In a coming article, we will explain how easy it is to achieve this.
(There is an advanced version of this that uses a SHACL ontology and a triple store such as Fuseki. We then use SPARQL queries to generate the HTML, CSS, and JS straight from the database. Wee ha! Weʼll get to that sometime soon, too. Promise.)
If anyone requests it, weʼll give a detailed explanation of the above code in a separate article. This one is long enough.
Top comments (8)
I appreciate DOM generation via Javascript and think it is more powerful than moste people think. My own library DML works pretty well with an event based approach, libraries like VanJS do a good job using a React-like style.
In general, the idea to store the DOM and rehydrate it after reloading sounds nice, but why do you use Javascript for DOM serialization? HTML is already a perfect serialization format, converting to js and storing as JSON sounds a bit crazy. It is like reinventing HTML again?!?
Sorry for the delay in responding. For some reason, I am only now seeing these comments.
That's an interesting idea and one that I used long ago.
In 2011, IIRC, I built a CMS from scratch for a big, annual event. I used a Scala framework called Circumflex (which was awesome) and the BaseX XML database.
The pages were created from XHTML fragments and templates, all of which were stored in the DB as XML. Each fragment allowed one or more Duration XML elements, which could have an "enabled" date-time, an "expires" date-time, or both.
When the user first fetched a page, an XQuery query would construct it from the various fragments. In the process, it would check the Duration elements and calculate the soonest expiration date-time. Obviously, only enabled fragments were included.
Essentially, it calculated the soonest time that a fragment would expire or become enabled, thus changing the page.
Then it served the compiled page and saved it back to the DB while setting its expiration to the calculated earliest Duration expiration. If I updated a fragment, then all pages using that element were immediately deleted from the "cache". They would be regenerated only when someone requested them.
Generating even the most complex page took only 500-600 ms. Pulling one from the cache was blindingly fast: 20-30 ms. It worked beautifully. I was actually astonished at how well it worked.
And my reasoning was exactly yours: why are we doing all this conversion when we can simply store the XHTML in an XML database and manipulate it with XPath, XSLT, etc.?
Sadly, the end of XHTML (I fought on the losing side in that war) pretty much killed that idea. No one cares about valid XML anymore. Few ever did.
The whole app was just that: XHTML, CSS, and JS stored in fragments in the BaseX DB and a thin Circumflex layer on top.
If something needed to be updated more frequently, it used client-side JS to do so, pulling fragments from the DB and injecting them into the DOM.
I loved this idea, but could not sell it to others and the UI for updating it was never finished. (Wow, does Ext-JS suck.) The client gave it up for a WordPress app. Sigh ...
The reason I convert to JS/JSON above is because it is easier to manipulate the elements in JavaScript/TypeScript. And this article is for people just coming around to the idea of using vanilla code instead of fat libraries of other people's code. The point is, simply, that things that you think are very difficult might not be difficult at all.
And the difficulty that remains is usually in the edge cases. Unless you are creating an open source library, you may never encounter any.
That said, my goal eventually is to use a triple store and a good ontology to describe the business and interface models in SHACL or OWL2, and then to use SPARQL to generate the HTML/CSS/JS directly from the triple store (e.g., Fuseki).
Single source of truth, baby! And the ability to run an inference engine on it. Hell, yeah.
Essentially, that's the BaseX/XQuery idea taken to its logical (heh) conclusion. I hope I live long enough to make that happen.
Also, note: the browser has no clue what HTML is. It actually understands only the DOM. The
domToJSfunction is really just anhtmlToDomfunction. It is the DOM I am recommending storing, with minor mapping to make it easier to understand.Does that make sense?
Maybe you misunderstood my motivation. Any static format (regardless if it´s HTML, XML or JSON) will limit us. But building the DOM should be highly dynamic. Building the DOM with Javascript has some advantages, that cannot be overestimated:
You can control the build process, reacting to system, properties, daytime or whatever you like to make your page responsive. This gives us resoonsiveness out of the box, now tools needed
The fact that all HTML-identifiers are global scoped is a main source of trouble. DOM elements generated from Javascript can be stored as local variables. You can build classes, that contain all code to generate and manipulate DOM-elements. The DOM elements generated by this class will be private to the class-objects. This is a completely different game, and it makes things like webcomponents superfluous.
You can distinguish between global scoped CSS, class coped CSS and object scoped CSS. This gives you much better control and removes about 80% of the CSS you have in traditional HTML-applications.
JSToDom makes it very easy to write content generators. This could be a markdown parser, a custom JSON-reader or you can simply inject HTML to a DOM element. This gives you freedom to choose the tools that suit best.
All this happens if you completely remove HTML and start building your pages from Javascript.
But why should I need a tool to read the DOM and generate any serialized format from it? If I have the DOM, the browser can write out some HTML?!?
I may have misunderstood, because this reply sounds a lot like what I've written and proposed, including my initial response to you. Sorry if I'm being a bit dense.
The function I called
domToJssimply takes the current DOM object and simplifies it. This gives me a JS object that can be persisted in some way. I am utterly lost as to why you think that this is somehow "static", especially given that I said that in this format (JS/JSON) it is easy to manipulate.But one obvious use for this is to create simple continuations: to save the current UI and return to it later. Is this the best way to do it? Who knows? It's a thought experiment.
As for the
jsToDomfunction, doesn't it do everything you list above? As for writing content generators, that is exactly where I'm going with this. My main point is that you don't need to load a bloated library for this. We can do it easily in vanilla code.Your examples are way more OOP than mine. I prefer to work with composable functions, not methods. But otherwise, they sound remarkably similar to what I'm arguing for. Are we actually agreeing here?
Also, do you have any links to discussion about scoped CSS and how this permits one to distinguish between global, class, and object (and why that matters)? Actually, any links to more information would be great.
Very interesting. I will think about this further and will try to get a clearer picture of what you're saying.
Hmm. I should also say (upon re-reading your comments for the fourth of fifth time), that I disagree about writing everything in JS, if that's what you're proposing. I am, frankly, quite strongly opposed to that.
HTML, CSS, and JS serve different purposes, and each is reasonably fit for purpose (and continually improving). I have some considerable experience over many years in generating HTML and CSS in JS. I believe it is a serious error. I have spent many painful hours trying to unravel ugly spaghetti code written by devs who thought that JS (or whatever) is all you need.
In my view, the separation of structure, presentation, and algorithmic behavior at the level of the code seen by humans is enormously beneficial. It vastly reduces cognitive load and makes reasoning about the code very easy. This is, for example, what is brilliant about JSX: it looks just like the code it generates, vastly reducing cognitive load.
But I find that some "programmers" (vs. "developers") are very biased toward "programming" languages and want to do everything in their code. I've done lots of this over the years – in PHP, C#, Java, Scala, etc. – and have regretted it every time. Frankly, I'd prefer mustache templates.
But have at it! I would not recommend that approach for the vast majority of developers. If anything, I think we need to be much more rigorous in our separation of structure, presentation, and behavior. But if it works for you, that's great.
Hy Charles, this was a long reply, so I´m unsure where to start...
I would be happy if this was true. You can use semantic HTML sinds HTML5, but this is far beyond what people do with HTML today. And this will limit you to fairly simple layouts. But people do much more with HTML today. See this example, which is quite typical: How wo build a traffic light in HTML?
See working example here
Now you would add some Javascript to make the lights blink. What a mess! Separation of concerns? Single source of truth? None! This is pure spaghetti code!
This is just to show, how things work in HTML, and many layouts are made like this. CSS has the same issue like HTML: Tasks are not clearly separated, it is used for styling, page formatting and even functional tasks. Everything is mixed up in this system, content, structural elements, semantic information, all connected with global scoped ID´s. That is the true reason why things are so complicated on the web.
As you might have seen, the DML-project provides all HTML-Tags as Javascript functions. Tag-Functions create new DOM elements and attach them directly to the DOM. So, a DIV can be created and formatted in a function like this:
This is a typical example of how I am using Javascript. "content1/2" are just loaded from a text file that contains some markdown or contain HTML or any other useful content.
The function "pageLayout()" provides everything related to the page-layout, div-containers, CSS-formatting and even some Javascript if necessary. Here it is easy to put everything in an externa library, or even use different page layouts depending on the device-type you use.
What it´s all about?
JsToDom can solve most of the issues we have with the HTML/CSS/JS-system. You do not need any ID´s anymore, no Web-components, no CSS-frameworks. JS can do all of this out of the box.
Let see how to build styled components with this approach, let say, you need a rounded button.
Simple, he? But these elements are still open for styling through conventional CSS, so you add only the functional CSS to the element, but allow styling also.
This is only the tip of an iceberg. If you follow this approach, you will enter a whole new world. Many things that make web development quite complicated simply dissolve.
The DML-homepage was created using this approach, but in a very early stage of this concept. So things are not that organized there. But some numbers about the concept may be interesting:
The page consists of two HTML-documents only, with 150 / 169 lines of HTML/JS. All styling is done in one CSS-file with less than 150 lines of CSS. That´s it! The page content is provided in a single content file for each sub-page and there are a number of images etc.. All navigation is generated automatically using very few manual definitions.
Ok, the page is not the most complex thing you can do, but it was done as a side project in some hours. I could not even imagine what a professional designer could do with this system.
So, for me there are very good reasons to follow this approach.
Is it relevant to make it an open source project?
All the code I write these days (I'm mostly retired) is open source and free to use. See the Creative Commons license on the Craft Code site. But really, unless you're using it commercially, I don't even need credit. Use any of it any way you want and have fun. Let me know if you make improvements.