Some time ago I've discovered a database called ThingsDB. I was curious about it and I did some reading. I've discovered they support TCP connectivity but they didn't have driver for some specific platforms therefore I've developed a driver for it for javascript and for php.
When I worked on javascript driver I've realized, it would be possible to use ThingsDB directly from frontend without any backend or middleware. You can open websocket (TCP) connection from browser, so I reached out to the authors of ThingsDB and they added support for websocket (available from ThingsDB version 1.6). This way my javascript driver can be used from frontend (browser) and also from javascript based backend (ex. node.js). I wrote article here about my php driver where I have received interesting feedback. People wanted to see more of the potential of ThingsDB. Based on that I chose not to write article about my javascript driver right after I've finished it, but I've decided it would be best to make demo.
To understand basics of ThingsDB and this demo I suggest you to read continuosly as I explain specific features on the way. I expect you are familiar with programming in general, at least basics. And maybe some javascript and jQuery.
ThingsDB structure introduction
First things first. Let me shortly explain the structure.
ThingsDB contains collections. Collection contains data, procedures, tasks, data types and enums. There is also prior collection (scope) @thingsdb which contains user access accounts and it can also contains procedures and tasks. Lastly there is @node scope which is not important right now.
- Data - persistent stored data
- Procedure - like function, can have arguments and it can return value
- Tasks - planned events, like cron
- Data types - like classes, it can have properties and methods
- Enums - enumerators
All named things like data, procedures, tasks, data types and enums are defined by developers implementing ThingsDB. New instance of this database only contains empty collection called @:stuff and user account admin. I use this collection as main one for this demo.
Talking to yourself
When you execute query or run procedure on ThingsDB, you have to specify on which collection it will run. That can be sometimes limiting and if you have a need to execute query or run procedure on another collection, there is a way how to achieve that. There is a module called thingsdb (book, GitHub) which allows you to access another collection from collection as specific user. My demo use this feature heavily when dealing with user accounts which is reason I mention it here. I have installed this module as explained in manual.
I'll explain permissions little bit later but fyi: User account I've made for this module has permissions Query, Change, Grant on collection @thingsdb and Change, Grant on collection @:stuff.
User accounts
I chose to use only ThingsDB and that means I had to use their user accounts. I had to deal with registration and login which was little bit tricky because of the absence of backend. Of course I could use some third party authentification server (auth0, etc.), but I didn't want to rely on anything else.
In the case somebody wants to implement 3rd party auth system, you can do HTTP requests from ThingsDB with Request module (book, GitHub).
To allow users to register I needed some user account to communicate with ThingsDB and execute the registration. But required credentials for this account would be published in javascript code which doesn't sound very secure. I didn't want to deal with all security problems but I wanted to implement at least the simple ones. ThingsDB support granting permissions for each user account against each collection specifically. Available permissions to grant are Query, Change, Grant, Join and Run.
- Query - query ThingsDB
- Change - allow execution of code which does change of data
- Grant - allow granting permissions
- Join - allow join rooms
- Run - run stored procedure
I'm not able to use Query at all. Because with this command you can execute anything on ThingsDB and opening this to the client browser pose huge security problem. The path was clear, I had to use procedures and just allow Run for client.
Important information to know is the user accounts doesn't have only password but also access tokens (with expiration if needed).
Prepare ThingsDB for registration
I've created collection @:auth and user account with name aa (auth account) and I gave him permission Run over this collection. Collection @:auth contains only one procedure called register. All this means, the user aa can do only one thing which is to run procedure called register. Therefore his access token can be published.
Procedure register does create new account and grant required permissions. The code looks like this:
new_procedure('register', |email, password| {
if (email.len() == 0 || password.len() == 0 || !is_email(email)) {
raise('required values not provided');
};
thingsdb.query('@t', "
if (has_user(email)) {
raise('email already registered');
};
new_user(email);
set_password(email, password);
grant('@:stuff', email, RUN | CHANGE);
", {
email:,
password:,
});
nil;
});
I guess this is your first time seeing code from ThingsDB. It is familiar to another programming languages just with slight changes. What the procedure does:
- Accepts two arguments
- Verify arguments email and password
- Use module
thingsdbexplained in section Talking to yourself to run query- Check if email is already registered
- Create new user account and set password
- Grant required permissions
email:, can be a little bit confusing but it's a shorthand when you want to pass variable to argument and argument and variable has the same name.
@t is shortcut for @thingsdb scope.
Frontend implemention of registration
With everything ready at ThingsDB side I've created simple website with registration form and few lines of javascript. The code snippet which manages to run procedure inside of ThingsDB looks like this:
const thingsdb = new ThingsDB();
thingsdb.connect()
.then(() => thingsdb.authToken(localStorage.getItem('aa')))
.then(() => thingsdb.run('@:auth', 'register', [
$('#email').val(),
$('#password1').val()
]))
- It performs authentification with token (for
aauser account) - Runs procedure
register
I keep access token of user aa in browser localStorage.
To see whole implementation look here:
Login
After user is able to register, next step was to implement login action. For login password is required but it would be not very safe to store user password in browser. The solution is to generate access token (with expiration) after login and return it to client, where it can be stored in browser (ex. sessionStorage). So I've created procedure in @:stuff collection where registered user account has required permissions.
new_procedure('login', || {
email = user_info().load().name;
if (is_email(email)) {
thingsdb.query('@t', "new_token(email, datetime().move('days', 1));", {email: })
.then(|token| token);
};
});
Creation of token has to be called on @thingsdb scope, in that case I use thingsdb module again. The javascript code snippet to call this procedure looks like this:
const thingsdb = new ThingsDB();
thingsdb.connect()
.then(() => thingsdb.auth($('#email').val(), $('#password').val()))
.then(() => thingsdb.run('@:stuff', 'login'))
.then(token => {
sessionStorage.setItem('token', token);
window.location.href = './overview.html';
})
Obtained access token is stored in sessionStorage.
Here you can check whole login page which contains login form and required javascript code:
Overview
After login user is redictered here where he has some account actions and list of his Todos. That required to specify structure, how Todo data will be stored and for this purpose we can use data types. I created Todo type which has name, user_id and items. Item type has description, checked status and Todo reference. Connection between Todo and Item is made with both ways relation (book, docs). Both types are defined in @:stuff collection.
new_type('Item');
new_type('Todo');
set_type('Item', {
description: "'str',"
checked: 'bool',
todo: 'Todo?',
});
set_type('Todo', {
name: 'str',
items: '{Item}',
user_id: 'int',
});
mod_type('Item', 'rel', 'todo', 'items');
In this piece of code you can see how the types are made, what properties with data types does they have and set up of relation between them.
But this is just definition. We need to store Todos somewhere. For that we create property directly on collection @:stuff like this. Without the dot it would be just variable and it won't be persistent.
.todos = set();
Now after the data structure is ready, let's go through each action.
Todos
Upon loading of overview page, request to load Todos of users to ThingsDB is made. First we need a procedure on @:stuff collection which returns list of Todos:
new_procedure('list_todos', || {
user_id = user_info().load().user_id;
.todos.filter(|t| t.user_id == user_id);
});
Filter is function available to call on set.
Now we can call this procedure with javascript code snippet like this (processing received data is omitted):
const thingsdb = new ThingsDB();
thingsdb.connect()
.then(() => thingsdb.authToken(sessionStorage.getItem('token')))
.then(() => thingsdb.run('@:stuff', 'list_todos'))
.then(todos => { })
You can check the whole implementation here:
Password change
For this action I've created procedure update_password which requires to use thingsdb module again. User accounts are stored in @thingsdb scope.
new_procedure('update_password', |password| {
email = user_info().load().name;
if (is_email(email)) {
thingsdb.query('@t', 'set_password(email, password);', {
email:,
password:,
});
};
});
I use html dialog tag to enter a new password and the javascript code snippet to handle it is very simple:
thingsdb.run('@:stuff', 'update_password', [$('#password1').val()])
I don't have to call authToken again because websocket connection is still open from the request to load Todos.
You can check the whole implementation here:
Delete account
Procedure for this action removes not only user account but also his Todos. It looks like this:
new_procedure('delete_user', || {
email = user_info().load().name;
if (is_email(email)) {
.todos.remove(|todo| todo.user_id == user_id);
thingsdb.query('@t', 'del_user(email);', {email: });
};
});
Remove is another function which can be called on set.
I had to use thingsdb module again. User accounts are stored in @thingsdb scope.
Call of this procedure can be done easily with javascript code snippet:
thingsdb.run('@:stuff', 'delete_user')
I don't have to call authToken again because websocket connection is still open from the request to load Todos.
Look at the whole implementation here:
Create Todo
User need a way to create new Todo. For that reason I made page new_todo and overview contains link to it. Form to create todo consist of todo name and items (descriptions). I decided to store new Todo with items in two steps, because originally I wanted to allow editing of Todo (which in the end didn't happen). Therefore I've created two new procedures.
new_procedure('create_todo', |name| {
t = Todo{
name:,
user_id: user_info().load().user_id,
};
.todos.add(t);
t.id();
});
new_procedure('add_todo_items', |todo_id, items| {
todo = thing(todo_id);
if (todo.user_id != user_info().load().user_id) {
raise('Not yours');
};
todo.items.clear();
items.each(|i| {
item = Item{
checked: false,
description: "i,"
};
todo.items.add(item);
});
});
First procedure to create todo returns it's id and second procedure deletes all items and adds new ones. I think if you read until here you are already getting hang of it and I don't have to explain .todos.add() or items.each() (set add, thing each).
What is new here is thing(todo_id). You can get reference to any thing (thing is like instance of class/data type) from collection by id. You don't have to know where is stored, you can just get it. Thing has assigned id when is stored persistently.
To perform defined action you just have to call it with javascript code snippet:
thingsdb.run('@:stuff', 'create_todo', [$('#name').val()])
.then((todo) => thingsdb.run('@:stuff', 'add_todo_items', [
todo, items.length ? items.map(function () {
return $(this).val();
}).get() : []
]))
Look at the whole implementation here:
Todo detail
Overview page shows list of user Todos. By clicking on it user is redirected to page where he can see Todo items, change their status and delete whole Todo list.
Load Todo data
To load one specific Todo I've created new procedure:
new_procedure('list_todo', |todo_id| {
todo = thing(todo_id);
if (todo.user_id != user_info().load().user_id) {
raise('Not yours');
};
return todo, 2;
});
Now you are propably asking why there is return todo, 2;? With return you can set depth of data you want to return. With number 2 here returned data contains not only Todo itself, but also Items the Todo has relation with.
Because Todo id is passed as uri get parameter, the javascript code snippet to call this procedure looks like this:
thingsdb.run('@:stuff', 'list_todo', [
parseInt(location.search.match(/id=(\d+)/)[1])
])
Look at the whole implementation here:
todo.html
todo.js
Change Todo item status
I render todo items as checklist, so to change status of item I've created new procedure:
new_procedure('mark_item', |item_id, checked| {
item = thing(item_id);
if (item.todo.user_id != user_info().load().user_id) {
raise('Not yours');
};
item.checked = checked;
nil;
});
Because you can also uncheck, not only check item, javascript code snippet has to be like this:
thingsdb.run('@:stuff', 'mark_item', [
parseInt(this.id),
$(this).is(':checked')
])
Look at the whole implementation here:
todo.html
todo.js
Delete Todo
If we want to delete Todo, we don't have to delete items because they are not stored separately. If Todo is removed, no other reference exists for its items and they are automatically removed.
new_procedure('delete_todo', |todo_id| {
todo = thing(todo_id);
if (todo.user_id != user_info().load().user_id) {
raise('Not yours');
};
.todos.remove(todo);
});
Now the javascript code snippet is simple:
thingsdb.run('@:stuff', 'delete_todo', [
parseInt(location.search.match(/id=(\d+)/)[1])
])
Look at the whole implementation here:
todo.html
todo.js
Installation guide
To simplify usage of this demo you can run ThingsDB in docker described in README.
Next simply open install.html which creates everything required in this ThingsDB instance and store access token of aa user to localStorage.
That's it. I hope I gave you basic insight into this technology. If you like my work you can buy me a tea.
No AI was used to generate this content, only the cover picture.
Top comments (0)