SirixDB - A versioned NoSQL data store for XML and JSON 🦁
Feel free to contribute on GitHub 💚
Hi all,
there's plenty of possibilties to work on a versioned document store, which preserves every revision of your data in a space-efficient manner. 🐬
SirixDB basically only stores changes between revisions, but any revison is indexed and can be reconstructed in O(n), whereas n is the number of nodes in the revision.
If you're eager to participate in #Hacktoberfest there are plenty of opportunities. For instance working on a just started frontend, the backend, a non-blocking REST API built with Vert.x, Kotlin and Keykloak, or a query optimizer (for instance to rewrite the AST based on cost-based rules). Or you might want to look into how to best horizontally scale SirixDB for instance via a distributed log (Apache BookKeeper?!) 😃
🐣 Frontend with Vue.js, TypeScript and D3.js
With the help of the community I just started a web frontend based on Vue.js, TypeScript and in the future D3.js a couple of days ago.
sirixdb
/
sirix-web-frontend
A web front-end for SirixDB based on Nuxt.js/Vue.js, D3.js and Typescript
Join us on Slack | Community Forum
Working on your first Pull Request? You can learn how from this free series How to Contribute to an Open Source Project on GitHub and another tutorial: How YOU can contribute to OSS, a beginners guide
SirixDB Web frontend - An Evolutionary, Versioned, Temporal NoSQL Document Store
Store and query revisions of your data efficiently
"Remember that you're lucky, even if you don't think you are, because there's always something that you can be thankful for." - Esther Grace Earl (http://tswgo.org)
Introduction
Discuss it in the Community Forum
This is the repository for a web frontend based on Vue.js, D3.js and TypeScript.
It'll provide several interaction possibilities to store, update and query databases in SirixDB. Furthermore the front-end will provide interactive visualizations for exploring and comparing revisions of resources stored in SirixDB based on different views.
Some ideas for…
There are a few issues I created, but basically we start completely from scratch, so we can use the best technologies which fit our needs:
Issues for the frontend gathered in our Community Forum
In general I'd love to provide a similar space filling approach (modified Sunburst-View) as the one I once developed for my master's thesis:
🚀 Backend written in Java and a non-blocking REST-API built with Vert.x and Kotlin
If you're more into backend stuff, you can either work directly on issues regarding SirixDB or work on Kotlin based stuff with Vert.x.
sirixdb
/
sirix
SirixDB facilitates effective and efficient storing and querying of your temporal data. Every commit stores a space-efficient snapshot. It is log-structured and never overwrites data. SirixDB uses a novel page-level versioning approach called sliding snapshot.
Download ZIP | Join us on Slack | Community Forum
Working on your first Pull Request? You can learn how from this free series How to Contribute to an Open Source Project on GitHub and another tutorial: How YOU can contribute to OSS, a beginners guide
SirixDB - An Evolutionary, Temporal NoSQL Document Store
Store and query revisions of your data efficiently
"Remember that you're lucky, even if you don't think you are, because there's always something that you can be thankful for." - Esther Grace Earl (http://tswgo.org)
We currently support the storage and (time travel) querying of both XML - and JSON-data in our binary encoding which is tailored to support versioning. Our index-structures and the whole storage engine has been written from scratch to support versioning natively. In the future, we might also support the storage and querying of other data formats.
Note: Work on a …
For instance Web Socket support in the non-blocking HTTP-Server:
Websockets to stream resource changes into the client / browser
#98
I'd put this into the rest-api module and build it with Vert.x and Kotlin.
OAuth2 authorization code flow:
OAuth2 auhtorization code flow for web applications
#96
Provide a configurable mechanism to switch between the authorization methods and implement the authorization code flow in SirixVerticle in the REST API.
Try to create a native image with the GraalVM (however Java 11 or soon 13 might not be supported, though :()
Create native images with the GraalVM
#77
For a fast startup time in containers... peek performance however might not be as good.
A Kotlin-based DSL for SirixDB:
Kotlin based wrapper APIs
#63
Therefore we also could create a nice DSL.
But in essence a basic version for the XdmNode-Transaction layer for instance would involve infix functions
trx moveTo 3 and alike, removing builders, just use default parameters...
However, I'd love to discuss this with Kotlin experts :-)
Maybe we can even convert the sirix-xquery bundle to Kotlin, as it provides a more high level API. That said, I'd love to change some things regarding the API:
Kotlin based wrapper APIs
#63
Therefore we also could create a nice DSL.
But in essence a basic version for the XdmNode-Transaction layer for instance would involve infix functions
trx moveTo 3 and alike, removing builders, just use default parameters...
However, I'd love to discuss this with Kotlin experts :-)
Query optimizations
Or if you would like to work on query optimizations, you can look into how to rewrite the AST produced by Brackit:
JSON with a database storage backend
#5
Hey Sebastian, sorry to bother you all the time, but maybe you can also give a good tip how you'd implement a binding for a native JSON storage backend. I'm currently finishing the work on a temporal JSON storage layer besides XML for my open source project (http://sirix.io) and would like to also use Brackit. Maybe I could also implement the interfaces you already provide (should be basically the same as for my XDM node transactions) and then implement some functions for opening a JSON resource...
Basically my JSON node layer looks like the one in Marklogic: https://docs.marklogic.com/guide/app-dev/json
We have plently of optimization potential here, as we currently don't take any storage inherent optimizations into account 👍
This will increase the performance of certain queries considerably.
A query looks like this:
let $doc := jn:open('database','resource', xs:dateTime('2019-04-13T16:24:27Z'))
let $statuses := $doc=>statuses
let $foundStatus := for $status in bit:array-values($statuses)
let $dateTimeCreated := xs:dateTime($status=>created_at)
where $dateTimeCreated > xs:dateTime("2018-02-01T00:00:00")
and not(exists(jn:previous($status)))
order by $dateTimeCreated
return $status
return {"revision": sdb:revision($foundStatus), $foundStatus{text}}
Have a look at Sebastian's thesis Separating Key Concernsin Query Processing
He worked under supervision of Theo Härder, who invented ACID transactions 👌
👾 Think about how to horizontally scale SirixDB
I've thought about it and would love to distribute the transaction log with Apache BookKeeper or Apache Pulsar (which uses Apache BookKeeper). It might have some advantages in comparison to Apache Kafka.
❤️ Happy coding
So, there's plenty of stuff to work on. It' a super diverse field. For sure beginners are invited to work on stuff as I'm myself a Frontend beginner for instance and I have not too much knowledge about query optimizers. However, I'm more than eager to learn... 👋
I hope the project is useful for some of you and I even got my first contributions after years of working alone, that's simply awesome. Thank you so much 😎
And if you like the project you might share it on twitter and so and and spread the word!? 🙈
Contribute on GitHub SirixDB and GitHub SirixDB Web Frontend 💚
Have a nice day and a great #Hacktoberfest
Johannes
Top comments (0)