DEV Community

Cover image for What is RocksDB (and its role in streaming)?

What is RocksDB (and its role in streaming)?

Mark Andreev on May 13, 2024

RocksDB, a high-performance database, is a hidden gem in the tech industry, often overlooked by developers. I’m excited to delve into the intrica...
Collapse
 
chester89 profile image
Gleb Chermennov

Nice article.
I have to point out that CockroachLabs stopped using RocksDb as its storage engine

Collapse
 
mrkandreev profile image
Mark Andreev

Thank you.

In my perspective, this story serves as a great illustration of using RocksDB as an initial step.

Collapse
 
franckpachot profile image
Franck Pachot

One big value of RocksDB is how it is adaptable and customizable, which make it a great foundation for all kind of databases. Here are some improvements made for YugabyteDB:
youtu.be/WwsiDu-qmFU?si=_mOTB3n1cK...

Collapse
 
tropicaldroid profile image
haris

Brilliant intro and context into RocksDB. I had one question about scaling; traditional relational databases come with horizontal scaling/sharding, etc. Given that RocksDB is embedded in the application, how does scaling work?

Collapse
 
mrkandreev profile image
Mark Andreev

RocksDB primarily concentrates on managing data within a single node. While it is not inherently a replicated system, it does offer auxiliary functions that allow users to construct their own replication system using RocksDB as a foundation. github.com/facebook/rocksdb/wiki/R...

So all scaling problems are end user problems.

Collapse
 
foookinaaa profile image
Yuliya Fokina

Thanks for your post! Does it makes sense to integrate RocksDB in python app?

Collapse
 
mrkandreev profile image
Mark Andreev

Yes, it is makes sense because RocksDB simplify persistence of your data. It is hard to maintain delete & update operations on one's own. The biggest challange is compaction ( github.com/facebook/rocksdb/wiki/C... ) which is required to remove inactive rows.

Collapse
 
daramasala profile image
Doron Tohar

Depends. I believe for most use cases it will be an overkill. sqlite, mysql, postgres, mongodb will probably be easier to work with.

Collapse
 
jakehov profile image
jakehov

Thank you!

I would like to share crate for rust docs.rs/rocksdb/latest/rocksdb/

Collapse
 
mrkandreev profile image
Mark Andreev • Edited

I would like to mention the limitation of this crate

The underlying RocksDB does allow column families to be created and dropped from multiple threads concurrently. But this crate doesn't allow it by default for compatibility. If you need to modify column families concurrently, enable crate feature called multi-threaded-cf, which makes this binding's data structures to use RwLock by default. Alternatively, you can directly create DBWithThreadMode without enabling the crate feature.
ref: docs.rs/crate/rocksdb/latest

Collapse
 
vewibep profile image
vewibep

Nice dive. Thank you!