Hey guys! This will be my first post in dev.to!
I wanted to ask if anyone of you have an experience on how to migrate a nosql dataset to a relational/sql dataset.
Thanks in advance!
Marc
Posted on
[question] How to migrate data from nosql to a relational database
For further actions, you may consider blocking this person and/or reporting abuse
Top comments (2)
It really depends on the data inside. The benefits of NoSQL surround the fact that you can work with unstructured data while relational databases require structure.
With that said, you could iterate through your records treating your keys as columns. An easy way to do that would be to pull them into a pandas DataFrame in Python. From there you could further clean/merge columns or do whatever you want or need to do to get it into the final structure you want in a SQL table (or multiple tables if you want to do database normalization).
Once you have the structure of your SQL database, there are different ways to put your data in. In S3/Redshift, you can do a
COPYquery and copy straight from a csv file you dump to. Otherwise, you could also use the SQL Alchemy package in Python (psycopg2 is an option if Postgres).Good luck!
Thanks man! Will try this out!