Run a DynamoDB instance locally with Node.js

Node.js + MongoDB is extremely popular when you need a fast and convenient development cycle but don't need to care much about the performance even though MongoDB is quite performant. Likewise, we can use NodeJS with DynamoDB which is scalable, affordable and also frees up your time from configuring database clusters.

This post will explain how you can set up a local DynamoDB instance with Node.js with or without an AWS account.

I assume you have the latest version of Node.js installed. I use a generator tool called express-generator, which creates an express application skeleton without any hassle.

DynamoDB Setup

• Download the latest version of DynamoDB
• Unzip the file contents into a new folder. For example, we can call it dynamodb

cd dynamodb
java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar -sharedDb

If you're greeted with the following log on your console, you have successfully started the DynamoDB instance locally. Good work!

Initializing DynamoDB Local with the following configuration:
Port:   8000
InMemory:   false
DbPath: null
SharedDb:   true
shouldDelayTransientStatuses:   false
CorsParams: *

Express App Setup

We will use the express-generator package to scaffold an express app quickly.

express dynamo-local
cd dynamo-local
npm install
npm start

Now, go to localhost:300. Awesome, you have set up the express app successfully.

Configure AWS Locally

Before we start creating tables, we need to configure AWS via CLI. Since we will not be using an AWS account, it's pretty easy to get going.

Install AWS CLI. Then run aws --version to check if it's properly installed. Then run aws configure and pass any string as access key and secret access key. For example,

AWS Access Key ID [None]: dunder
AWS Secret Access Key [None]: mifflin
Default region name [None]: local
Default output format [None]: json

Let's get to work

Awesome!. We have set the ball rolling. Now we can create a table on our very own locally brewed DynamoDB instance.

Before we create a table, let's see if everything working properly.
Run aws dynamodb list-tables --endpoint-url http://localhost:8000
and you'll be greeted with an object which has a key named TableNames on your console.

{
"TableNames": []
}

The Table Creation requires not only setting a name, but also the primary key, which identifies table items. No two items share a key. DynamoDB uses two types of primary keys. Partition Key − This simple primary key consists of a single attribute referred to as the “partition key.” Internally, DynamoDB uses the key value as input for a hash function to determine storage. Partition Key and Sort Key − This key, known as the “Composite Primary Key”, consists of two attributes.

*The partition key and

*The sort key.

DynamoDB applies the first attribute to a hash function, and stores items with the same partition key together; with their order determined by the sort key. Items can share partition keys, but not sort keys.

Haha, okay. I hope you get the drift and I will stop with my theoretical babblings. Let's create our first table. Create a javascript file with the following code snippet. For example, in this example, I named my file scrantonTable.js.

//scrantonTable.js
const AWS = require("aws-sdk");
AWS.config.update({
  region: "local",
  endpoint: "http://localhost:8000"
});
var dynamodb = new AWS.DynamoDB();
var params = {
    TableName : "Scranton",
    KeySchema: [
        { AttributeName: "id", KeyType: "HASH"},  //Partition key
],
    AttributeDefinitions: [
        { AttributeName: "id", AttributeType: "N" },
],
    ProvisionedThroughput: {
        ReadCapacityUnits: 5,
        WriteCapacityUnits: 5
    }
};
dynamodb.createTable(params, function(err, data) {
    if (err) {
        console.error("Error JSON.", JSON.stringify(err, null, 2));
    } else {
        console.log("Created table.", JSON.stringify(data, null, 2));
    }
});

Run node scrantonTable.js. Viola!

Pretty cool right?. Additionally, Run
aws dynamodb list-tables --endpoint-url http://localhost:8000
to double-check.

{
"TableNames": [
"Scranton"
]
}


Now that our table is set-up, we can feed some data into it using a JSON file.

//scrantonData.json
[
  {
    "id": 1,
    "type": "Sales",
    "name": "Dwight Schrute",
    "description": "Battlestar Galatica"
  },
  {
    "id": 2,
    "type": "Accounting",
    "name": "Kevin Malone",
    "description": "M&Ms"
  },
  {
    "id": 3,
    "type": "Reception",
    "name": "Erin",
    "description": "Glee party"
  }
]

To load this data into the table, we'll need a javascript file with which we can use the PutItem method of DynamoDB.

//putScrantonData.js
const AWS = require("aws-sdk");
const fs = require('fs');
AWS.config.update({
    region: "local",
    endpoint: "http://localhost:8000"
});
var docClient = new AWS.DynamoDB.DocumentClient();
console.log("Importing Scranton into DynamoDB. Please wait.");
let scranton = JSON.parse(fs.readFileSync('scrantonData.json', 'utf8'));
scranton.forEach(function(scran) {
  console.log(scran)
var params = {
        TableName: "Scranton",
        Item: {
            "id": scran.id,
            "type": scran.type,
            "name": scran.name,
            "description": scran.description
        }
    };
docClient.put(params, function(err, data) {
       if (err) {
           console.error(err);
       } else {
           console.log("PutItem succeeded:", scran.name);
       }
    });
});

run node putScrantonData.js.

Read More

Well, that's it, folks!. Thanks for reading. I'll be back with something exciting very soon.

DynamoDB Documentation for Node.js.

Reference 1
Reference 2

Comments

[livijohnson]

Hi Ajin,

Thanks for the great post. I'm so new to JSON and DynamoDB, so this was really helpful. I just got hung up in one spot. I was able to create the table, but had trouble loading the data. I saved scrantonData.js, but the code to parse and load is looking for a json file. I resaved it as a json, but the parser doesn't like the contents of the file. Errored out once it reached the "i" in "id". Got any tips for me?

[Ajin Kabeer]

Hello Livi,

Thanks for reaching out to me. It's actually my mistake. Firstly, the file should have been saved as a .json file and secondly with proper JSON syntax. I have updated former scrantonData.js into scrantonData.json file with proper syntax.

Copy the JSON file to your IDE and then create a new file called putScrantonData.js and copy the last code snippet and run node putScrantonData.js

If you run into any trouble, I'll be here.

[livijohnson]

Worked perfectly, thank you!

[Ajin Kabeer]

Awesome :)

[Jose Jaimes]

Hi Ajin,

Is it possible to declare a nested field in a table? Ex:

{
"id": 3,
"type": "Reception",
"name": "Erin",
"description": "Glee party",
"nestedField": [
{
"field1": "val1",
"field2": "val2",
},
{
"field1": "val3",
"field2": "val4",
}

]
}

[Mukash Wasti]

exactly what i was looking for
Thanks