Schema Validation in Amazon DynamoDB

Introduction

Validating data in DynamoDB is crucial for maintaining data integrity, ensuring application reliability, and reducing errors. Unlike relational databases, DynamoDB does not enforce strict schema constraints, giving you the flexibility to store a variety of data formats. However, this flexibility means you must take responsibility for validating data before it is written to the database. In this article, we’ll go through an approach to validate data in DynamoDB using AWS Lambda.

Why Data Validation Matters in DynamoDB

Without strict schema enforcement, data in DynamoDB can end up in inconsistent or incorrect formats. If validation is neglected, problems may arise, such as:

• Application Errors: Missing or incorrectly formatted data can lead to errors and application crashes.
• Data Inconsistencies: Querying inconsistent data formats can complicate analytics and reporting.
• Scalability Issues: Unvalidated data may lead to inefficient data structure and increased storage costs.

To avoid these issues, validating data before saving it to DynamoDB is essential.

How to validate schema using AWS Lambda

In serverless architectures, AWS Lambda is a common approach for server-side validation. By implementing validation within Lambda functions, you can control data before it reaches DynamoDB.

Here’s how you can incorporate validation in a Lambda function with Node.js:

const Joi = require('joi');

// The schema definition using Joi
const tableSchema = Joi.object({
    id: Joi.string().required(),
    name: Joi.string().required(),
    email: Joi.string().pattern(new RegExp("^[\\w-\\.]+@([\\w-]+\\.)+[\\w-]{2,4}$")).required(),
    phone: Joi.string().optional()
})

const validateObject = async (data) => {
    try{
        const value = await tableSchema.validateAsync(data);
        return value;
    }catch(error){
        return {
            body: JSON.stringify({ message: "Invalid request body", error: err.message || err }),
        }
    }
}

module.exports.lambdaHandler = async (event, context) => {
  try{
    const body = event.body
    const result = await writeToDynamodbTable(body);
    return result;
    }
  }catch (err) {
    return {
        statusCode: 400,
        body: JSON.stringify({ message: "Invalid request body", error: err.message || err }),
    };
  }
}

const writeToDynamodbTable = async (data) => {
  try{
    const value = await validateObject(data)
    const params = {
        TableName: 'YourTableName',
        Item: {
          id: value.id,
          name: value.name,
          email: value.email,
          phone: value.phone
        },
        ConditionExpression: "attribute_not_exists(id)"
  };;
    const res = await dynamodbClient.send(new PutCommand(params));
    return res;
  }catch(error){
    throw error
  }
};

In the code snippet above we:
1- Define the schema using Joi Schema
2- Validate our object in the validateObject function
3- Configure our params object
3- Send request to DynamoDB using AWS Lambda

SAMPLE ERROR:

If we send a mismatch object like this:

{
  "body": {
    "id": "value1",
    "name": "value2",
    "email": "[e-mail]"
  }
}

Note: phone is not included in the object but it is a required key in the schema we defined.
We will get an error like

{ statusCode: 400,
  body: { message: "Invalid request body",
          error: ""phone" is required"
        }
}

Conclusion

Validating data in DynamoDB is a fundamental step to ensure data quality and avoid issues down the line. Whether you choose to validate data on the client, the server, or both, using a structured approach with reusable validation schemas will streamline the process. By following best practices, you can improve the reliability and performance of your DynamoDB-backed applications.

——————————————

Follow my social handles for more articles:
Click and follow on

• LinkedIn
• Twitter
• Dev
• Medium