M. Abdullah Bin Aftab

Posted on Oct 16 • Edited on Oct 19

How to Import Pandas(library) in AWS Lambda Functions - AWS Lambda Layers

#aws #lambda #python #cloudcomputing

Imagine you need to run a Python script on the AWS Lambda function and you get this error 🔻

{
  "errorMessage": "Unable to import module 'lambda_function': No module named 'pandas',
  "errorType": "Runtime.ImportModuleError"
  ...
}

Don't worry this is a common error and I am not going to make this long

How do you import Pandas in AWS Lambda Functions?

There are several ways but I am going to give you the easiest way to import pandas in AWS Lambda Function is to add Lambda Layer 🥪

What is AWS Lambda Layer?

It is a 🧀 cheese layer in Lambda Function containing additional code like libraries, dependencies, etc.

In Simple Words

AWS Lambda Layers are like building blocks for your functions. Imagine you need extra tools (like the Pandas library) to complete a project.

Instead of packing all those tools inside every single project (which wastes space and time), AWS allows you to create layers of tools (libraries, dependencies, or shared code).

These layers sit outside your main function but are always available when your function needs them.

In short, Lambda Layers help you:

Separate the main logic from the extra libraries to save space in your code. You can reuse libraries and code across multiple Lambda functions.

Easily update or manage your dependencies without changing your core function code.

Think of layers as an extra storage box attached to your Lambda function, holding everything your function needs to work smoothly. You can stack multiple layers on your function without cluttering your main code.

Steps to add Lambda Function Layer and Import Pandas

It takes only 3 steps to run Pandas in your Lambda Function successfully

Step 1 - Open Lambda Function through your AWS Management Console

As you can see we have an option Layers under the name of our Lambda Function, in my case, it's "import-pandas-function" and the Layers count is 0

Step 2- Add Script in your AWS Lambda Function

This step is further divided into two steps because we need to add a Python script that contains some Pandas code and write a test event in JSON to verify whether the code is running correctly.

2.1 - Add Python script - you can copy this code 🡻

import json
import pandas as pd

def lambda_handler(event, context):
    data = event.get('data', [])

    df = pd.DataFrame(data)

    if not df.empty:
        mean_value = df['column_name'].mean()

        result = {
            "mean_value": mean_value,
            "data_shape": df.shape,
            "summary": df.describe().to_dict()
        }
    else:
        result = {
            "message": "Empty DataFrame"
        }

    # Return the response
    return {
        'statusCode': 200,
        'body': json.dumps(result)
    }

2.2 - Add Test script in Json in the test tab - you can copy this code 🡻

{
  "data": [
    {"column_name": 10, "other_column": "A"},
    {"column_name": 20, "other_column": "B"},
    {"column_name": 30, "other_column": "C"},
    {"column_name": 40, "other_column": "D"}
  ]
}

Press the test button you probably got the 🔴error:-
"errorMessage": "Unable to import module 'lambda_function': No module named 'pandas', "errorType": "Runtime.ImportModuleError" ...

Step 3 - Add AWS Lambda Layer to Successfully run the Pandas in your Code

Scroll down to your Lambda Function, you probably can see the "Layers" separate section at the end of the page

3.1 - Click "Add a Layer"

After Clicking the "Add a Layer" you can see the page which has a couple of sections "Function runtime settings" and "Choose a layer"

3.2 - Click "AWS layers"

You can see three options in the "Choose a layer" section click the "AWS layers".

3.3 - Choose "AWS layers"

After selecting the AWS layers you can see the dropdown under "AWS layers".

3.4 - Choose "AWS layers" and "Version"

In a dropdown of "AWS layers" select -> AWSSDKPandas-Python312

In a dropdown of "Version" select -> 13(select the most one)

click the "Add" button

3.5 - Make sure the "Function Overview"

When your page is directed to the function overview you can see the layer is added below the function name "import-pandas-function"

Step 4 - Test the Function

You've successfully got the Response "statusCode": 200

{
  "statusCode": 200,
  "body": "{\"mean_value\": 25.0, \"data_shape\": [4, 2], \"summary\": {\"column_name\": {\"count\": 4.0, \"mean\": 25.0, \"std\": 12.909944487358056, \"min\": 10.0, \"25%\": 17.5, \"50%\": 25.0, \"75%\": 32.5, \"max\": 40.0}}}"
}

Keep Coding 😎

DEV Community