DEV Community

Cover image for Automating Image Thumbnails with AWS Lambda and S3: A Step-by-Step Guide
Nada Ahmed
Nada Ahmed

Posted on

Automating Image Thumbnails with AWS Lambda and S3: A Step-by-Step Guide

Introduction:

I recently completed an exciting project automating image thumbnail generation using AWS Lambda and Amazon S3. The best part? It's entirely serverless, meaning I didn’t have to worry about managing infrastructure—just focused on building the solution. This workflow is ideal for dynamically processing large-scale images without any overhead!

In this guide, I’ll walk you through the steps to build this system from scratch, sharing insights and challenges I faced along the way. Let's get started!

Step 1: Setting Up the S3 Buckets
To begin, we need two Amazon S3 buckets:
-Source Bucket: This is where users will upload the original images.
-Destination Bucket: Thumbnails will be stored here after Lambda processes the images.

Here’s a quick breakdown:

  1. Go to the AWS Management Console and create the two S3 buckets.
  2. Configure access permissions, ensuring the correct bucket policies are applied.

Image descriptionFigure1 Access S3 console.
Image descriptionFigure 1.1 create bucket.
Image descriptionFigure 2 Complete create bucket

Step 2: Configuring IAM Role for Lambda Execution

Security is crucial. We need a role that allows Lambda to:

  • Access the S3 buckets for reading/writing images.
  • Log events to CloudWatch for monitoring.

I used a managed policy (AmazonS3FullAccess) alongside custom permissions for logging in CloudWatch.

  1. Create and attach the necessary policies for S3 and CloudWatch.

Image descriptionFigure 3 IAM dashboard
Image descriptionFigure 3.1 Create policy
Image descriptionFigure 3.1.1 Complete create policy
Image descriptionFigure 3.1.2 Complete create policy
Image descriptionFigure 3.1.3 Policy created

  1. In the IAM Console, create a new role with basic Lambda execution permissions.

Image descriptionFigure 4 Create role

Image descriptionFigure 4.1 Choose service
Image descriptionFigure 4.1.1 Select lambda service

Image descriptionFigure 4.1.2 Add permission

Image descriptionFigure 4.1.3 Review
Image descriptionFigure 4.1.4 Complete configure role
Image descriptionFigure 4.1.5 Complete configure role
Image descriptionFigure 4.1.6 Role created

Step 3: Creating the Lambda Function

Now, let's move on to the Lambda function itself.

  • Navigate to AWS Lambda, and click "Create Function."
  • Select Python 3.9 as the runtime (you can use other languages, but I used Python for this project).

The function listens for an S3 event, processes the image, generates a thumbnail using the Pillow library, and uploads it to the second bucket.

Image descriptionfigure 5 Lambda function console
Image descriptionFigure 5.1 Create lambda function
Image descriptionFigure 5.1.1 Complete create lambda
Image descriptionFigure 5.1.2 Add code to lambda function

Here’s a snippet of the code I used:

import boto3
import os
import sys
import uuid
from urllib.parse import unquote_plus
from PIL import Image

s3_client = boto3.client('s3')

def resize_image(image_path, resized_path):
    with Image.open(image_path) as image:
        image.thumbnail(tuple(x / 2 for x in image.size))
        image.save(resized_path)

def lambda_handler(event, context):
    for record in event['Records']:
        bucket = record['s3']['bucket']['name']
        key = unquote_plus(record['s3']['object']['key'])
        tmpkey = key.replace('/', '')
        download_path = '/tmp/{}{}'.format(uuid.uuid4(), tmpkey)
        upload_path = '/tmp/resized-{}'.format(tmpkey)
        s3_client.download_file(bucket, key, download_path)
        resize_image(download_path, upload_path)
        s3_client.upload_file(upload_path, '{}-resized'.format(bucket), 'resized-{}'.format(key))
Enter fullscreen mode Exit fullscreen mode

Step 4: Adding an S3 Trigger

The Lambda function is triggered automatically whenever an image is uploaded to the source bucket.

  • In the Lambda function's configuration, scroll down to "Add Trigger" and select S3.

Image descriptionFigure 6 Add trigger (s3 source bucket)
Image descriptionFigure 6.1 Complete add trigger
Image descriptionFigure 6.1.1 Function overview

  • Configure it to trigger on PUT events, meaning any new image upload will start the function.

Image descriptionFigure 6.2 create test event

Here's example of test event

{
  "Records": [
    {
      "eventVersion": "2.0",
      "eventSource": "aws:s3",
      "awsRegion": "us-east-1",
      "eventTime": "1970-01-01T00:00:00.000Z",
      "eventName": "ObjectCreated:Put",
      "userIdentity": {
        "principalId": "xxxxxxxxxx"
      },
      "requestParameters": {
        "sourceIPAddress": "127.0.0.1"
      },
      "responseElements": {
        "x-amz-request-id": "EXAMPLE123456789",
        "x-amz-id-2": "EXAMPLE123/5678abcdefghijklambdaisawesome/mnopqrstuvwxyzABCDEFGH"
      },
      "s3": {
        "s3SchemaVersion": "1.0",
        "configurationId": "testConfigRule",
        "bucket": {
          "name": "photo-project1",
          "ownerIdentity": {
            "principalId": "xxxxxxxx"
          },
          "arn": "arn:aws:s3:::photo-project1"
        },
        "object": {
          "key": "cute-kitten-4k-im-1600x900.jpg",
          "size": 1024,
          "eTag": "0123456789abcdef0123456789abcdef",
          "sequencer": "0A1B2C3D4E5F678901"
        }
      }
    }
  ]
}

Step 5: Testing the Solution

Now, upload a sample image to the source S3 bucket. The Lambda function should:

  1. Automatically get triggered.
  2. Process the image and create a thumbnail.
  3. Store the thumbnail in the destination bucket.

Image descriptionFigure 7 upload two photos in source bucket
Image descriptionFigure 7.1 Two photos after resizing
Image descriptionFigure 7.2 Photo resized
Image descriptionFigure 7.3 Photo resized

💡 Challenge:
I initially encountered issues with image formats and timeouts. Optimizing the image processing code to handle larger images and using the appropriate timeout settings resolved these.

Step 6: Monitoring with CloudWatch

Once the function is up and running, you can use CloudWatch to monitor the executions and logs. It’s essential for checking if the function ran successfully or troubleshooting issues like failures.

Conclusion:

This project demonstrated the power of serverless architecture and how easily AWS services like Lambda and S3 can handle dynamic tasks such as image processing at scale. By leveraging AWS’s scalability, this solution is efficient and cost-effective, especially for large-scale workloads. I also gained valuable insights into optimizing Lambda functions and managing event-driven architectures.

Key AWS Services:

  • AWS Lambda
  • Amazon S3
  • IAM for security
  • CloudWatch for monitoring

If you're working on automating processes in the cloud, AWS Lambda is a game-changer! Feel free to try this solution and let me know how it goes.

Top comments (0)