From my previous articles, you’ve seen several ways to use AWS Serverless Lambda functions, both for API-based integration in non-generative AI use cases and for generative AI use cases integrated with Amazon Bedrock. In all these examples, I used synchronous invocation for Lambda integration.
In this article, let’s explore asynchronous invocation supported by AWS Lambda. When invoked asynchronously, Lambda returns a 202 status code, indicating that the call was successful and the event has been passed for asynchronous processing. However, it’s important to plan for monitoring and managing the subsequent processing steps. Depending on your use case, you may need to track whether the processing was completed successfully or if it failed.
For example, consider an address book application where you add a new address record using an asynchronous call. While Lambda might successfully pass the data to the backend for processing, the operation could fail in later steps. In such cases, you would want to be informed about the failure. One effective approach is to configure a Lambda destination to capture failed events.
In November 2019, AWS introduced support for destinations in Lambda. This feature allows you to choose where to send invocation results, including destinations like another Lambda function, SQS, SNS, or EventBridge.
More recently, in November 2024, AWS announced support for using S3 as a destination for failed events. This enhancement allows you to specify an S3 bucket where all failed events can be stored for further analysis.
In this article, we’ll explore how to configure S3 as a destination for failed events when invoking an AWS Lambda function asynchronously.
Let's look at the architecture diagram!
Review Lambda Function - Hello from Lambda!
The primary purpose of this article is to demonstrate how destinations work for AWS Lambda functions, with a specific focus on configuring an S3 bucket as a destination for failed events. To keep it simple, the Lambda function will implement a basic "Hello, World" example that returns a "Hello from Lambda!" message. This baseline function will serve as the foundation for demonstrating the destination feature.
For this walkthrough, I will use the AWS Management Console to create the function and then add the destination configuration.
Navigate to the AWS Management Console for Lambda functions and click Create Function.
Create a function named HelloWorldGB as shown below.
For this function, I am using Python Runtime 3.13.
Example code is shown in the screenshot below.
This view is from the updated Lambda code editor. If you’re unfamiliar with this console, check out my video that explains how to build a Lambda function using the new editor and SAM templates. The link to the video is in the description.
Add a destination - S3 bucket
From the Lambda console, add a destination. This destination will be configured to store events on failure. Note that destinations can be configured for both On Success and On Failure events.
For this use case, I am only configuring it for failure scenarios.
In the dropdown menu, you will see various destination options, including:
- SNS
- SQS
- Another Lambda function
- EventBridge
- S3 Bucket
For this example, I will select S3 Bucket as the destination.
Make sure that the Lambda function has the necessary permissions to write objects to the S3 bucket. Without this permission, you will encounter an error when the function tries to write events to the destination bucket.
Add code to raise exception in Lambda
Since the destination is configured for failure scenarios, it will only log events when the Lambda function encounters a failure. To test this, I will update the code to intentionally raise an exception, ensuring the function fails and generates a qualifying event.
Invoke the Lambda function after making this change. Due to the forced exception, you should see an error message, as shown in the screenshot below.
Review CloudWatch Log
Next, review the CloudWatch logs. Since the Lambda function has been invoked, a CloudWatch log group will be created. The logs will contain details about the invocation, including the error.
Review Destination S3 Bucket
Navigate to the S3 bucket that you configured as the destination and check whether the failed event has been posted there.
As you can see, while the lambda function got invoked and generated a failure, the failed event is not logged in the destination s3 bucket.
Reason for this is that destination function only works when Lambda function is invoked asynchronously. It will not work when function is being invoked synchronously.
Invoke Lambda using AWS CLI
A Lambda function can be invoked asynchronously through various AWS service integrations, such as EventBridge or S3. In the example command below, I demonstrate how to invoke it asynchronously using an AWS CLI command:
`aws lambda invoke \
--function-name arn:aws:lambda:us-east-1:24xxxxxxxx7:function:HelloWorldGB \
--invocation-type Event \
--cli-binary-format raw-in-base64-out \
--payload '{ "name": "GB" }' \
--region us-east-1 \
response.Json
Result/response
{
"StatusCode": 202
}`
Review the Destination S3 Bucket Again
Now that the result shows a status code of 202, it indicates that the asynchronous call to Lambda was successful. I will now return to the destination S3 bucket to review the results.
As shown in the S3 bucket screenshot below, the destination bucket now contains the logged failed event. This event can be further analyzed to identify and understand the cause of the failure.
Cleanup - Delete the Lambda function & S3 bucket
Once you have completed the setup, ensure you delete the Lambda function to avoid unnecessary resource usage. Additionally, delete the S3 bucket and CloudWatch log group. If you created any new roles during the process, remember to delete those as well.
Conclusion
In this article, I demonstrated how to configure S3 as a destination for a Lambda function.
Additionally, I showcased how to invoke the lambda function asynchronously using AWS CLI command.
I hope you found this article helpful and informative!
Thank you for reading!
Watch the video here:
https://www.youtube.com/watch?v=QqZspZoOYsY
𝒢𝒾𝓇𝒾𝓈𝒽 ℬ𝒽𝒶𝓉𝒾𝒶
𝘈𝘞𝘚 𝘊𝘦𝘳𝘵𝘪𝘧𝘪𝘦𝘥 𝘚𝘰𝘭𝘶𝘵𝘪𝘰𝘯 𝘈𝘳𝘤𝘩𝘪𝘵𝘦𝘤𝘵 & 𝘋𝘦𝘷𝘦𝘭𝘰𝘱𝘦𝘳 𝘈𝘴𝘴𝘰𝘤𝘪𝘢𝘵𝘦
𝘊𝘭𝘰𝘶𝘥 𝘛𝘦𝘤𝘩𝘯𝘰𝘭𝘰𝘨𝘺 𝘌𝘯𝘵𝘩𝘶𝘴𝘪𝘢𝘴𝘵
Top comments (0)