I recently had to provide some numbers on AWS Lambda cold starts. These aren’t readily available in the AWS console. In fact, I couldn’t find a way anywhere online. There are lots of articles about AWS Lambda cold starts, how to avoid them, but none on how to measure them (without code changes). Here’s at least one way you can list the amount and duration of your cold starts.
What Is a Cold Start?
AWS Lambda functions are (intended to be) small pieces of code that are executed on demand. The idea is to have multiple functions work together to form the application you are trying to create.
This has some consequences. For example, you can’t just share state in memory between functions. Rather, you have to give a function what it needs by providing it with the call to that function, or by putting the data somewhere (a database for example).
Another consequence of Lambda functions is that you can’t expect the function to keep on “living” like a traditional Windows service or Unix daemon. AWS decides when to kill the Lambda function.
When there is no instance of your Lambda running, or all running instances are busy, AWS may decide to start up a new instance. This is a cold start. A cold start can require more work than just invoking an instance that is already running: packages must be loaded, data could be fetched, initialization code should be run, etc.
Identifying Cold Starts By Logging
One way to identify cold starts I’ve seen online is to add a logging statement outside the handler of your function:
console.log("This is a cold start");
function handler() {
console.log("This will be logged on every invocation");
}
When AWS Lambda starts an instance, it will run the first line. But after that, the instance stays alive and the first line isn’t executed anymore. Every invocation of your Lambda just calls the handler function and so only the second “console.log” is called.
This is one way of identifying cold starts, but it only works if you’ve already put this method in your function. If you want to see the cold starts of a function that doesn’t have this mechanism included, this technique doesn’t work. Neither does it show you the duration of the cold start.
Using CloudWatch
When AWS Lambda starts a new Lambda instance, it also starts a new log stream. This log stream is tied to this Lambda instance. So we could use this (unofficial) knowledge to find the amount of cold starts.
I was looking for the cold starts in a certain month. The log stream name starts with a year and a month (e.g. “2019/05”) so we can use that to find the relevant streams.
And lucky for us, each invocation includes a log message added by AWS where it states the duration of the invocation. Something that looks like this:
REPORT RequestId: 0858f0d2-8e67-4150-b705-7c1b4329192f
Duration: 177.70 ms Billed Duration: 200 ms Memory Size: 2048 MB
Max Memory Used: 141 MB
We can use this to extract the duration of the cold start.
In short, we can:
- find all relevant log streams (these are all the cold starts)
- find the first REPORT message in each stream
- extract the duration of this REPORT message
This leads to the following code that you can use to find the amount and duration of your AWS Lambda cold starts. It writes the results to a CSV file:
You can also check out the StackOverflow question I originally asked.
A word of warning: I believe the log stream per Lambda cold start is undocumented or at least unofficial.
Another word of warning: I ran this script for the data of 2 months, and it took more than 12 hours (for almost 60000 cold starts). It can probably be improved, but if you have a large amount of cold starts, this script could take some time.
What To Do With This Data?
Should you do anything with this data? Is it worth reducing the amount of cold starts you have? It really depends on how your cold starts compare to your normal invocations. That is data you can find in the AWS console. If the amount of cold starts pales in comparison with the amount of invocations, or if they’re not much slower than normal invocations, you could just leave it as it is. If not, search around for some techniques to improve your cold start time.
At least with the above technique, you can now get some numbers on your AWS Lambda cold starts.
Top comments (1)
You can use the built-in AWS tools like CloudWatch or AWS X-Ray, but you will quickly see that it doesn’t give you everything you need to debug real-world issues quickly. I recommend using a third-party product such as Lumigo. Lumigo is Effortless AWS Lambda Monitoring. Here are its main features:
Find & Fix Issues in Seconds with Visual Debugging
Automatic Distributed Tracing
Identify & Remove Performance Bottlenecks
Serverless-Specific Smart Alerts
visualizes your entire serverless stack,
including all your favorite services
And many more!
You can try the free tier plan, and I'm sure you'll fall in love with it. Good luck!