AWS SnapStart - Part 25 Measuring cold and warm starts with Java 21 using Lambda layer (2)

Introduction

In the blog post How to create, publish and use layers for Java 21 Lambda functions we explained how to publish our first Lambda layer with Java 21. In the article Measuring cold and warm starts with Java 21 using Lambda layer (1) we created the application using this Lambda layer and then measured cold and warm start times without SnapStart enabled, with SnapStart enabled and also applied DynamoDB invocation priming optimization and compare the results with our measurements without using the Lambda layers and providing all dependencies in the POM file which we did in the article Measuring cold and warm starts with Java 21 using different Lambda memory settings. In this article we'll create another Lambda layer will all dependencies and use this layer in our application, make the same measurements and compare the result with the previous experiment.

Measuring cold and warm starts with Java 21 using Lambda layer with all dependencies

For the sake of exploration we'll use the sample Lambda layer for creating the Lambda layer with Java 21 runtime packaging the all dependencies into the layer :

• dynamodb
• lambda
• apache-client
• aws-lambda-java-core
• aws-lambda-java-events
• org-crac
• slf4j-simple
• jackson-dataformat-xml

We'll also use the sample application. There are basically 2 Lambda functions defined in the AWS SAM template which both respond to the API Gateway requests and retrieve product by id received from the API Gateway from DynamoDB. One Lambda function GetProductByIdWithPureJava21LambdaWithAllLayer can be used with and without SnapStart and the second one GetProductByIdWithPureJava21LambdaAndPrimingWithAllLayer uses SnapStart and DynamoDB request invocation priming.

In order to use the Lambda layer with all dependencies created previously for the Lambda functions in the AWS SAM template we have to add Layers parameter to the Lambda function like this:

    Type: AWS::Serverless::Function
    Properties:
      FunctionName: GetProductByIdWithPureJava21LambdaWithAllLayer
      AutoPublishAlias: liveVersion
      Layers:
        - !Sub arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:layer:aws-pure-java-21-common-lambda-layer:1
      Handler: software.amazonaws.example.product.handler.GetProductByIdHandler::handleRequest

Please replace the Layer ARN (including the version) with your own which is the output if the publish layer command (aws lambda publish-layer-version).

In pom.xml you see all dependencies with the scope provided (by the Lambda layer attached).

The results of the experiment below were based on reproducing more than 100 cold and approximately 100.000 warm starts with experiment which ran for approximately 1 hour. For it (and experiments from my previous article) I used the load test tool hey, but you can use whatever tool you want, like Serverless-artillery or Postman.
I ran all these experiments by giving our Lambda functions 1024 MB memory and by passing the following compilation option via the environment variable: JAVA_TOOL_OPTIONS: "-XX:+TieredCompilation -XX:TieredStopAtLevel=1" (client compilation without profiling).

In the tables below I'll also provide the results with our measurements without using the Lambda layers (and providing all dependencies in the POM file) which we did in the article Measuring cold and warm starts with Java 21 using different Lambda memory settings and the measurements when using common Lambda layer to directly have the comparison.
Abbreviation c is for the cold start and w is for the warm start.

Cold (c) and warm (w) start times without SnapStart in ms:

Experiment	c p50	c p75	c p90	c p99	c p99.9	c max	w p50	w p75	w p90	w p99	w p99.9	w max
with all dependencies Lambda Layer	2824.33	2884.24	2963.14	3324.07	3622.44	3625.58	5.50	6.20	7.16	15.50	46.19	1278.41
with common Lambda Layer	3497.91	3597.18	3695.58	3800.47	3908.33	4011.71	5.82	6.72	8.00	17.97	55.48	1709.13
w/o Lambda Layer	3157.6	3213.85	3270.8	3428.2	3601.12	3725.02	5.77	6.50	7.81	20.65	90.20	1423.63

Cold (c) and warm (w) start times with SnapStart without Priming in ms:

Experiment	c p50	c p75	c p90	c p99	c p99.9	c max	w p50	w p75	w p90	w p99	w p99.9	w max
with all dependencies Lambda Layer	1706.64	1767.40	1893.59	2314.91	2646.68	2647.33	5.59	6.25	7.21	15.75	48.06	1403.71
with common Lambda Layer	2047.12	2124.24	2439.49	2705.52	2735.43	2831.59	5.68	6.40	7.45	17.06	48.45	2139.74
w/o Lambda Layer	1626.69	1741.10	2040.99	2219.75	2319.54	2321.64	5.64	6.41	7.87	21.40	99.81	1355.09

Cold (c) and warm (w) start times with SnapStart and with DynamoDB invocation Priming in ms:

Experiment	c p50	c p75	c p90	c p99	c p99.9	c max	w p50	w p75	w p90	w p99	w p99.9	w max
with all dependencies Lambda Layer	747.47	786.56	932.23	1099.38	1666.18	1666.62	5.42	5.91	7.39	16.39	45.09	574.61
with common Lambda Layer	713.88	766.38	1141.94	1181.41	1214.94	1215.32	5.59	6.30	7.39	16.39	45.09	574.61
w/o Lambda Layer	702.55	759.52	1038.50	1169.66	1179.05	1179.36	5.73	6.51	7.87	21.75	92.19	328.41

Conclusion

In this article we created the application using the Lambda layer with all dependencies and then measured cold and warm start times without SnapStart enabled, with SnapStart enabled and also applied DynamoDB invocation priming optimization and compared the results with our measurements without using the Lambda layers (and providing all dependencies in the POM file) which we did in the article Measuring cold and warm starts with Java 21 using different Lambda memory settings and using common Lambda layer.

Even if I had some deviations in the results but the trend was always the same after multiple measurements using the Lambda layer with all dependencies:

• When not enabling SnapStart the cold start with this Lambda layer was up to several hundred milliseconds lower than without usage the Lambda layer for percentiles up to p90 (which surprised me).
• When enabling SnapStart but not using priming of the DynamoDB request the cold starts varied in favor of the Lambda layer with all dependencies or in favor of the Lambda which doesn’t use Lambda layers at all depending on the percentile.
• When using SnapStart with priming of the DynamoDB request the cold starts were close for both use cases for percentiles up to p99 and then higher for p99.9 and above when using the Lambda layer with all dependencies. So, the results varied a lot for each use case. What I always observed is that the result’s range was quite big (800-900ms difference between p50 and max values) for all measurements when using the Lambda layer with all dependencies and the same range was much lower when not using Lambda layers at all.
• When comparing the measurements in this article with measurements with common Lambda layer we observe we got lower cold starts for using all dependencies in the Lambda layers for the first 2 experiments (no SnapStart enabled and SnapStart enabled but no priming applied) and vice when SnapStart is enabled and DynamoDB invocation is primed.
• The warm starts/execution times of the Lambda function were quite close for all use cases.

So the usage of the Lambda layers (depending what you put there and what you ship as a dependency in your application) adds some unpredictability and you should always do your own measurements!