Over the past two decades, various infrastructure trends have emerged and been replaced by new ones every few years. Initially, we relied on bare metal servers housed in company basements, requiring us to purchase hardware, replace failed hard drives, and manage power outages. With the advent of virtualization and cloud providers, we shifted our focus primarily to software management, though we still had to patch operating systems. Containers further reduced maintenance and enhanced scalability, but we remained responsible for managing the runtime environment. The serverless approach has taken this a step further, allowing us to concentrate solely on our application code and dependencies, while the cloud provider handled the rest.
AWS Lambda is turning 10
At re:Invent 2014, AWS introduced Lambda, marking the beginning of the serverless infrastructure paradigm. While many AWS services, such as SQS and S3, have been serverless from the start, Lambda catalyzed this architectural shift and popularized the term “serverless.” Over the past decade, the ecosystem has matured significantly and has become mainstream. New startups, including ours, are proudly building their entire infrastructure on serverless components.
A decade is a long time in technology, and innovation is relentless. It was inevitable that a new trend would emerge, and it seems to me that it has. We just don’t have a buzzword for it yet. Before we can start speculating about what comes after serverless, let's discuss what a modern application running on AWS looks like in 2024.
Modern Application Stack
From a simplified perspective, most web and mobile applications share fundamental similarities on the backend. They typically provide authentication, accept various forms of user input, send that information via APIs, and store it in a database. This data is then transformed or aggregated and presented back to the user.
This pattern is evident in e-commerce, CRM systems, internet banking, bookkeeping, and many other applications. Despite these commonalities, each piece of software has unique features that add value for users. Additionally, elements like machine learning models, integrations with other systems, and cryptocurrency transactions further enhance the value of modern applications. However, without the basic components of user input, APIs, and databases, these applications would be largely ineffective.
When building a new application today, you might leverage Lambda for your event-driven (EDA) microservices (Authenticator, API, Notifier, …). Combined with DynamoDB, these choices are mainstream and safe for a modern cloud-based system.
But what if there is an alternative?
Lambda Alternatives
Fortunately, we no longer need to develop all these microservices ourselves as AWS provides serverless solutions for many common scenarios:
- Authenticator - Cognito
- API - API Gateway, AppSync
- Static Hosting - S3 + CloudFront
- Async jobs - StepFunctions
- Database - DynamoDB
- …
Creating these services independently can easily consume over 50% of your project time. This effort, known as “Undifferentiated Heavy Lifting,” is essential but doesn’t set your application apart from competitors. Leveraging existing services allows you to concentrate resources on areas where your application delivers unique value.
Real Application Use Case
Imagine a truly serverless application constructed solely by integrating existing AWS services, even going so far as to omit a Lambda function. By "truly serverless," I refer to the original serverless manifesto, which advocates for resilience, pay-per-use pricing, the ability to scale to zero, and more.
Whether you opt for a REST API or GraphQL, the architectural variations may differ slightly, but the core concepts remain consistent. Thanks to API Gateway's ability to build custom HTTP requests using the VTL templating language, you can directly call services like DynamoDB to resolve complex queries, retrieve data, and create or update records. The OpenAPI schema defines validation rules for each endpoint, while integration with Cognito manages permissions and security.
AppSync offers a similar experience but with a few notable enhancements. It includes JavaScript resolvers in addition to standard VTL and Pipeline Resolvers for executing a sequence of steps in a specific GraphQL query or mutation. AppSync also supports GraphQL subscriptions, enabling real-time communication with your frontend over WebSockets. While similar to API Gateway’s capabilities, AppSync provides easier and more comprehensive WebSocket access compared to REST API implementations. Moreover, all AWS services have an HTTP API, allowing you to leverage services like AWS Rekognition for object detection in images or Bedrock for LLM prompting.
Asynchronous data processing is a common application requirement. Typically, this would involve a Lambda function consuming messages from an SQS queue or EventBridge. However, in our case, we'll use Step Functions to manage tasks such as chaining DynamoDB calls, executing parallel API calls to S3 or sending notifications to the user via SNS, and more.
Step Functions, combined with API Gateway or AppSync, can also handle synchronous jobs, but you get the idea by now, right? Lambda functions are great but in many cases, you can achieve the same result without them just by “gluing together” existing AWS services.
We can live without Lambda, but why should we?
As you can see we can get quite far by this approach. To be honest, from my experience of trying to build Lambdaless applications for several years already, I have to say it is not the most pleasant developer experience. Thankfully, it gets better with every single re:Invent. But why would one do all that hustle if writing a simple Lambda function can get you the same result without the headache?
I'm glad you asked. I'm not suggesting you abandon using Lambdas where they make sense. Instead, I'm highlighting an alternative worth considering when designing new infrastructure. Lambdas enables the use of familiar technologies, libraries, and frameworks, likely getting you to the desired result faster. However, this speed comes with long-term maintenance costs.
Lambda runtimes are regularly updated and deprecated, libraries and frameworks receive security patches and improvements, and the tech stack evolves rapidly. Keeping everything up-to-date, especially in complex architectures, is a significant task. Yet, many internal applications are developed once for a specific use case and expected to operate with minimal maintenance for years, leading to technical debt - a costly and cumbersome issue.
AWS services, in contrast, have stable APIs with minimal breaking changes. If a security issue arises, AWS typically addresses it faster than individual developers could manage for a library or framework. As a side effect, you end up configuring existing AWS services for most of your applications rather than programming in the traditional way. While becoming a YAML developer might seem less exciting, maintaining configuration is much easier over time than managing a Python or JavaScript project.
By "gluing together" AWS services, you improve operations predictability, uptime, and reliability. Native services handle traffic well, and scalability issues are more often architectural than technological. Additionally, your operation costs align closely with actual usage, which businesses appreciate. While Lambdas are quick to deploy, they can be costly to maintain over time. Relying solely on AWS services may be more expensive to develop initially but is cheaper to operate in the long run. The benefits are great. The question is whether you are willing to exchange the safety and familiarity of standard programming languages for significantly lower maintenance.
Back to reality
It’s very likely that you will occasionally need a Lambda function or even a container, and that's perfectly fine. When serverless concepts emerged almost a decade ago, it seemed unlikely that fully serverless systems would become a reality. Yet today, even healthcare and financial sectors rely significantly on serverless architectures, and startups are building fully serverless applications.
I strongly believe that with the constantly improving developer experience around AWS services, the need to write our own Lambdas for undifferentiated heavy-lifting tasks will diminish. Instead, we will integrate these ready-made "LEGO blocks" as needed for the majority of the use cases with significantly lower maintenance costs and increased reliability at the same time. This will enable us to quickly prototype applications without requiring major refactorings later on. I encourage you to give these ideas a chance and carefully consider whether you truly need your next Lambda function.
So how about the buzzword?
Backendless, Backend as a Service, and Backend as Configuration are a few of my own suggestions that, while not perfect, might spark some inspiration. If you come across a catchy buzzword for this architectural style, please share it in the comments.
Top comments (0)