The Development Landscape Is Ever Changing
In recent years there have been huge shifts in both back-end and front-end development. As clients become more and more stateful, the services backing them become slimmer and stateless. The complexity of managing all of these tiny services and functions is being abstracted away, allowing those without devops skills to take advantage of serverless technologies. As the clients grew thicker, so did the complexity of managing state, and interactions with any number of backing services or APIs. Tooling and frameworks have started to emerge, abstracting this away from the developer, allowing them to focus on building product quickly.
So, why am I bringing this up? Because I feel like all of this change is culminating in a new way of building apps. One that takes advantage of the changes we’ve seen over the last few years, both on the front-end and the back-end, and allows developers to build really interesting applications on top of serviceful services in very little time.
Serviceful Serverless
On the back-end we've seen compute power become a commodity, sparking the evolution of serverless. However, if you have ever built anything in a truly serverless fashion, then you know that the scalability and cost savings you get come with a heaping pile of complexity.
If you thought managing microservices was bad, try breaking all the functions in one of them out into their own locations and gluing them back together through events. 😱
While serverless definitely has it's advantages it's not for the feint of heart. It's not easy to get right and often requires months of work from a team of back-end developers. Ben Kehoe covers the WHYs of serverless better than I could in the linked post, but suffice to say, you pick serverless for the value of focus.
At the end of the day we want to create value quickly. Hopefully not at the expense of things like developer experience and security, but we do want to write as little code as possible to solve a business problem. Maybe even none...
How can leverage the benefits of serverless without having to manage that new complexity ourselves? Well, generally as a given technology evolves, it goes through a shift in complexity. While serverless itself consists of lambda functions, topic configuration, and event management, there have been some higher level abstractions built on top of it that make it actually quite friendly to those who may consider themselves front-end or product developers.
This in turn brought about a bunch of serviceful serverless products which take advantage of the scalability of serverless, but abstract it away behind a meaningful product that solves a particular problem domain. Nader Dabit lists some great examples of serviceful services in the post I linked earlier in the paragraph. In his article he says:
A few examples of serviceful services include Auth0 / Amazon Cognito (managed authentication), Algolia (managed search), Contentful (content infrastructure), AWS AppSync / Cloud Firestore (managed API services), Amazon Lex / Rekognition / Textract (machine learning services), and Cloudinary (managed image & video hosting service). - Nader Dabit, Full-Stack Development in the Era of Serverless Computing
UI Toolkits and Frameworks
On the front-end we've also seen a large increase in complexity, borrowing from Nader Dabit again, he explains what has changed, making things so much more complex:
Because of the rise of SPAs, more complex data concerns, multiple device targets and increased expectations of user experience, client-side development has gotten more complex over the past decade or so. - Nader Dabit, Full-Stack Development in the Era of Serverless Computing
However, with this complexity came some really amazing tooling and frameworks. These abstractions make it easier to manage the new complexity, allowing developers to focus on creating value. The best part is, we don't have to do it at the expense of things like accessibility and security. We get this baked right into the solutions.
It's pretty fair to say that these tools are providing focus. They allow you to deal strictly with business problems and handle all the technical problems for you. This rings of the same changes on the back-end and the evolution of serverless computing.
Frameworks like Expo, Ionic, Gatsby, and Meteor are making it possible to build production ready, quality apps, based on their building blocks.
The Best of Both Worlds
What we're seeing now is platforms that encompass the best of both worlds. Providing solutions for the entire development life cycle of an application. Making it much easier to build fully fledged applications that can scale without question and costs less than building out your own serverless infrastructure and tooling. Solutions like Firebase, Netlify, Zeit, and AWS Amplify provide full-stack features such as toolkits and APIs, data storage, file storage, and build and deployment services. I refer to these types of platforms as Full-stack Serverless. They provide feature sets that generally span from the command line tooling to back-end services, and everything in between.
I recently ran a poll on Twitter where I asked, "When architecting a new application, what do you consider to be the most crucial concern when choosing solutions?"
What wasn't very surprising is that most people voted for "Maintenance and Longevity" with "Creating Value Quickly" coming in at second. It's not surprising to see maintenance and longevity at the top because when you are responsible for your own server or making sure your application is secure and performant, there is a lot of extra technical problem code. Code that solves technology problems and not business problems.
This is why I feel the future is one in which we don't handle maintenance and longevity of services or applications because we offset that burden to serviceful services. Doing so let's you focus on building product which leads to creating value quickly.
In the poll developer experience came in right behind delivering value, which is interesting because I feel that in order for solutions to operate in this space, developer experience is everything. A lot of people fear vendor lock in... unless what they are using provides such a pleasant developer experience that they trust you get what they need and are in fact concerned with their success.
When you opt-in it's because you feel like the platform is increasing your productivity enough that it wouldn't make sense not to use it. You're one of the ones who seeks to create value quickly and focus on building your app, not building out your infrastructure. This is the bucket I fall in, it's why I'm always gushing about solutions like Gatsby and Expo, I want to build product immediately and know that the rest is handled for me, now you're telling me I no longer have to concern myself with choosing how to handle data, auth, users, and content. Sign. Me. Up.
I think previously there was always a concern with scale which made buying into these full-stack solutions difficult, but with the movement of serverless, I see this idea as a lot more approachable. I think we're in the very early stages of this new era of full-stack serverless platforms and I'm pretty excited to see where things go.
Top comments (10)
It seems like a lot of this 'serverless' stuff is an excuse for a cloud hosting provider to lock you into their service. There are a lot of proprietary implementations.
There is also a trend of breaking up web application elements into tons of tiny units, which means more maintenance work and thinking about how all the components interoperate.
Admittedly i don't understand the appeal or advantages of this stuff, other than some potential cost savings, but that comes at the expense of a more complicated architecture to manage, and locking yourself further into a provider.
I think it would much be better to write code in a performance oriented language instead, if saving resources is the goal.
I don't think it's just about saving resources and squeezing more performance out of your services. It's about focus on solving business problems over solving technical problems. All of those things you mentioned sound like something that ideally, I wouldn't have to do. I like to create value quickly and spending time dealing with performance issues and and maintenance seems like a waste of valuable time.
Unless you're hosting your own servers, you're already partly locked in, no? If you store your data on a cloud provider then why not also use the higher level abstractions they provide. If you do find issues with them, at the end of the day you still have your low level services and are free to build whatever you want on top of that (for most of the cloud providers anyway).
I hate to say it, but i really don't understand the business case for serverless architecture at all. There are so many good tools for orchestrating servers and dynamically scaling them. I don't understand why they need to be taken out of the picture, unless there is some cost saving involved.
Having a server, even on AWS, doesn't lock you in very much. I can grab my configs and plop them on another server, change the DNS, and get back to rolling in less than a few hours. The only 'gotchas' i can think of is that AWS instances handle certificate generation and have a nice firewall configuration system, but this can all be handled at the command line on any linux box.
If you're dependent on the high level stuff and the provider you're moving to doesn't provide feature X or have any import facility, you're stuck. Which is exactly what your hosting provider would prefer.
Amazon keeps adding more and more of these proprietary things. It reminds me of how Microsoft managed to lock people into their technologies for a long time.
I think it's too early to say much about this.
Maybe the serverless people are right, and serverless tech helps people to build better systems without the need for extra DevOps folk and these small companies will overtake the bigger ones who have pay for and manage their giant IT teams.
Maybe containers and VMs will win because serverless was a false promise and you have to have these giant expensive teams to manage those systems.
I have the feeling the stuff you are talking about, orchestration and scaling, will move into the cloud and people who do their own stuff in-house will burn money and time and lose in the long run to the people who have more money and time left because they didn't do it.
But that's just my opinion and it can be wrong.
You know, i run a pair of servers for one company i also do web development for. We got away from managed servers because it was just too expensive.
I might spend an hour a month on average tinkering on these servers. And both of these servers have to be PCI compliant, so there is a decent amount of work to do, and i have to keep up with security.
This has been my responsibility for over 7 years.
I imagine that when you write an app to work "serverless", you have to write it around the idea of a serverless architecture. It may be worse than that, and that you have to target amazon or microsoft's architecture.
In either case, you are potentially backing yourself into a corner when you develop a new app.
Yes, someone has to be the test hamster for these new technologies.. i just can't see a good reason to tell any of my clients that we should go this way.
I don't think the serverless fad will last long as currently incarnated on AWS. It is too freaking slow for many tasks to have a lambda function that potentially needs an environment spun up (cold start). If the function has enough concurrency it is guaranteed there will be cold starts. If it has too much concurrency then it may just fail because you went over some provider determined quota.
In comparison I can and have created servers that could stand up to 1000 hits/sec per process. And every single one of them responding quicker than with lambda functions, even ones in warm start.
On top of this there is quite a bit of new tooling and deployment dictating best coding practices and modularization within the software development environment. That is usually considered a bad idea. Many of the frameworks are geared really to one language and don't do too well with others. For instance serverless framework is written in node and doesn't do a great job for python based lambda functions that have dependencies on other projects that may be local to an organization. It has no concept of or support for a python virtualenv.
And how the heck do a reason about a pure serverless backend whose each component can take anywhere from 50 ms in overhead latency to many seconds on any random invocation that hits it? I don't know how to do this.
Seems like you're talking about FaaS and not serverless in general.
I know Lambda is seen as the very incarnation of serverless, but actually it's just a tiny part of it and many services have serverless properties (S3, API-Gateway, DynamoDB, AppSync, Cognito, etc.) and those usually don't have the latency problems you mention.
What I am talking about is the serverless hype that everything should be serverless in the app stack. That is clearly highly problematic. For a fuller read on this I recommend theregister.co.uk/2018/12/19/serve...
which points to a good study from UCB at www2.eecs.berkeley.edu/Pubs/TechRp....
I am not at all against these various services. What I am against is yet another set of baseless solve everything silver bullet claims. I have seen organizations go so overboard on "no servers of our own" that they produce much more complicated and inflexible stacks requiring understanding of a dozen different technologies or so just for the plumbing bits and not having to do much in house. This on a project that would have been fairly trivial to do most of in a more conventional manner at lower money, time, aggravation, lost opportunity cost. They are not alone.
It's just a quesion of risk distribution.
If you think it's harder to learn these technologies and easier and more cost efficient to manage your own infra, could be that you're right. I know many people who think that way.
This is the best way to describe why Serverless? Well put.